From owner-freebsd-arch Sun Feb 17 0:13:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mass.dis.org (dhcp45-21.dis.org [216.240.45.21]) by hub.freebsd.org (Postfix) with ESMTP id D455237B419 for ; Sun, 17 Feb 2002 00:13:24 -0800 (PST) Received: from mass.dis.org (localhost [127.0.0.1]) by mass.dis.org (8.11.6/8.11.6) with ESMTP id g1H8DAV01750; Sun, 17 Feb 2002 00:13:16 -0800 (PST) (envelope-from msmith@mass.dis.org) Message-Id: <200202170813.g1H8DAV01750@mass.dis.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: "mike varga" Cc: freebsd-arch@freebsd.org Subject: Re: reverse vtophys() In-reply-to: Your message of "Fri, 15 Feb 2002 12:06:57 PST." <003701c1b65c$53138840$b210a8c0@netscreen5> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 17 Feb 2002 00:13:10 -0800 From: Michael Smith Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > Is there a function that exists=20 > that converts a physical address to=20 > a process's virtual address? No; the concept isn't valid. - A given physical address may be mapped in more than one virtual location within the process' address space. - There is an unavoidable race with unwired memory where a physical page may "move" as it is used to back more than a single physical page, resulting in a bad answer. Perhaps you should explain what it is that you're trying to do, rather than narrowing down to what you think is the solution... -- To announce that there must be no criticism of the president, or that we are to stand by the president, right or wrong, is not only unpatriotic and servile, but is morally treasonable to the American public. - Theodore Roosevelt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 0:18:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 2B1B037B400; Sun, 17 Feb 2002 00:18:13 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1H8ID067573; Sun, 17 Feb 2002 00:18:13 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 00:18:13 -0800 (PST) From: Matthew Dillon Message-Id: <200202170818.g1H8ID067573@apollo.backplane.com> To: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: buildworld comparison stable vs current Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Here are the results of a buildworld test building the -current source tree on a SMP stable box and on a SMP current box. There was some interest at BSDCon for a comparison just to see where we were. Everything matches up except time and context switches, the context switches I presume is because interrupts require context switches in -current. The time difference is 1800 seconds (stable) verses 2219 seconds (current), making stable 1.23 times faster then current at the moment. I consider this a fairly good number for where we are, and to be expected considering the lack of optimizations in current. -Matt (note: source is NFS mounted. /usr/obj is local disk). buildworld -j 10 of current on an SMP stable box: 1800.50 real 1445.93 user 760.32 sys 25860 maximum resident set size 971 average shared memory size 1164 average unshared data size 130 average unshared stack size 11062987 page reclaims 297 page faults 0 swaps 2013 block input operations 5649 block output operations 1540227 messages sent 1530792 messages received 6 signals received 2467550 voluntary context switches 1357575 involuntary context switches buildworld -j 10 of current on an SMP current box (same hardware specs): 2219.13 real 1796.37 user 968.04 sys 25884 maximum resident set size 912 average shared memory size 1045 average unshared data size 126 average unshared stack size 11677133 page reclaims 537 page faults 0 swaps 8438 block input operations 6774 block output operations 1439793 messages sent 1439774 messages received 6 signals received 28670930 voluntary context switches 2836255 involuntary context switches To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 0:20:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id CFA5137B405; Sun, 17 Feb 2002 00:20:11 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020217082011.RGEQ2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Sun, 17 Feb 2002 08:20:11 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id AAA44788; Sun, 17 Feb 2002 00:02:15 -0800 (PST) Date: Sun, 17 Feb 2002 00:02:14 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: Alfred Perlstein , Poul-Henning Kamp , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday() In-Reply-To: <200202170728.g1H7SV651230@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG What speed do you get with NO INVARIANTS and no WITNESS? p.s what does TG stand for? (I need to look back to see what the 4.4 NON SMP speed was...) On Sat, 16 Feb 2002, Matthew Dillon wrote: > :> I would like to see the invariants sections simply removed. > :> > :> I like to run with INVARIANTS set, even on production systems, because > :> I like the extra assertions it makes. > : > :I agree but jhb wants to have it that way... He wants to be able to catch > :anyone accessing the ucred of a thread that is in user space. I > :personally think we should just remove the code. > > Another problem with this is that most -current developers are (I assume) > running with INVARIANTS (because they'd be fools not to). This means > that the code you just comitted is not going to be exercised at all > due to the crfree()'s. > > --- > > In anycase, your code plus #if 0'ing the INVARIANTS and the KASSERT's > that were checking td_ucred == NULL results in a significant improvement > in syscall performance. I am now getting: > > CURRENT 2xCPU SMP BUILD W/Julian's code and ucred invariants blasted to bits > and Matt's mutex pool code and > removal of Giant for gettimeofday(). > one TG running: 462478/sec > two TGs running: 361962/sec per TG > > > prior tests: > > CURRENT 2xCPU SMP BUILD Original gettimeofday() code > one TG running: 350000/sec > two TGs running: 55000/sec per TG (no, that isn't a type-o) > CURRENT 2xCPU SMP BUILD W/Giant removed from gettimeofday() > one TG running: 375000/sec > two TGs running: 301000/sec per TG (due to ucred Giant issues) > CURRENT, gettimeofday() SMP patch, Matt's original ucred patch: > 1 TG running: 396365 calls/sec > 2 TGs running: 322000 calls/sec each. > > -Matt > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 0:30: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 3436D37B402; Sun, 17 Feb 2002 00:30:01 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1H8TME68268; Sun, 17 Feb 2002 00:29:22 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 00:29:22 -0800 (PST) From: Matthew Dillon Message-Id: <200202170829.g1H8TME68268@apollo.backplane.com> To: Julian Elischer Cc: Alfred Perlstein , Poul-Henning Kamp , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday() References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :either way, could you do you timing tests on a NON-INVARIANT :kernel to judge the difference this makes? :In the mean while, John, is it really necessary to have this there? :I can remove it (and the KASSERTS) in a flash if you'll let me... Sure, here they are... actually fairly significant, I didn't expect complete removal of invariants to have this much of an effect. I fear someone has done something with invariants that's eating more cpu then it should :-( As a bonus, I've also included the patch set I am using against the current -current which identifies the KASSERTs and INVARIANTS I removed to make your code work as advertised with INVARIANTS turned on. This patch set is fairly well tested. I did not include the patch for removing Giant from gettimeofday() (I'll be comitting the gettimeofday giant removal patch on sunday). I'll also re-run the buildworld test with invariants turned off and post a followup to my buildworld posting tomorrow. -Matt (matt's mutex pool adjustments and julian's initial commit now part of current) CURRENT 2xCPU SMP BUILD and removal of Giant for gettimeofday() INVARIANTS TURNED OFF GLOBALLY. one TG running: 576089/sec two TGs running: 429141/sec per TG CURRENT 2xCPU SMP BUILD and ucred invariants blasted to bits and removal of Giant for gettimeofday(). INVARIANTS TURNED ON GLOBALLY one TG running: 462478/sec two TGs running: 361962/sec per TG Index: i386/i386/trap.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/trap.c,v retrieving revision 1.212 diff -u -r1.212 trap.c --- i386/i386/trap.c 17 Feb 2002 01:09:54 -0000 1.212 +++ i386/i386/trap.c 17 Feb 2002 08:19:55 -0000 @@ -255,7 +255,9 @@ sticks = td->td_kse->ke_sticks; td->td_frame = &frame; +#if 0 KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#endif if (td->td_ucred != p->p_ucred) cred_update_thread(td); @@ -643,7 +645,7 @@ userret(td, &frame, sticks); mtx_assert(&Giant, MA_NOTOWNED); userout: -#ifdef INVARIANTS +#if 0 mtx_lock(&Giant); crfree(td->td_ucred); mtx_unlock(&Giant); @@ -954,7 +956,9 @@ sticks = td->td_kse->ke_sticks; td->td_frame = &frame; +#if 0 KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#endif if (td->td_ucred != p->p_ucred) cred_update_thread(td); params = (caddr_t)frame.tf_esp + sizeof(int); @@ -1099,7 +1103,7 @@ */ STOPEVENT(p, S_SCX, code); -#ifdef INVARIANTS +#if 0 mtx_lock(&Giant); crfree(td->td_ucred); mtx_unlock(&Giant); Index: kern/kern_fork.c =================================================================== RCS file: /home/ncvs/src/sys/kern/kern_fork.c,v retrieving revision 1.131 diff -u -r1.131 kern_fork.c --- kern/kern_fork.c 17 Feb 2002 01:09:55 -0000 1.131 +++ kern/kern_fork.c 17 Feb 2002 08:19:55 -0000 @@ -799,7 +799,7 @@ kthread_exit(0); } PROC_UNLOCK(p); -#ifdef INVARIANTS +#if 0 mtx_lock(&Giant); crfree(td->td_ucred); mtx_unlock(&Giant); Index: kern/subr_trap.c =================================================================== RCS file: /home/ncvs/src/sys/kern/subr_trap.c,v retrieving revision 1.208 diff -u -r1.208 subr_trap.c --- kern/subr_trap.c 17 Feb 2002 01:09:55 -0000 1.208 +++ kern/subr_trap.c 17 Feb 2002 08:19:57 -0000 @@ -131,7 +131,9 @@ #endif KASSERT(TRAPF_USERMODE(framep), ("ast in kernel mode")); +#if 0 KASSERT(td->td_ucred == NULL, ("leaked ucred")); +#endif #ifdef WITNESS if (witness_list(td)) panic("Returning to user mode with mutex(s) held"); @@ -187,7 +189,7 @@ } userret(td, framep, sticks); -#ifdef INVARIANTS +#if 0 mtx_lock(&Giant); crfree(td->td_ucred); mtx_unlock(&Giant); To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 0:34:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 7D59D37B416; Sun, 17 Feb 2002 00:34:32 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1H8Y0v68305; Sun, 17 Feb 2002 00:34:00 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 00:34:00 -0800 (PST) From: Matthew Dillon Message-Id: <200202170834.g1H8Y0v68305@apollo.backplane.com> To: Julian Elischer Cc: Alfred Perlstein , Poul-Henning Kamp , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday() References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :What speed do you get with NO INVARIANTS and no WITNESS? (just posted). NOTE: All of my tests have been, are, and will be without witness. No witness. It skews things too much. :p.s what does TG stand for? The testgtod.c program I posted about 20 messages back. testgtod.c -> 'TG' abbreviation. :-) :(I need to look back to see what the 4.4 NON SMP speed was...) All my kernels under test are SMP builds, so you are on your own for non-SMP build testing. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 0:40:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hub.freebsd.org (Postfix) with ESMTP id 2F43C37B400; Sun, 17 Feb 2002 00:40:10 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020217084009.VMGR1147.rwcrmhc52.attbi.com@InterJet.elischer.org>; Sun, 17 Feb 2002 08:40:09 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id AAA44888; Sun, 17 Feb 2002 00:28:28 -0800 (PST) Date: Sun, 17 Feb 2002 00:28:28 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: buildworld comparison stable vs current In-Reply-To: <200202170818.g1H8ID067573@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG is this with the ucred change? On Sun, 17 Feb 2002, Matthew Dillon wrote: > Here are the results of a buildworld test building the -current > source tree on a SMP stable box and on a SMP current box. There was > some interest at BSDCon for a comparison just to see where we were. > Everything matches up except time and context switches, the context > switches I presume is because interrupts require context switches > in -current. > > The time difference is 1800 seconds (stable) verses 2219 seconds (current), > making stable 1.23 times faster then current at the moment. I consider > this a fairly good number for where we are, and to be expected > considering the lack of optimizations in current. > > -Matt > > (note: source is NFS mounted. /usr/obj is local disk). > > buildworld -j 10 of current on an SMP stable box: > > 1800.50 real 1445.93 user 760.32 sys > 25860 maximum resident set size > 971 average shared memory size > 1164 average unshared data size > 130 average unshared stack size > 11062987 page reclaims > 297 page faults > 0 swaps > 2013 block input operations > 5649 block output operations > 1540227 messages sent > 1530792 messages received > 6 signals received > 2467550 voluntary context switches > 1357575 involuntary context switches > > buildworld -j 10 of current on an SMP current box (same hardware specs): > > 2219.13 real 1796.37 user 968.04 sys > 25884 maximum resident set size > 912 average shared memory size > 1045 average unshared data size > 126 average unshared stack size > 11677133 page reclaims > 537 page faults > 0 swaps > 8438 block input operations > 6774 block output operations > 1439793 messages sent > 1439774 messages received > 6 signals received > 28670930 voluntary context switches > 2836255 involuntary context switches > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 10:25:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id D100B37B41A; Sun, 17 Feb 2002 10:24:49 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1HIOnw71118; Sun, 17 Feb 2002 10:24:49 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 10:24:49 -0800 (PST) From: Matthew Dillon Message-Id: <200202171824.g1HIOnw71118@apollo.backplane.com> To: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: revised buildworld comparison stable vs current References: <200202170818.g1H8ID067573@apollo.backplane.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG (note: source is NFS mounted. /usr/obj is local disk. Witness is turned off on all -current builds). stable: 1800 seconds current/invariants: 2219 seconds current/no-invariants 2142 seconds -Matt Matthew Dillon buildworld -j 10 of current on an SMP stable box: 1800.50 real 1445.93 user 760.32 sys 25860 maximum resident set size 971 average shared memory size 1164 average unshared data size 130 average unshared stack size 11062987 page reclaims 297 page faults 0 swaps 2013 block input operations 5649 block output operations 1540227 messages sent 1530792 messages received 6 signals received 2467550 voluntary context switches 1357575 involuntary context switches buildworld -j 10 of current on an SMP current box (same hardware specs): INVARIANTS TURNED ON 2219.13 real 1796.37 user 968.04 sys 25884 maximum resident set size 912 average shared memory size 1045 average unshared data size 126 average unshared stack size 11677133 page reclaims 537 page faults 0 swaps 8438 block input operations 6774 block output operations 1439793 messages sent 1439774 messages received 6 signals received 28670930 voluntary context switches 2836255 involuntary context switches buildworld -j 10 of current on an SMP current box (same hardware specs): INVARIANTS TURNED OFF 2142.18 real 1780.33 user 847.15 sys 25884 maximum resident set size 932 average shared memory size 1066 average unshared data size 126 average unshared stack size 11677329 page reclaims 521 page faults 0 swaps 8429 block input operations 12769 block output operations 1440497 messages sent 1431739 messages received 6 signals received 26925025 voluntary context switches 2818646 involuntary context switches To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 10:35:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 3074437B405; Sun, 17 Feb 2002 10:35:52 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1HIZK071270; Sun, 17 Feb 2002 10:35:20 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 10:35:20 -0800 (PST) From: Matthew Dillon Message-Id: <200202171835.g1HIZK071270@apollo.backplane.com> To: Julian Elischer Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: buildworld comparison stable vs current References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :is this with the ucred change? : Sheesh Julian, it's not as though I'm swapping out 8 different kernels for testing! YES! This is with all the changes comitted to -current plus the patch that blows away the ucred invariants when invariants are enabled, plus the gettimeofday giant removal. As described in the comments. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 10:41:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 0C81537B449; Sun, 17 Feb 2002 10:40:10 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020217184010.XVVI2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Sun, 17 Feb 2002 18:40:10 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id KAA46943; Sun, 17 Feb 2002 10:38:58 -0800 (PST) Date: Sun, 17 Feb 2002 10:38:58 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: buildworld comparison stable vs current In-Reply-To: <200202171835.g1HIZK071270@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG that was a really old email posted last night, and I think our mails crossed each other. On Sun, 17 Feb 2002, Matthew Dillon wrote: > > : > :is this with the ucred change? > : > > Sheesh Julian, it's not as though I'm swapping out 8 different > kernels for testing! YES! This is with all the changes comitted to > -current plus the patch that blows away the ucred invariants when > invariants are enabled, plus the gettimeofday giant removal. As > described in the comments. > > -Matt > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 10:42:32 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 74BF637B47F; Sun, 17 Feb 2002 10:40:17 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020217184017.XVWO2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Sun, 17 Feb 2002 18:40:17 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id KAA46941; Sun, 17 Feb 2002 10:36:09 -0800 (PST) Date: Sun, 17 Feb 2002 10:36:08 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current In-Reply-To: <200202171824.g1HIOnw71118@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG please (I know it's a lot of work) try WITNESS turned off too.. (as it didn' exist in STABLE) On Sun, 17 Feb 2002, Matthew Dillon wrote: > (note: source is NFS mounted. /usr/obj is local disk. Witness is > turned off on all -current builds). > > stable: 1800 seconds > current/invariants: 2219 seconds > current/no-invariants 2142 seconds > > -Matt > Matthew Dillon > > > > buildworld -j 10 of current on an SMP stable box: > > 1800.50 real 1445.93 user 760.32 sys > 25860 maximum resident set size > 971 average shared memory size > 1164 average unshared data size > 130 average unshared stack size > 11062987 page reclaims > 297 page faults > 0 swaps > 2013 block input operations > 5649 block output operations > 1540227 messages sent > 1530792 messages received > 6 signals received > 2467550 voluntary context switches > 1357575 involuntary context switches > > buildworld -j 10 of current on an SMP current box (same hardware specs): > INVARIANTS TURNED ON > > 2219.13 real 1796.37 user 968.04 sys > 25884 maximum resident set size > 912 average shared memory size > 1045 average unshared data size > 126 average unshared stack size > 11677133 page reclaims > 537 page faults > 0 swaps > 8438 block input operations > 6774 block output operations > 1439793 messages sent > 1439774 messages received > 6 signals received > 28670930 voluntary context switches > 2836255 involuntary context switches > > buildworld -j 10 of current on an SMP current box (same hardware specs): > INVARIANTS TURNED OFF > > 2142.18 real 1780.33 user 847.15 sys > 25884 maximum resident set size > 932 average shared memory size > 1066 average unshared data size > 126 average unshared stack size > 11677329 page reclaims > 521 page faults > 0 swaps > 8429 block input operations > 12769 block output operations > 1440497 messages sent > 1431739 messages received > 6 signals received > 26925025 voluntary context switches > 2818646 involuntary context switches > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 11: 1:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 0DCB837B402; Sun, 17 Feb 2002 11:01:47 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1HJ1En72334; Sun, 17 Feb 2002 11:01:14 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 11:01:14 -0800 (PST) From: Matthew Dillon Message-Id: <200202171901.g1HJ1En72334@apollo.backplane.com> To: Julian Elischer Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :please (I know it's a lot of work) try WITNESS turned off too.. :(as it didn' exist in STABLE) : :On Sun, 17 Feb 2002, Matthew Dillon wrote: : :> (note: source is NFS mounted. /usr/obj is local disk. Witness is :> turned off on all -current builds). Julian, WITNESS IS TURNED OFF! I've only said it a billion times now. I do not run kernels with witness turned on unless I'm forced to to track down a mutex deadlock. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 11: 6:22 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 3D56937B400; Sun, 17 Feb 2002 11:06:18 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.3/8.11.3) with ESMTP id g1HJ6Hi76734; Sun, 17 Feb 2002 12:06:17 -0700 (MST) (envelope-from imp@village.org) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.11.6/8.11.6) with ESMTP id g1HJ6GL57707; Sun, 17 Feb 2002 12:06:16 -0700 (MST) (envelope-from imp@village.org) Date: Sun, 17 Feb 2002 11:06:08 -0800 (PST) Message-Id: <20020217.110608.94337769.imp@village.org> To: julian@elischer.org Cc: dillon@apollo.backplane.com, arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current From: "M. Warner Losh" In-Reply-To: References: <200202171824.g1HIOnw71118@apollo.backplane.com> X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: Julian Elischer writes: : please (I know it's a lot of work) try WITNESS turned off too.. : (as it didn' exist in STABLE) looks like witness was turned OFF. See below: : On Sun, 17 Feb 2002, Matthew Dillon wrote: : : > (note: source is NFS mounted. /usr/obj is local disk. Witness is ^^^^^^^^^^ : > turned off on all -current builds). ^^^^^^^^^^ >> stable: 1800 seconds >> current/invariants: 2219 seconds >> current/no-invariants 2142 seconds So it looks like -current w/o invariants is 19% slower than -stable for the world-stone. Matt, is it safe to assume that the same tuning stuff is done on both -stable and -current? And that the -current numbers don't represent building, say profile, while the -stable ones don't. My gut tells me they are done the same since 19% is the ballpark of slowdown in -current I'd expect... Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 11:16:29 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id EC7B137B400; Sun, 17 Feb 2002 11:16:25 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1HJGG681111; Sun, 17 Feb 2002 11:16:16 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 11:16:16 -0800 (PST) From: Matthew Dillon Message-Id: <200202171916.g1HJGG681111@apollo.backplane.com> To: "M. Warner Losh" Cc: julian@elischer.org, arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current References: <200202171824.g1HIOnw71118@apollo.backplane.com> <20020217.110608.94337769.imp@village.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :So it looks like -current w/o invariants is 19% slower than -stable :for the world-stone. : :Matt, is it safe to assume that the same tuning stuff is done on both :-stable and -current? And that the -current numbers don't represent :building, say profile, while the -stable ones don't. My gut tells me :they are done the same since 19% is the ballpark of slowdown in :-current I'd expect... : :Warner Yes, my stable builds are fairly close to what you would see in production. However, I do leave invariants turned on in -stable. I believe invariants in -current create more of a burden then invariants in stable so the comparison is still in the ballpark and we get a good spread to judge things by when looking at the current-with-invariants and current-without-invariants numbers vs the stable-with-invariants number. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 11:40:22 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id 0C1CD37B41D; Sun, 17 Feb 2002 11:40:17 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020217194016.EPBZ1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Sun, 17 Feb 2002 19:40:16 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA47164; Sun, 17 Feb 2002 11:29:37 -0800 (PST) Date: Sun, 17 Feb 2002 11:29:35 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current In-Reply-To: <200202171901.g1HJ1En72334@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I didnt see it.. sorry On Sun, 17 Feb 2002, Matthew Dillon wrote: > > : > :please (I know it's a lot of work) try WITNESS turned off too.. > :(as it didn' exist in STABLE) > : > :On Sun, 17 Feb 2002, Matthew Dillon wrote: > : > :> (note: source is NFS mounted. /usr/obj is local disk. Witness is > :> turned off on all -current builds). > > Julian, WITNESS IS TURNED OFF! I've only said it a billion times now. > I do not run kernels with witness turned on unless I'm forced to to > track down a mutex deadlock. > > -Matt > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 12: 0: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 7DE8437B402; Sun, 17 Feb 2002 12:00:02 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g1HJupN05407; Sun, 17 Feb 2002 20:56:51 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Matthew Dillon Cc: Terry Lambert , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: Your message of "Sat, 16 Feb 2002 22:32:16 PST." <200202170632.g1H6WGt43386@apollo.backplane.com> Date: Sun, 17 Feb 2002 20:56:51 +0100 Message-ID: <5405.1013975811@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter and I actually had a sligthly different idea: Add a new syscall: int getkernstuff(struct kernstuff *kp); struct kernstuff { u_int32_t version; pid_t pid, ppid; uid_t uid, euid ... gid_t gid, guid ... signal masks ... } The idea here being that the userland process registers a single static structure with the kernel. Inside libc, this structure can be used to speed up signal processing and much more. The kernel accesses the structure in userland with copyin/copyout/fubyte, and the usage of this feature is entirely optional, programs don't have to do it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 12: 6:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id 419B837B400; Sun, 17 Feb 2002 12:06:34 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id 150CFAE607; Sun, 17 Feb 2002 12:06:34 -0800 (PST) Date: Sun, 17 Feb 2002 12:06:34 -0800 From: Alfred Perlstein To: Matthew Dillon Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current Message-ID: <20020217200634.GJ12136@elvis.mu.org> References: <200202170818.g1H8ID067573@apollo.backplane.com> <200202171824.g1HIOnw71118@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200202171824.g1HIOnw71118@apollo.backplane.com> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Matthew Dillon [020217 10:25] wrote: > (note: source is NFS mounted. /usr/obj is local disk. Witness is > turned off on all -current builds). Malloc options disabled as well? :) -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 12:12: 8 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id AF49337B42C; Sun, 17 Feb 2002 12:11:59 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1HKBsv88526; Sun, 17 Feb 2002 12:11:54 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 12:11:54 -0800 (PST) From: Matthew Dillon Message-Id: <200202172011.g1HKBsv88526@apollo.backplane.com> To: Poul-Henning Kamp Cc: Terry Lambert , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <5405.1013975811@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Peter and I actually had a sligthly different idea: : :Add a new syscall: : : int getkernstuff(struct kernstuff *kp); : : struct kernstuff { : u_int32_t version; : pid_t pid, ppid; : uid_t uid, euid ... : gid_t gid, guid ... : signal masks : ... : } : :The idea here being that the userland process registers a single :static structure with the kernel. Inside libc, this structure :can be used to speed up signal processing and much more. This would make time-of-day updates rather costly. I would much prefer if the kernel was responsible for mapping the page(s) and for being told what the user process is actually wants to look at. That way the kernel has the maximum flexibility in regards to dealing with shareable information like the time of day, and unshareable information like the pid. Frankly, there isn't much point mapping the pid, uid, etc... those calls are not in the critical path. The signal mask would be useful. But, again, I think it must be the kernel that does the mapping. It is far too dangerous otherwise IMHO. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 12:12:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 2426637B404; Sun, 17 Feb 2002 12:12:43 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1HKCgR88552; Sun, 17 Feb 2002 12:12:42 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 12:12:42 -0800 (PST) From: Matthew Dillon Message-Id: <200202172012.g1HKCgR88552@apollo.backplane.com> To: Alfred Perlstein Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current References: <200202170818.g1H8ID067573@apollo.backplane.com> <200202171824.g1HIOnw71118@apollo.backplane.com> <20020217200634.GJ12136@elvis.mu.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :* Matthew Dillon [020217 10:25] wrote: :> (note: source is NFS mounted. /usr/obj is local disk. Witness is :> turned off on all -current builds). : :Malloc options disabled as well? :) : :-Alfred Poul reminded me. Answer: I was running AH on the -current box. I'll rerun the tests later with just A but I don't expect a huge difference in timing. I am not running the expensive options on any machine. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 12:32:16 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id B85CF37B400; Sun, 17 Feb 2002 12:31:55 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g1HKSlN06010; Sun, 17 Feb 2002 21:28:47 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Matthew Dillon Cc: Terry Lambert , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: Your message of "Sun, 17 Feb 2002 12:11:54 PST." <200202172011.g1HKBsv88526@apollo.backplane.com> Date: Sun, 17 Feb 2002 21:28:47 +0100 Message-ID: <6008.1013977727@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200202172011.g1HKBsv88526@apollo.backplane.com>, Matthew Dillon wri tes: >:Peter and I actually had a sligthly different idea: >: >:Add a new syscall: >: >: int getkernstuff(struct kernstuff *kp); >: >: struct kernstuff { >: u_int32_t version; >: pid_t pid, ppid; >: uid_t uid, euid ... >: gid_t gid, guid ... >: signal masks >: ... >: } >: >:The idea here being that the userland process registers a single >:static structure with the kernel. Inside libc, this structure >:can be used to speed up signal processing and much more. > > This would make time-of-day updates rather costly. The above was not meant for time-of-day stuff but for all the "per process" stuff, in particular signal masks. If we want to do fancy timekeeping, I have/had a patch which put the timecounters on a single page which a process could map (hacked with a special device driver). Provided that the process has acccess to reading the timecounter (== not i8254) all the time-calculations can be done in userland without any calls into the kernel. This was a design parameter for timecounters, I have just never gotten around to commit the stuff. It is an optimization which I think is not quite needed yet and I certainly have other things in my queue with higher priority right now. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 12:39:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 8DA0C37B402; Sun, 17 Feb 2002 12:39:17 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1HKdDX91151; Sun, 17 Feb 2002 12:39:13 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 12:39:13 -0800 (PST) From: Matthew Dillon Message-Id: <200202172039.g1HKdDX91151@apollo.backplane.com> To: Poul-Henning Kamp Cc: Terry Lambert , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <6008.1013977727@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :with a special device driver). Provided that the process has acccess :to reading the timecounter (== not i8254) all the time-calculations :can be done in userland without any calls into the kernel. : :This was a design parameter for timecounters, I have just never :gotten around to commit the stuff. : :It is an optimization which I think is not quite needed yet and I :certainly have other things in my queue with higher priority right :now. : :-- :Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 I think we can safely say that all of this is in the 'still arguing about the design' stage. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 14:36:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 4367537B404; Sun, 17 Feb 2002 14:36:38 -0800 (PST) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 18DBA5341; Sun, 17 Feb 2002 23:36:35 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Matthew Dillon Cc: Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current References: <200202170818.g1H8ID067573@apollo.backplane.com> <200202171824.g1HIOnw71118@apollo.backplane.com> <20020217200634.GJ12136@elvis.mu.org> <200202172012.g1HKCgR88552@apollo.backplane.com> From: Dag-Erling Smorgrav Date: 17 Feb 2002 23:36:35 +0100 In-Reply-To: <200202172012.g1HKCgR88552@apollo.backplane.com> Message-ID: Lines: 12 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.1 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon writes: > Poul reminded me. Answer: I was running AH on the -current box. I'll > rerun the tests later with just A but I don't expect a huge difference > in timing. I am not running the expensive options on any machine. It's not enough to just remove H, you have to add j to turn off 0xd0ing, which is on by default in -CURRENT. Since A is also on by default, a simple 'ln -s j /etc/malloc.conf' should do the trick. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 14:40:39 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 0588C37B402; Sun, 17 Feb 2002 14:40:37 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1HMeYA94373; Sun, 17 Feb 2002 14:40:34 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 14:40:34 -0800 (PST) From: Matthew Dillon Message-Id: <200202172240.g1HMeYA94373@apollo.backplane.com> To: Dag-Erling Smorgrav Cc: Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current References: <200202170818.g1H8ID067573@apollo.backplane.com> <200202171824.g1HIOnw71118@apollo.backplane.com> <20020217200634.GJ12136@elvis.mu.org> <200202172012.g1HKCgR88552@apollo.backplane.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :0xd0ing, which is on by default in -CURRENT. Since A is also on by :default, a simple 'ln -s j /etc/malloc.conf' should do the trick. : :DES :-- :Dag-Erling Smorgrav - des@ofug.org Aarrrrrrg! Ok, I'll rerun all the tests. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 14:58:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id 588E037B400; Sun, 17 Feb 2002 14:58:46 -0800 (PST) Received: from pool0152.cvx21-bradley.dialup.earthlink.net ([209.179.192.152] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16caGa-0005WU-00; Sun, 17 Feb 2002 14:58:45 -0800 Message-ID: <3C70359B.54867FBC@mindspring.com> Date: Sun, 17 Feb 2002 14:58:35 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current References: <200202170818.g1H8ID067573@apollo.backplane.com> <200202171824.g1HIOnw71118@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > > (note: source is NFS mounted. /usr/obj is local disk. Witness is > turned off on all -current builds). > > stable: 1800 seconds > current/invariants: 2219 seconds > current/no-invariants 2142 seconds Basically, you aren't going to win back the lock overhead unless you also get the increased concurrency. When SVR4 ES/MP -- SVR4 4.2, also known as UnixWare 2.x -- went to SMP compiled by default for the UP systems, there was a performance improvement of 30% overall over the previous UP-only kernel. This was even with the network speed losses from going to the ODI drivers (these losses were not inconsiderable; if I owned SCO like Caldera does now, that's the first thing I'd revert, for another 15%). The reason this happened was because of the increased concurrency, even in the UP case, from work to make UFS (FFS) reentrant, the TCP stack reentrant, etc.. For this particular "benchmark", the places where there are natural stall points are: 1) FS reentrancy on a directory. 2) Stalling once a buffer is scheduled for write, and goes read-only, but stays in the pipeline for a long time. The first one is pretty obvious; basically, moving to the FS owning the vnodes, rather than getting them from a system pool, and getting rid of the VOP_LOCK that's required by the second chance cache and the vnode reclaimer in the system, are enough to increase the concurrency there. The second one is harder, but easier. Intention mode locking would reduce the stall, but typical usage in a compile situation means that the win there would not be as much as you might naievely expect. The problem is that just blocking the operations on the buffers *only* when they have gone to the driver, rather than once they've entered the BIO system doesn't stop the final stall on the object file creation. THe way you have to deal with that (if you are unwilling to change the directory layout management) is to lock directory blocks individually, and add additional directory blocks, up to a small number, rather than blocking writes to the entire directory when a single block write is in progress. By treating directory blocks as atomic, and allowing a little bloat by allowing creates in a new block when creates in the prior block are concurrently prohibited (reading it to verify that the file doesn't already exist is OK), you should get a 60% or so increase in stall, at least from my simple statistics gathering on a build world, counting the stalls based on this. Lots of "low hanging fruit" there... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 15: 8:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id D839C37B405; Sun, 17 Feb 2002 15:08:43 -0800 (PST) Received: from pool0152.cvx21-bradley.dialup.earthlink.net ([209.179.192.152] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16caQC-0000Ec-00; Sun, 17 Feb 2002 15:08:41 -0800 Message-ID: <3C7037EE.E2D6B02@mindspring.com> Date: Sun, 17 Feb 2002 15:08:30 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: Matthew Dillon , arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised buildworld comparison stable vs current References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer wrote: > > please (I know it's a lot of work) try WITNESS turned off too.. > (as it didn' exist in STABLE) Uh... he said before that *all* his tests were without WITLESSS, because it was so harsh on performance that it wasn't even worth testing. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 16: 7:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 42AFA37B402; Sun, 17 Feb 2002 16:07:16 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id LAA07090; Mon, 18 Feb 2002 11:06:52 +1100 Date: Mon, 18 Feb 2002 11:06:51 +1100 (EST) From: Bruce Evans X-X-Sender: To: Poul-Henning Kamp Cc: Matthew Dillon , Terry Lambert , Julian Elischer , Alfred Perlstein , , , , Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: <5405.1013975811@critter.freebsd.dk> Message-ID: <20020218110408.N3938-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 17 Feb 2002, Poul-Henning Kamp wrote: > Peter and I actually had a sligthly different idea: > > Add a new syscall: > > int getkernstuff(struct kernstuff *kp); > > struct kernstuff { > u_int32_t version; > pid_t pid, ppid; > uid_t uid, euid ... > gid_t gid, guid ... > signal masks > ... > } > > The idea here being that the userland process registers a single > static structure with the kernel. Inside libc, this structure > can be used to speed up signal processing and much more. This might have been useful in 1980, but gettimeofday() in practice has 10^6 times as much resolution now. However, updating the kernel struct 10^6 times per second is beyond the capabilities of even current machiens, in software. It is possible in hardware of course. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 16:13:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from harrier.prod.itd.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by hub.freebsd.org (Postfix) with ESMTP id E7D5437B405; Sun, 17 Feb 2002 16:13:29 -0800 (PST) Received: from pool0152.cvx21-bradley.dialup.earthlink.net ([209.179.192.152] helo=mindspring.com) by harrier.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16cbQe-0002jq-00; Sun, 17 Feb 2002 16:13:13 -0800 Message-ID: <3C70470E.BD2F048D@mindspring.com> Date: Sun, 17 Feb 2002 16:13:02 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Matthew Dillon , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <5405.1013975811@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > Peter and I actually had a sligthly different idea: > > Add a new syscall: > > int getkernstuff(struct kernstuff *kp); > > struct kernstuff { > u_int32_t version; > pid_t pid, ppid; > uid_t uid, euid ... > gid_t gid, guid ... > signal masks > ... > } > > The idea here being that the userland process registers a single > static structure with the kernel. Inside libc, this structure > can be used to speed up signal processing and much more. > > The kernel accesses the structure in userland with copyin/copyout/fubyte, > and the usage of this feature is entirely optional, programs don't > have to do it. I guess this would be read/write? I don't think copyin() access would be the best way to do it, if that's the case, since the user could lie to the kernel that way. That makes the user space data a reflection of the kernel data, which is, in a way, acceptable, I guess. I personally wouldn't be adverse to mapping the proc structure read-only into user space, and page aligning the allocations. The problem is with the page sized requirement eating a potentially large chunk of the address space. For example, a "safe" thing for the kernel to access would be a per process dual mapped page (e.g. a kernel page pointed to by the proc struct, where the PG_U bit was simply set on the page, and the address known to user space), which wouldbe used as mailbox for threads, and where it would be OK for the user space threads scheduler to keep context the kernel might want, like current thread ID. Per process stuff is much more difficult because of the address space bloat and the page size restriction, than something like getimeofday, which could be read-only in all user space processes fairly painlessly. In any case, soing this for other rarely changed values, where the copy is reflected out but not in, could work with a user registration, as you suggest. It'd probably be a lot better way of handling the getpid/getgid/etc. stuff than crossing the system call boundary, and taking lots of credential locks, etc.. Frankly, I think there are a lot of places where there is a credential present where there is really not a lot of need for the credential to be there, and that making the interfaces homogeneous in places where it doesn't make a lot of sense to do that is really costing FreeBSD some performance. I never really understood the requirement for getimeofday to have these restrictions, to point back to the current discussion. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 16:24:25 2002 Delivered-To: freebsd-arch@freebsd.org Received: from harrier.prod.itd.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by hub.freebsd.org (Postfix) with ESMTP id 3F1F737B402; Sun, 17 Feb 2002 16:24:19 -0800 (PST) Received: from pool0152.cvx21-bradley.dialup.earthlink.net ([209.179.192.152] helo=mindspring.com) by harrier.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16cbbK-0006Up-00; Sun, 17 Feb 2002 16:24:14 -0800 Message-ID: <3C7049A4.15412853@mindspring.com> Date: Sun, 17 Feb 2002 16:24:04 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <5405.1013975811@critter.freebsd.dk> <200202172011.g1HKBsv88526@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > :Peter and I actually had a sligthly different idea: [ ... ] > This would make time-of-day updates rather costly. I'm pretty darn sure that Poul was referring to the getpid section of the conversation, and not to a gettimeofday specifically. > Frankly, there isn't much point mapping the pid, uid, etc... those > calls are not in the critical path. The signal mask would be useful. > But, again, I think it must be the kernel that does the mapping. It > is far too dangerous otherwise IMHO. See other post; it's a reflection out. The signal mask would fall into the same area as "current active user space thread", where writing it from user space would not be a bad thing. This gives us another 8k per process, and another 4k of global (if you want to throw gettimeofday into the mix). I guess the question is whether Poul's reflection suggestion for things that would normally be read-only from user space and add 4k with the simple expedient approach I siggested would be worth it. IMO, it probably would: changing gid our uid is not a common operation, and requires a system call anyway; an extra copyout isn't terrible overhead. There _is_ a question of whether or not the overhead of doing a dual mapping is too expensive in terms of KVA space mapping (you would need two: one for read-only, and another for read-write in my approach). For process writable values, I think a page makes more sense, but the number of these is pretty small; of them, the signal mask would show incredible overall value if it existed in 4.x right now, particularly for libc_r. However, there is much less value in 5.x, I think, for this. The libc_r wrappers that manipulate the signal mask go away with the KSE code. In addition, we could probably get rid of the problem in libc_r, even in 4.x, by thinking about how to solve it correctly. It seems to me that we could unmask all signals, and set up a multiplex signal handler for all of them, in any threaded program, and apply the mask and do the dispatch from the multiplex handler, instead. In other words, turn all the signal functions visible to programs in a threaded context into user space calls, which would get rid of all those extra system calls. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 16:30:47 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id DB87737B402; Sun, 17 Feb 2002 16:30:33 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1I0UU309210; Sun, 17 Feb 2002 16:30:30 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 16:30:30 -0800 (PST) From: Matthew Dillon Message-Id: <200202180030.g1I0UU309210@apollo.backplane.com> To: Terry Lambert Cc: Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <5405.1013975811@critter.freebsd.dk> <200202172011.g1HKBsv88526@apollo.backplane.com> <3C7049A4.15412853@mindspring.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I'm just going to point something out here, and that is my proposed system call fully supports the kernel saying, in effect, 'I don't want you accessing this particular parameter from shared memory, use the old way of doing it'. For the vast majority of processes in a system this is a perfectly reasonable response. The shared memory feature would only need to be enabled for those few processes, like web servers, databases, and big threaded programs that really need it. So we can afford to waste some memory for the few processes that actually need the feature as long as we don't waste any for the processes that don't. -Matt Matthew Dillon :Matthew Dillon wrote: :> :Peter and I actually had a sligthly different idea: : :[ ... ] : :> This would make time-of-day updates rather costly. : :I'm pretty darn sure that Poul was referring to the :getpid section of the conversation, and not to a :gettimeofday specifically. : :> Frankly, there isn't much point mapping the pid, uid, etc... those :> calls are not in the critical path. The signal mask would be useful. :> But, again, I think it must be the kernel that does the mapping. It :> is far too dangerous otherwise IMHO. : :See other post; it's a reflection out. : :The signal mask would fall into the same area as :"current active user space thread", where writing it :from user space would not be a bad thing. : :This gives us another 8k per process, and another 4k :of global (if you want to throw gettimeofday into the :mix). : :I guess the question is whether Poul's reflection :suggestion for things that would normally be read-only :from user space and add 4k with the simple expedient :approach I siggested would be worth it. : :IMO, it probably would: changing gid our uid is not a :common operation, and requires a system call anyway; :an extra copyout isn't terrible overhead. There _is_ :a question of whether or not the overhead of doing a :dual mapping is too expensive in terms of KVA space :mapping (you would need two: one for read-only, and :another for read-write in my approach). : :For process writable values, I think a page makes more :sense, but the number of these is pretty small; of them, :the signal mask would show incredible overall value if :it existed in 4.x right now, particularly for libc_r. : :However, there is much less value in 5.x, I think, for :this. The libc_r wrappers that manipulate the signal :mask go away with the KSE code. : :In addition, we could probably get rid of the problem :in libc_r, even in 4.x, by thinking about how to solve :it correctly. : :It seems to me that we could unmask all signals, and set :up a multiplex signal handler for all of them, in any :threaded program, and apply the mask and do the dispatch :from the multiplex handler, instead. In other words, :turn all the signal functions visible to programs in a :threaded context into user space calls, which would get :rid of all those extra system calls. : :-- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 16:32:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 4F3E237B404; Sun, 17 Feb 2002 16:32:36 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id LAA10351; Mon, 18 Feb 2002 11:32:20 +1100 Date: Mon, 18 Feb 2002 11:32:19 +1100 (EST) From: Bruce Evans X-X-Sender: To: Poul-Henning Kamp Cc: Matthew Dillon , Terry Lambert , Julian Elischer , Alfred Perlstein , , , , Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: <6008.1013977727@critter.freebsd.dk> Message-ID: <20020218112751.T3970-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 17 Feb 2002, Poul-Henning Kamp wrote: > In message <200202172011.g1HKBsv88526@apollo.backplane.com>, Matthew Dillon wri > tes: > >:Peter and I actually had a sligthly different idea: > >: > >:Add a new syscall: > >: > >: int getkernstuff(struct kernstuff *kp); > >:... > >:The idea here being that the userland process registers a single > >:static structure with the kernel. Inside libc, this structure > >:can be used to speed up signal processing and much more. > > > > This would make time-of-day updates rather costly. > > The above was not meant for time-of-day stuff but for all the > "per process" stuff, in particular signal masks. I think the synchronization costs for this would be high. > If we want to do fancy timekeeping, I have/had a patch which put > the timecounters on a single page which a process could map (hacked > with a special device driver). Provided that the process has acccess > to reading the timecounter (== not i8254) all the time-calculations > can be done in userland without any calls into the kernel. I know the synchronization costs for this would be high. They are too high even in the kernel, so we skip them for the get*time() family, at the cost of getting unsynchronised times. This may be acceptable in some applications. The read-only variable in kernel memory has similar synchronisation problems. It corresponds fairly directly with the variable read by the get*time() family. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 16:52: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 2C93537B428; Sun, 17 Feb 2002 16:51:56 -0800 (PST) Received: from pool0152.cvx21-bradley.dialup.earthlink.net ([209.179.192.152] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16cc1n-0004xN-00; Sun, 17 Feb 2002 16:51:35 -0800 Message-ID: <3C70500B.31282873@mindspring.com> Date: Sun, 17 Feb 2002 16:51:23 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bruce Evans Cc: Poul-Henning Kamp , Matthew Dillon , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <20020218112751.T3970-100000@gamplex.bde.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bruce Evans wrote: > > If we want to do fancy timekeeping, I have/had a patch which put > > the timecounters on a single page which a process could map (hacked > > with a special device driver). Provided that the process has acccess > > to reading the timecounter (== not i8254) all the time-calculations > > can be done in userland without any calls into the kernel. > > I know the synchronization costs for this would be high. They are too > high even in the kernel, so we skip them for the get*time() family, > at the cost of getting unsynchronised times. This may be acceptable > in some applications. The read-only variable in kernel memory has > similar synchronisation problems. It corresponds fairly directly > with the variable read by the get*time() family. I *know* that it is a significant win to flip-flop the timecounter context into a reflected user space page on each and every clock interrupt. The benefits to doing this for squid alone, which must make about 5 gettimeofday() calls per transaction in order to do "correct" logging (IMO, logging is eye candy, though some idiots insist on billing based on post processing log files), are more than significant, they are a doubling of the transaction rate. Crossing protection domains for read-only data which is frequently accessed is a really, really bad idea. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 16:58:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 6932D37B404; Sun, 17 Feb 2002 16:58:07 -0800 (PST) Received: from pool0152.cvx21-bradley.dialup.earthlink.net ([209.179.192.152] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16cc83-0003j1-00; Sun, 17 Feb 2002 16:58:03 -0800 Message-ID: <3C705190.8B56A024@mindspring.com> Date: Sun, 17 Feb 2002 16:57:52 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <5405.1013975811@critter.freebsd.dk> <200202172011.g1HKBsv88526@apollo.backplane.com> <3C7049A4.15412853@mindspring.com> <200202180030.g1I0UU309210@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > I'm just going to point something out here, and that is my proposed > system call fully supports the kernel saying, in effect, 'I don't want > you accessing this particular parameter from shared memory, use the > old way of doing it'. > > For the vast majority of processes in a system this is a perfectly > reasonable response. The shared memory feature would only need to be > enabled for those few processes, like web servers, databases, and > big threaded programs that really need it. > > So we can afford to waste some memory for the few processes that actually > need the feature as long as we don't waste any for the processes that > don't. When I was talking about the 4k/8k per process KVA mapping costs, I was not really concerned with the per process costs, I was merely documenting them, in case someone else was concerned. Personally, I think the number of clients goes up significantly faster than the number of processes or threads; obviously this would not be true if there were a thread per client, but I think that people who code that way will never beat my numbers on total clients per host, so I don't care about them anyway, since they are already shooting themselves before they even have a chance to enter the race. So actually, I would prefer that the call succeed always, and that the failure be fatal. The only reason to maintain support for system-call based access to the data is for legacy applications, IMO. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 17:39:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id B1FA137B402; Sun, 17 Feb 2002 17:39:40 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1I1deU09585; Sun, 17 Feb 2002 17:39:40 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 17:39:40 -0800 (PST) From: Matthew Dillon Message-Id: <200202180139.g1I1deU09585@apollo.backplane.com> To: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: revised II buildworld comparison stable vs current w/ malloc.conf fixes References: <200202170818.g1H8ID067573@apollo.backplane.com> <200202171824.g1HIOnw71118@apollo.backplane.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is with Aj for /etc/malloc.conf on the current box, making it compatible with the stable box. Summary: stable/invariants: 1800 seconds current/invariants: 2097 seconds At least in regards to buildworld tests, current is very close to stable. This is with witness is turned off, 'Aj' (i.e. nominal performance) malloc options, Invariants turned on, SMP build. -Matt buildworld -j 10 of current on an SMP stable box: 1800.50 real 1445.93 user 760.32 sys 25860 maximum resident set size 971 average shared memory size 1164 average unshared data size 130 average unshared stack size 11062987 page reclaims 297 page faults 0 swaps 2013 block input operations 5649 block output operations 1540227 messages sent 1530792 messages received 6 signals received 2467550 voluntary context switches 1357575 involuntary context switches buildworld -j 10 of current on an SMP current box: 2097.53 real 1662.69 user 907.48 sys 25868 maximum resident set size 934 average shared memory size 1055 average unshared data size 126 average unshared stack size 11062160 page reclaims 539 page faults 0 swaps 8473 block input operations 6701 block output operations 1439371 messages sent 1430534 messages received 6 signals received 23879443 voluntary context switches 2564878 involuntary context switches To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 17:42:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id A6FC537B434; Sun, 17 Feb 2002 17:42:06 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1I1g4L09623; Sun, 17 Feb 2002 17:42:04 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 17:42:04 -0800 (PST) From: Matthew Dillon Message-Id: <200202180142.g1I1g4L09623@apollo.backplane.com> To: Terry Lambert Cc: Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <5405.1013975811@critter.freebsd.dk> <200202172011.g1HKBsv88526@apollo.backplane.com> <3C7049A4.15412853@mindspring.com> <200202180030.g1I0UU309210@apollo.backplane.com> <3C705190.8B56A024@mindspring.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :So actually, I would prefer that the call succeed :always, and that the failure be fatal. The only :reason to maintain support for system-call based :access to the data is for legacy applications, IMO. : :-- Terry I have to disagree. Structural based shared memory has severe issues with forwards and backwards compatibility, even if you stick a version number on it. So it's nice to have but the kernel should not be forced to supply it. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 19:53:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id E8A4537B400; Sun, 17 Feb 2002 19:53:10 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id OAA01710; Mon, 18 Feb 2002 14:52:52 +1100 Date: Mon, 18 Feb 2002 14:52:51 +1100 (EST) From: Bruce Evans X-X-Sender: To: Terry Lambert Cc: Poul-Henning Kamp , Matthew Dillon , Julian Elischer , Alfred Perlstein , , , , Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout(). Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: <3C70500B.31282873@mindspring.com> Message-ID: <20020218144148.F4583-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 17 Feb 2002, Terry Lambert wrote: > Bruce Evans wrote: > > I know the synchronization costs for this would be high. They are too > > high even in the kernel, so we skip them for the get*time() family, > > at the cost of getting unsynchronised times. This may be acceptable > > in some applications. The read-only variable in kernel memory has > > similar synchronisation problems. It corresponds fairly directly > > with the variable read by the get*time() family. > > I *know* that it is a significant win to flip-flop the > timecounter context into a reflected user space page > on each and every clock interrupt. That can't be used to implement gettimeofday(). It can only be used to implement the userland equivalants of get_micro[up]time() and get_nano[up]time(). I don't like using these even for stamping file times in seconds in the kernel (though they have more than enough precision for this), since they give times that are incoherent relative to other ways of determining the time. > The benefits to doing this for squid alone, which must > make about 5 gettimeofday() calls per transaction in > order to do "correct" logging (IMO, logging is eye candy, > though some idiots insist on billing based on post > processing log files), are more than significant, they > are a doubling of the transaction rate. Apparently they don't actually look at the timestamps and notice that they (the timestamps) are often the same for different transactions because the timestamps have low resolution. You can fake the increment, but then you can fake the time too. > Crossing protection domains for read-only data which is > frequently accessed is a really, really bad idea. Does it really matter for logging? Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 20:10: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 0E39A37B404; Sun, 17 Feb 2002 20:10:01 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1I49vj10455; Sun, 17 Feb 2002 20:09:57 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 20:09:57 -0800 (PST) From: Matthew Dillon Message-Id: <200202180409.g1I49vj10455@apollo.backplane.com> To: Bruce Evans Cc: Terry Lambert , Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , , , , Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <20020218144148.F4583-100000@gamplex.bde.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :> The benefits to doing this for squid alone, which must :> make about 5 gettimeofday() calls per transaction in :> order to do "correct" logging (IMO, logging is eye candy, :> though some idiots insist on billing based on post :> processing log files), are more than significant, they :> are a doubling of the transaction rate. : :Apparently they don't actually look at the timestamps and notice :that they (the timestamps) are often the same for different :transactions because the timestamps have low resolution. You :can fake the increment, but then you can fake the time too. :... Just a note: The gettimeofday() has an overhead of only 2-3 uS in -current. Squid would have to be doing an aweful lot of transactions for it to matter and even if it did, if that actually turned out to be the bottleneck I'll eat my hat. And if I wind up eating my hat the next thing I'll do is spend the necessary 5 seconds writing a little code to make squid only call gettimeofday every 5th time. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 20:32:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 5DDC637B400; Sun, 17 Feb 2002 20:32:14 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.3/8.11.3) with ESMTP id g1I4WAi79498; Sun, 17 Feb 2002 21:32:10 -0700 (MST) (envelope-from imp@village.org) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.11.6/8.11.6) with ESMTP id g1I4VqL61477; Sun, 17 Feb 2002 21:31:56 -0700 (MST) (envelope-from imp@village.org) Date: Sun, 17 Feb 2002 20:31:41 -0800 (PST) Message-Id: <20020217.203141.90997753.imp@village.org> To: dillon@apollo.backplane.com Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: revised II buildworld comparison stable vs current w/ malloc.conf fixes From: "M. Warner Losh" In-Reply-To: <200202180139.g1I1deU09585@apollo.backplane.com> References: <200202170818.g1H8ID067573@apollo.backplane.com> <200202171824.g1HIOnw71118@apollo.backplane.com> <200202180139.g1I1deU09585@apollo.backplane.com> X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <200202180139.g1I1deU09585@apollo.backplane.com> Matthew Dillon writes: : This is with Aj for /etc/malloc.conf on the current box, making it : compatible with the stable box. Summary: : : stable/invariants: 1800 seconds : current/invariants: 2097 seconds : : At least in regards to buildworld tests, current is very close to : stable. This is 16.5% Without invariants before, another percent or two was gained by turning invariants off. I think that this means that we have a good chance to hit the 10% worse mark on worldstone... Of course, my -current box has other strange interrupt interactions that is making psm spit out messages because interrupt latency is too high. But I suspect that will also improve as we move towards release... Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 20:42: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 7BBAF37B402; Sun, 17 Feb 2002 20:41:57 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id PAA07679; Mon, 18 Feb 2002 15:41:41 +1100 Date: Mon, 18 Feb 2002 15:41:40 +1100 (EST) From: Bruce Evans X-X-Sender: To: Matthew Dillon Cc: Terry Lambert , Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , , , , Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout(). Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: <200202180409.g1I49vj10455@apollo.backplane.com> Message-ID: <20020218151249.E4728-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 17 Feb 2002, Matthew Dillon wrote: > Just a note: The gettimeofday() has an overhead of only 2-3 uS in > -current. Squid would have to be doing an aweful lot of transactions And that is only on slow machines and/or under SMP. On my Athlon1600, it has an overhead of 0.3-0.4 nsec. I have been benchmarking it for many years and recently had to change the benchmark program to use clock_gettime(2) instead of gettimeofday() when getttimeofday()'s resolution became too small. > for it to matter and even if it did, if that actually turned out to be the > bottleneck I'll eat my hat. And if I wind up eating my hat the next > thing I'll do is spend the necessary 5 seconds writing a little code > to make squid only call gettimeofday every 5th time. Or use an alarm to set it... Summary of overheads for gettimeofday() (clock_gettime() for sub-usec ones) in a loop:: -my-current on Athlon1600 @ 1532 0.381 usec (TSC) -current on Athlon1600 @ 1532 0.512 usec (TSC) 4.5RC on Athlon1600 @ 1532 0.345 usec (TSC) -my-current-2000 on Celeron366 @ 522 1.389 usec (TSC) -my-current-1999 on Celeron400 @ 450 1.683 usec (TSC) -my-current-1998 on K6-1/233 2.241 usec (TSC) -my-current-2001 on Pentium1 @ 133 5.474 usec (TSC) (reduce SMPng pessim) -my-current-2001 on Pentium1 @ 133 6.786 usec (TSC) (more SMPng pessim) -my-current-2000 on Pentium1 @ 133 5.376 usec (TSC) (SMPng pessim) -my-current-1999 on Pentium1 @ 133 4.630 usec (TSC) -my-current-1998 on Pentium1 @ 133 4.615 usec (TSC) (timecounter pessim) -my-current-1998 on Pentium1 @ 133 3.443 usec (TSC) (pre-timecounter) -my-current-1996 on Pentium1 @ 133 3.320 usec (TSC) -my-current-2000 on 486DX2/66 21.843 usec (i8254) (SMPng pessim) -my-current-1995 on 486DX2/66 14.276 usec (i8254) Except for the non-TSC timecounter, this mostly measure overheads for syscalls generally. The core part of gettimeofday(), i.e., microtime() takes less than 100 nsec on the Athlon. This stuff is fun to optimize, but I haven''t found a case where optimising all syscalls (not just gettimeofday()) makes a difference in a normal application. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 20:50:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mass.dis.org (dhcp45-21.dis.org [216.240.45.21]) by hub.freebsd.org (Postfix) with ESMTP id 6858437B400 for ; Sun, 17 Feb 2002 20:50:10 -0800 (PST) Received: from mass.dis.org (localhost [127.0.0.1]) by mass.dis.org (8.11.6/8.11.6) with ESMTP id g1I4nwA03857; Sun, 17 Feb 2002 20:49:59 -0800 (PST) (envelope-from msmith@mass.dis.org) Message-Id: <200202180449.g1I4nwA03857@mass.dis.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Bruce Evans Cc: arch@FreeBSD.ORG Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) In-reply-to: Your message of "Mon, 18 Feb 2002 11:06:51 +1100." <20020218110408.N3938-100000@gamplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 17 Feb 2002 20:49:58 -0800 From: Michael Smith Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > This might have been useful in 1980, but gettimeofday() in practice > has 10^6 times as much resolution now. However, updating the kernel > struct 10^6 times per second is beyond the capabilities of even current > machiens, in software. It is possible in hardware of course. ... except that no vendor would ever make it work properly, forcing us to poll the structure countless times to detect whether it was valid. *sigh* -- To announce that there must be no criticism of the president, or that we are to stand by the president, right or wrong, is not only unpatriotic and servile, but is morally treasonable to the American public. - Theodore Roosevelt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 22:21:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 676C137B405; Sun, 17 Feb 2002 22:21:11 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g1I6I0N32875; Mon, 18 Feb 2002 07:18:00 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Terry Lambert Cc: Matthew Dillon , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: Your message of "Sun, 17 Feb 2002 16:13:02 PST." <3C70470E.BD2F048D@mindspring.com> Date: Mon, 18 Feb 2002 07:18:00 +0100 Message-ID: <32873.1014013080@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <3C70470E.BD2F048D@mindspring.com>, Terry Lambert writes: >I guess this would be read/write? Yes. >I don't think copyin() access would be the best way to >do it, if that's the case, since the user could lie to >the kernel that way. ...and consequently coredump... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 22:29: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id B00DE37B400; Sun, 17 Feb 2002 22:29:00 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g1I6PoN33060; Mon, 18 Feb 2002 07:25:51 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Bruce Evans Cc: Terry Lambert , Matthew Dillon , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout(). Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: Your message of "Mon, 18 Feb 2002 14:52:51 +1100." <20020218144148.F4583-100000@gamplex.bde.org> Date: Mon, 18 Feb 2002 07:25:50 +0100 Message-ID: <33058.1014013550@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020218144148.F4583-100000@gamplex.bde.org>, Bruce Evans writes: >> I *know* that it is a significant win to flip-flop the >> timecounter context into a reflected user space page >> on each and every clock interrupt. > >That can't be used to implement gettimeofday(). It can only be used >to implement the userland equivalants of get_micro[up]time() and >get_nano[up]time(). I don't like using these even for stamping file >times in seconds in the kernel (though they have more than enough >precision for this), since they give times that are incoherent >relative to other ways of determining the time. Erhm... I am going to ask to let this this discussion die out now, nobody seems to have entirely grasped the idea so I think we shall postpone it until I have some patches. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 23: 7:53 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 6986037B41E; Sun, 17 Feb 2002 23:06:44 -0800 (PST) Received: from pool0305.cvx21-bradley.dialup.earthlink.net ([209.179.193.50] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16chsi-0006ya-00; Sun, 17 Feb 2002 23:06:36 -0800 Message-ID: <3C70A7F0.C409B43F@mindspring.com> Date: Sun, 17 Feb 2002 23:06:24 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bruce Evans Cc: Poul-Henning Kamp , Matthew Dillon , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout().Is copyout() MPSAFE on non-i386 archs? ) References: <20020218144148.F4583-100000@gamplex.bde.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bruce Evans wrote: > > > I know the synchronization costs for this would be high. They are too > > > high even in the kernel, so we skip them for the get*time() family, > > > at the cost of getting unsynchronised times. This may be acceptable > > > in some applications. The read-only variable in kernel memory has > > > similar synchronisation problems. It corresponds fairly directly > > > with the variable read by the get*time() family. > > > > I *know* that it is a significant win to flip-flop the > > timecounter context into a reflected user space page > > on each and every clock interrupt. > > That can't be used to implement gettimeofday(). It can only be used > to implement the userland equivalants of get_micro[up]time() and > get_nano[up]time(). I don't like using these even for stamping file > times in seconds in the kernel (though they have more than enough > precision for this), since they give times that are incoherent > relative to other ways of determining the time. I guess you want me to implement it again, outside the context of the code I implemented for ClickArray, and donate it back to the project? The only dependency is that the context is not updated during the copy operation itself, which can be guaranteed by a single "generation" counter save before/test after around the copy (trivial). In actual practice, I've never seen this happen in the time it took the inline function version of the expanded gettimeofday() to run, even under heavy load. I think that's because everything fits in cache, though I can't guarantee that missing two time slices has ever happened in that application (in any case, it's protected). > > The benefits to doing this for squid alone, which must > > make about 5 gettimeofday() calls per transaction in > > order to do "correct" logging (IMO, logging is eye candy, > > though some idiots insist on billing based on post > > processing log files), are more than significant, they > > are a doubling of the transaction rate. > > Apparently they don't actually look at the timestamps and notice > that they (the timestamps) are often the same for different > transactions because the timestamps have low resolution. You > can fake the increment, but then you can fake the time too. Not really. The time stamps are spread over a fairly high latency path of client connection, request complete, connection to back end, request sent, response received, response sent, disconnect. While concurrent requests may end up with the same stamp, individual client timings are pretty accurate -- at least as accurate as calling the kernel kettimeofday() instead, within a very small variance (~.1ms, measured). > > Crossing protection domains for read-only data which is > > frequently accessed is a really, really bad idea. > > Does it really matter for logging? If all you were doing was logging, no, but since the logging is indicative of doing work, yes. 8-) 8-). A significant amout of the system call overhead in the processing of a kevent based program is in the logging. One approach we considered was to add a timestamp to the kevents themselves, and log using that, instead, but it turns out that the logging frequency was lower than the even frequency by enough of a margin that the expense of doing that was too high. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 23:12:16 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id B2C6D37B404; Sun, 17 Feb 2002 23:12:12 -0800 (PST) Received: from pool0305.cvx21-bradley.dialup.earthlink.net ([209.179.193.50] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16chy2-00028r-00; Sun, 17 Feb 2002 23:12:06 -0800 Message-ID: <3C70A93B.8E1D24E8@mindspring.com> Date: Sun, 17 Feb 2002 23:11:55 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: Bruce Evans , Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout().Is copyout() MPSAFE on non-i386 archs? ) References: <20020218144148.F4583-100000@gamplex.bde.org> <200202180409.g1I49vj10455@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > Just a note: The gettimeofday() has an overhead of only 2-3 uS in > -current. Squid would have to be doing an aweful lot of transactions > for it to matter and even if it did, if that actually turned out to be the > bottleneck I'll eat my hat. And if I wind up eating my hat the next > thing I'll do is spend the necessary 5 seconds writing a little code > to make squid only call gettimeofday every 5th time. It was a lot of transactions... 22,000 HTTP 1.1 requests per second. Obviously, it wasn't squid code (8-)), but the logging requirement was the same because of the preexisiting squid log digest software, and the statistics gathering necessary to be comparable with competing products. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 23:24:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 6A29637B405; Sun, 17 Feb 2002 23:22:05 -0800 (PST) Received: from pool0305.cvx21-bradley.dialup.earthlink.net ([209.179.193.50] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16ci7b-0007YC-00; Sun, 17 Feb 2002 23:21:59 -0800 Message-ID: <3C70AB8B.96589869@mindspring.com> Date: Sun, 17 Feb 2002 23:21:47 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bruce Evans Cc: Matthew Dillon , Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout().Is copyout() MPSAFE on non-i386 archs? ) References: <20020218151249.E4728-100000@gamplex.bde.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bruce Evans wrote: > On Sun, 17 Feb 2002, Matthew Dillon wrote: > > > Just a note: The gettimeofday() has an overhead of only 2-3 uS in > > -current. Squid would have to be doing an aweful lot of transactions > > And that is only on slow machines and/or under SMP. On my Athlon1600, > it has an overhead of 0.3-0.4 nsec. I have been benchmarking it for > many years and recently had to change the benchmark program to use > clock_gettime(2) instead of gettimeofday() when getttimeofday()'s > resolution became too small. .4nsec * 22,000 transactions/sec * 5 timestamps/transaction = 440us/sec spent on time stamps. Up that to 2uS and... = 220ms/sec on time stamps = ~1/4 of all available time spent on time stamps. [ ... other stuff not really applicable; can discuss why off list, if necessary ... ] -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 23:33:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 8120A37B400; Sun, 17 Feb 2002 23:33:48 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1I7Xej12207; Sun, 17 Feb 2002 23:33:40 -0800 (PST) (envelope-from dillon) Date: Sun, 17 Feb 2002 23:33:40 -0800 (PST) From: Matthew Dillon Message-Id: <200202180733.g1I7Xej12207@apollo.backplane.com> To: Terry Lambert Cc: Bruce Evans , Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout().Is copyout() MPSAFE on non-i386 archs? ) References: <20020218151249.E4728-100000@gamplex.bde.org> <3C70AB8B.96589869@mindspring.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :.4nsec * 22,000 transactions/sec * 5 timestamps/transaction := 440us/sec spent on time stamps. : :Up that to 2uS and... = 220ms/sec on time stamps = ~1/4 of :all available time spent on time stamps. : :[ ... other stuff not really applicable; can discuss why : off list, if necessary ... ] : :-- Terry This is a meaningless exercise if you don't take into account everything the program in question is doing on a per transaction basis. To say that a single 2uS system call takes 25% of available resources also infers that all your other code requires only 6uS per transaction of overhead, which means it doesn't do very much eh? -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 23:36: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id B77C737B42C; Sun, 17 Feb 2002 23:35:55 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id SAA26665; Mon, 18 Feb 2002 18:35:39 +1100 Date: Mon, 18 Feb 2002 18:35:38 +1100 (EST) From: Bruce Evans X-X-Sender: To: Terry Lambert Cc: Matthew Dillon , Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , , , , Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout().Is copyout() MPSAFE on non-i386 archs? ) In-Reply-To: <3C70AB8B.96589869@mindspring.com> Message-ID: <20020218182727.C5246-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 17 Feb 2002, Terry Lambert wrote: > Bruce Evans wrote: > > On Sun, 17 Feb 2002, Matthew Dillon wrote: > > > > > Just a note: The gettimeofday() has an overhead of only 2-3 uS in > > > -current. Squid would have to be doing an aweful lot of transactions > > > > And that is only on slow machines and/or under SMP. On my Athlon1600, > > it has an overhead of 0.3-0.4 nsec. I have been benchmarking it for argh, usec > > many years and recently had to change the benchmark program to use > > clock_gettime(2) instead of gettimeofday() when getttimeofday()'s > > resolution became too small. > > .4nsec * 22,000 transactions/sec * 5 timestamps/transaction > = 440us/sec spent on time stamps. This is not long :-), but 440 msec is. > Up that to 2uS and... = 220ms/sec on time stamps = ~1/4 of > all available time spent on time stamps. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 23:37:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id B104C37B404; Sun, 17 Feb 2002 23:37:23 -0800 (PST) Received: from pool0305.cvx21-bradley.dialup.earthlink.net ([209.179.193.50] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16ciMH-0007lt-00; Sun, 17 Feb 2002 23:37:10 -0800 Message-ID: <3C70AF1A.DC5BB938@mindspring.com> Date: Sun, 17 Feb 2002 23:36:58 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: Bruce Evans , Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout().Is copyout() MPSAFE on non-i386 archs? ) References: <20020218151249.E4728-100000@gamplex.bde.org> <3C70AB8B.96589869@mindspring.com> <200202180733.g1I7Xej12207@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > :.4nsec * 22,000 transactions/sec * 5 timestamps/transaction > := 440us/sec spent on time stamps. > : > :Up that to 2uS and... = 220ms/sec on time stamps = ~1/4 of > :all available time spent on time stamps. > : > :[ ... other stuff not really applicable; can discuss why > : off list, if necessary ... ] > > This is a meaningless exercise if you don't take into account > everything the program in question is doing on a per transaction > basis. To say that a single 2uS system call takes 25% of available > resources also infers that all your other code requires only 6uS > per transaction of overhead, which means it doesn't do very much eh? Or that the overhead associated with what it does do is very low because the code was cluefully written. ;^). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 23:42:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id 4DAA237B402; Sun, 17 Feb 2002 23:42:38 -0800 (PST) Received: from pool0305.cvx21-bradley.dialup.earthlink.net ([209.179.193.50] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16ciRR-0002ki-00; Sun, 17 Feb 2002 23:42:29 -0800 Message-ID: <3C70B05A.B41BCF9E@mindspring.com> Date: Sun, 17 Feb 2002 23:42:18 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bruce Evans Cc: Matthew Dillon , Poul-Henning Kamp , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()andcopyout().Is copyout() MPSAFE on non-i386 archs? ) References: <20020218182727.C5246-100000@gamplex.bde.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bruce Evans wrote: > > .4nsec * 22,000 transactions/sec * 5 timestamps/transaction > > = 440us/sec spent on time stamps. > > This is not long :-), but 440 msec is. > > > Up that to 2uS and... = 220ms/sec on time stamps = ~1/4 of > > all available time spent on time stamps. In practice, the number was closer to 8-10% of time burnt on the extra protection domain crossings. YMMV, depending on the hardware you end up using, but 10% was low hanging fruit, and so worth winning back. The 25% cost for the 2uS number is probably important to people running blatantly low end hardware, which is is why I calculated it from there (well, that, and the overhead I actually had was not in the set of numbers presented by other people for use in calculations ;^)). I guess if we could all afford to have $800 ServerWorks motherboards sontributing to our COGS for our products, this would be a better world... 8-) 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 17 23:45:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id 5D40137B405; Sun, 17 Feb 2002 23:45:06 -0800 (PST) Received: from pool0305.cvx21-bradley.dialup.earthlink.net ([209.179.193.50] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16ciTu-000452-00; Sun, 17 Feb 2002 23:45:03 -0800 Message-ID: <3C70B0F3.21426E19@mindspring.com> Date: Sun, 17 Feb 2002 23:44:51 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Matthew Dillon , Julian Elischer , Alfred Perlstein , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and crhold()/crfree() (was Re: gettimeofday()and copyout(). Is copyout() MPSAFE on non-i386 archs? ) References: <32873.1014013080@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > In message <3C70470E.BD2F048D@mindspring.com>, Terry Lambert writes: > > >I guess this would be read/write? > > Yes. > > >I don't think copyin() access would be the best way to > >do it, if that's the case, since the user could lie to > >the kernel that way. > > ...and consequently coredump... Heh. The "death penalty" for lying to the kernel... I can't say I don't approve. I was more worried about peple rewriting the uid in the reflected copy and getting bogus results. The degenerate case for local mail delivery, SAMBA, and other things that play musical credentials would be bad, though. 8-(. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 1: 0:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 61FF437B404; Mon, 18 Feb 2002 01:00:10 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020218090009.QRHH2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Mon, 18 Feb 2002 09:00:09 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id AAA50393; Mon, 18 Feb 2002 00:50:53 -0800 (PST) Date: Mon, 18 Feb 2002 00:50:52 -0800 (PST) From: Julian Elischer To: arch@freebsd.org, dillon@freebsd.org, jhb@freebsd.org Subject: that ucred invariant stuff. Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG John, do you REALLY want that invariant stuff to clear ucreds in user space. Matt and I discussed it and we'd really prefer to just shoot it. I'm not sure what it gives you but I'm planning on having a flag on teh thread that says when the thread is supposed to be in user mode so maybe you can test that instead if you suspect that you're accessing a thread presently in userland. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 13: 0:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 596D537B400; Mon, 18 Feb 2002 13:00:30 -0800 (PST) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id QAA13553; Mon, 18 Feb 2002 16:00:27 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g1IKxvG39928; Mon, 18 Feb 2002 15:59:57 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15473.27469.127329.884265@grasshopper.cs.duke.edu> Date: Mon, 18 Feb 2002 15:59:57 -0500 (EST) To: Matthew Dillon Cc: Poul-Henning Kamp , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: timing results for gettimeofday() (without any credential fixes) In-Reply-To: <200202162115.g1GLFSe32349@apollo.backplane.com> References: <200202162115.g1GLFSe32349@apollo.backplane.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon writes: > CURRENT 2xCPU SMP BUILD Original gettimeofday() code > > one TG running: 350000/sec > two TGs running: 55000/sec per TG (no, that isn't a type-o) > <..> > STABLE 2xCPU SMP BUILD (note gettimeofday() on stable is marked MPSAFE): > one TG running: 192402/sec > two TGs running: 95900/sec > To add 2 datapoints to that: CURRENT 1CPU UP BUILD top-of-tree as of noon EST, no patches, no WITNESS, INVARIANTS: one TG running: 145000/sec -STABLE 1CPU UP BUILD, no patches one TG running: 625000/sec So that's a factor of 4 slowdown for syscalls between -current and -stable on UP alphas... Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 13: 5:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 942EB37B404; Mon, 18 Feb 2002 13:05:41 -0800 (PST) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id QAA13712; Mon, 18 Feb 2002 16:05:41 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g1IL5Bj39945; Mon, 18 Feb 2002 16:05:11 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15473.27783.127191.533025@grasshopper.cs.duke.edu> Date: Mon, 18 Feb 2002 16:05:11 -0500 (EST) To: Matthew Dillon Cc: arch@freebsd.org, phk@critter.freebsd.dk, jhb@freebsd.org, peter@wemm.org, jake@locore.ca Subject: Re: gettimeofday() and copyout(). Is copyout() MPSAFE on non-i386 archs? In-Reply-To: <200202160500.g1G50j145638@apollo.backplane.com> References: <200202160500.g1G50j145638@apollo.backplane.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon writes: > Poul indicated today that the microtime() call that gettimeofday() uses > is MPSAFE. copyout() on I386 is MPSAFE, and from my perusal of the > other archs it appears to be MPSAFE for alpha, ia64, and sparc64 > as well. > > This would seem to indicate that Giant can be removed from the > gettimeofday() system call. I would like those familiar with the > other archs to verify that copyout() is MPSAFE on them. I am testing > Giant removal for this syscall on i386 now. I also think copyout on alpha is mpsafe. However, I have no mp alphas, so I have no way of testing this. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 13:21:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hub.freebsd.org (Postfix) with ESMTP id BD3E737B439; Mon, 18 Feb 2002 13:20:42 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020218212031.GQOM1147.rwcrmhc52.attbi.com@InterJet.elischer.org>; Mon, 18 Feb 2002 21:20:31 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA53277; Mon, 18 Feb 2002 13:19:53 -0800 (PST) Date: Mon, 18 Feb 2002 13:19:52 -0800 (PST) From: Julian Elischer To: Andrew Gallatin Cc: Matthew Dillon , Poul-Henning Kamp , arch@FreeBSD.ORG, jhb@FreeBSD.ORG, peter@wemm.org, jake@locore.ca Subject: Re: timing results for gettimeofday() (without any credential fixes) In-Reply-To: <15473.27469.127329.884265@grasshopper.cs.duke.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG turn off invariants to get the fast path on syscalls On Mon, 18 Feb 2002, Andrew Gallatin wrote: > > Matthew Dillon writes: > > CURRENT 2xCPU SMP BUILD Original gettimeofday() code > > > > one TG running: 350000/sec > > two TGs running: 55000/sec per TG (no, that isn't a type-o) > > > <..> > > STABLE 2xCPU SMP BUILD (note gettimeofday() on stable is marked MPSAFE): > > one TG running: 192402/sec > > two TGs running: 95900/sec > > > > To add 2 datapoints to that: > > CURRENT 1CPU UP BUILD top-of-tree as of noon EST, no patches, no > WITNESS, INVARIANTS: ^^ > one TG running: 145000/sec > > -STABLE 1CPU UP BUILD, no patches > one TG running: 625000/sec > > So that's a factor of 4 slowdown for syscalls between -current and > -stable on UP alphas... > > > Drew > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 16:25:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 9E16E37B405; Mon, 18 Feb 2002 16:25:45 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g1J0PdD89924; Mon, 18 Feb 2002 19:25:39 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Mon, 18 Feb 2002 19:25:38 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Julian Elischer Cc: arch@freebsd.org, dillon@freebsd.org, jhb@freebsd.org Subject: Re: that ucred invariant stuff. In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG My understanding was that John saw this primarily as a debugging aid. No doubt: #ifdef OPTIONS_NULL_TD_UCRED ... #else ... #endif would be appropriate for general use once we're sure we're handling td_ucred fine, especially given the performance difference. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services On Mon, 18 Feb 2002, Julian Elischer wrote: > > John, do you REALLY want that invariant stuff to clear ucreds in user > space. Matt and I discussed it and we'd really prefer to just shoot it. > I'm not sure what it gives you but I'm planning on having a flag on teh > thread that says when the thread is supposed to be in user mode > so maybe you can test that instead if you suspect that you're accessing > a thread presently in userland. > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 16:32:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 3EAC437B404; Mon, 18 Feb 2002 16:32:20 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g1J0WGD89979; Mon, 18 Feb 2002 19:32:16 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Mon, 18 Feb 2002 19:32:16 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: arch@FreeBSD.org Cc: jhb@FreeBSD.org Subject: John Balwdin's proc-locking patch Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG For review, John Baldwin's proc locking patch. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services ---------- Forwarded message ---------- Date: Mon, 18 Feb 2002 19:43:35 -0500 From: Jake Burkholder To: David O'Brien Cc: Robert Watson , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org Subject: Re: cvs commit: src/sys/kern kern_time.c Apparently, On Mon, Feb 18, 2002 at 02:34:13PM -0800, David O'Brien said words to the effect of; > On Mon, Feb 18, 2002 at 12:49:19PM -0500, Robert Watson wrote: > > With all do respect, I'd like to ask you to hold off for a couple of days > > until John is back in communication again from his travel to/from BSDCon. > > Over BSDCon he was talking about committing it within the next four days, > > I just spoke with JHB on the phone -- he will be out of things until > Thursday morning. However, JHB said it would be OK to pull his patches > out of his Perforce branches and post them so people could see what he > has. I'll probably need some help pulling the patches out, but I hope to > make them available tonight. I've put up a diff of the jhb_proc branch here: http://people.freebsd.org/~jake/jhb_proc.diff Jake To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 17:47:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 16EAA37B400 for ; Mon, 18 Feb 2002 17:47:38 -0800 (PST) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id UAA21456; Mon, 18 Feb 2002 20:47:37 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g1J1l7H44551; Mon, 18 Feb 2002 20:47:07 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15473.44699.351070.683358@grasshopper.cs.duke.edu> Date: Mon, 18 Feb 2002 20:47:07 -0500 (EST) To: Julian Elischer Cc: arch@FreeBSD.ORG Subject: Re: timing results for gettimeofday() (without any credential fixes) In-Reply-To: References: <15473.27469.127329.884265@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer writes: > turn off invariants to get the fast path on syscalls Not on alpha. There's no #ifdef INVARIANTS around the crfree stuff there. There probably should be. Anyway, on alpha a kernel w/INVARIANTS_SUPPORT but w/o INVARIANTS crashes on boot in the first trap: <..> Mounting root from ufs:/dev/da0a da0 at isp0 bus 0 target 1 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers (20.000MHz, offset 8, 16bit), Tagged Queueing Enabled da0: 2007MB (4110480 512 byte sectors: 255H 63S/T 255C) panic: already have a ucred panic Stopped at Debugger+0x34: zapnot v0,#0xf,a0 db> tr Debugger() at Debugger+0x34 panic() at panic+0x100 trap() at trap+0xa0 XentMM() at XentMM+0x2c --- memory management fault (from ipl 0) --- --- user mode --- db> show pcpu cpuid = 0 curthread = 0xfffffe0009a69820: pid 1 "init" curpcb = 0x829e58 fpcurthread = none idlethread = 0xfffffe0009a692c0: pid 10 "idle" ipis = 0x0 next ASN = 3 db> This may very well be the first trap the process takes. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 18:50:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 8620F37B402 for ; Mon, 18 Feb 2002 18:50:15 -0800 (PST) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id VAA23159; Mon, 18 Feb 2002 21:50:15 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g1J2njU44664; Mon, 18 Feb 2002 21:49:45 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15473.48457.11078.351769@grasshopper.cs.duke.edu> Date: Mon, 18 Feb 2002 21:49:45 -0500 (EST) To: Julian Elischer Cc: arch@FreeBSD.ORG Subject: Re: timing results for gettimeofday() (without any credential fixes) In-Reply-To: <15473.44699.351070.683358@grasshopper.cs.duke.edu> References: <15473.27469.127329.884265@grasshopper.cs.duke.edu> <15473.44699.351070.683358@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Andrew Gallatin writes: > > Julian Elischer writes: > > turn off invariants to get the fast path on syscalls > > Not on alpha. There's no #ifdef INVARIANTS around the crfree stuff > there. There probably should be. > > Anyway, on alpha a kernel w/INVARIANTS_SUPPORT but w/o INVARIANTS > crashes on boot in the first trap: Bah.. stale .o's laying around. Nevermind... Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 18:57:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by hub.freebsd.org (Postfix) with ESMTP id B9C2837B400 for ; Mon, 18 Feb 2002 18:57:22 -0800 (PST) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id g1J2vMG69676 for ; Mon, 18 Feb 2002 21:57:22 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Mon, 18 Feb 2002 21:57:22 -0500 (EST) From: Jeff Roberson To: arch@freebsd.org Subject: Prioritized bio patches. Message-ID: <20020218214128.C12686-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I have developed an extension to the bio queue interface to support priorities. It may be useful in place of the nice value related sleep that is currently in bioqdisksort. There are 6 queues in each bio queue head. The first is the default sorted queue. It is used if the bio priority (bio_prio) is BIO_PRIO_DEFAULT. Then there are 5 unsorted queues for use with different nice levels. The priority is automatically picked at insert time based on the process nice value. Un-nice processes still use the default IO queue to prevent starvation. The lower priority IO is not sorted because it may be interrupted at any time by higher priority io, thus defeating the sort. This also saves a bit of structure bloat and processing time. The main advantage over the current solution is that, if the system has spare IO cycles, nice processes can run at full speed. Also, this was implemented in such a way that it is completely transparent to users of the bioq interfaces. Since cam keeps a deep queue of IOs it may want to adjust it's priorities for the IO as well. This would prevent a possible delay for high priority IO while cam works through a queue full of low priority IO. As it is though the response time should be reasonable. This patch does not introduce any higher priority levels. It would be easy to do, but I don't see any demand for it right now. The patch is available at http://www.chesapeake.net/~jroberson/prio.patch Thanks, Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 19: 0:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id D921A37B422 for ; Mon, 18 Feb 2002 19:00:12 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020219030012.SBEC1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Tue, 19 Feb 2002 03:00:12 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id SAA54529; Mon, 18 Feb 2002 18:55:21 -0800 (PST) Date: Mon, 18 Feb 2002 18:55:20 -0800 (PST) From: Julian Elischer To: Andrew Gallatin Cc: arch@FreeBSD.ORG Subject: Re: timing results for gettimeofday() (without any credential fixes) In-Reply-To: <15473.48457.11078.351769@grasshopper.cs.duke.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG you still should add the diffs equiv to the i386 ones to get a huge speedup. On Mon, 18 Feb 2002, Andrew Gallatin wrote: > > Andrew Gallatin writes: > > > > Julian Elischer writes: > > > turn off invariants to get the fast path on syscalls > > > > Not on alpha. There's no #ifdef INVARIANTS around the crfree stuff > > there. There probably should be. > > > > Anyway, on alpha a kernel w/INVARIANTS_SUPPORT but w/o INVARIANTS > > crashes on boot in the first trap: > > Bah.. stale .o's laying around. Nevermind... > > Drew > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 19: 0:39 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id E178A37B402 for ; Mon, 18 Feb 2002 19:00:22 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020219030022.SBIE1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Tue, 19 Feb 2002 03:00:22 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id SAA54513; Mon, 18 Feb 2002 18:50:55 -0800 (PST) Date: Mon, 18 Feb 2002 18:50:54 -0800 (PST) From: Julian Elischer To: Andrew Gallatin Cc: arch@FreeBSD.ORG Subject: Re: timing results for gettimeofday() (without any credential fixes) In-Reply-To: <15473.44699.351070.683358@grasshopper.cs.duke.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Oh Sh*t I just realised that the changes that we made to i386/i386/trap.c need to be duplicated into the other architectures. (DOH!) Here is the diff we made for i386: =================================================================== RCS file: /c/ncvs/src/sys/i386/i386/trap.c,v retrieving revision 1.211 retrieving revision 1.212 diff -u -p -r1.211 -r1.212 --- src/sys/i386/i386/trap.c 2002/01/10 11:49:54 1.211 +++ src/sys/i386/i386/trap.c 2002/02/17 01:09:54 1.212 @@ -35,7 +35,7 @@ * SUCH DAMAGE. * * from: @(#)trap.c 7.4 (Berkeley) 5/13/91 - * $FreeBSD: /c/ncvs/src/sys/i386/i386/trap.c,v 1.211 2002/01/10 11:49:54 bde Exp $ + * $FreeBSD: /c/ncvs/src/sys/i386/i386/trap.c,v 1.212 2002/02/17 01:09:54 julian Exp $ */ /* @@ -256,9 +256,8 @@ trap(frame) sticks = td->td_kse->ke_sticks; td->td_frame = &frame; KASSERT(td->td_ucred == NULL, ("already have a ucred")); - PROC_LOCK(p); - td->td_ucred = crhold(p->p_ucred); - PROC_UNLOCK(p); + if (td->td_ucred != p->p_ucred) + cred_update_thread(td); switch (type) { case T_PRIVINFLT: /* privileged instruction fault */ @@ -644,10 +643,12 @@ user: userret(td, &frame, sticks); mtx_assert(&Giant, MA_NOTOWNED); userout: +#ifdef INVARIANTS mtx_lock(&Giant); crfree(td->td_ucred); mtx_unlock(&Giant); td->td_ucred = NULL; +#endif out: return; } @@ -954,9 +955,8 @@ syscall(frame) sticks = td->td_kse->ke_sticks; td->td_frame = &frame; KASSERT(td->td_ucred == NULL, ("already have a ucred")); - PROC_LOCK(p); - td->td_ucred = crhold(p->p_ucred); - PROC_UNLOCK(p); + if (td->td_ucred != p->p_ucred) + cred_update_thread(td); params = (caddr_t)frame.tf_esp + sizeof(int); code = frame.tf_eax; orig_tf_eflags = frame.tf_eflags; @@ -1099,10 +1099,12 @@ bad: */ STOPEVENT(p, S_SCX, code); +#ifdef INVARIANTS mtx_lock(&Giant); crfree(td->td_ucred); mtx_unlock(&Giant); td->td_ucred = NULL; +#endif #ifdef WITNESS if (witness_list(td)) { panic("system call %s returning with mutex(s) held\n", On Mon, 18 Feb 2002, Andrew Gallatin wrote: > > Julian Elischer writes: > > turn off invariants to get the fast path on syscalls > > Not on alpha. There's no #ifdef INVARIANTS around the crfree stuff > there. There probably should be. > > Anyway, on alpha a kernel w/INVARIANTS_SUPPORT but w/o INVARIANTS > crashes on boot in the first trap: > > <..> > Mounting root from ufs:/dev/da0a > da0 at isp0 bus 0 target 1 lun 0 > da0: Fixed Direct Access SCSI-2 device > da0: 40.000MB/s transfers (20.000MHz, offset 8, 16bit), Tagged > Queueing Enabled > da0: 2007MB (4110480 512 byte sectors: 255H 63S/T 255C) > panic: already have a ucred > panic > Stopped at Debugger+0x34: zapnot v0,#0xf,a0 > db> tr > Debugger() at Debugger+0x34 > panic() at panic+0x100 > trap() at trap+0xa0 > XentMM() at XentMM+0x2c > --- memory management fault (from ipl 0) --- > --- user mode --- > db> show pcpu > cpuid = 0 > curthread = 0xfffffe0009a69820: pid 1 "init" > curpcb = 0x829e58 > fpcurthread = none > idlethread = 0xfffffe0009a692c0: pid 10 "idle" > ipis = 0x0 > next ASN = 3 > db> > > This may very well be the first trap the process takes. > > Drew > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 19: 4:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 321F537B400 for ; Mon, 18 Feb 2002 19:04:34 -0800 (PST) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id WAA23579; Mon, 18 Feb 2002 22:04:33 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g1J343w44688; Mon, 18 Feb 2002 22:04:03 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15473.49315.654565.361497@grasshopper.cs.duke.edu> Date: Mon, 18 Feb 2002 22:04:03 -0500 (EST) To: Julian Elischer Cc: arch@FreeBSD.ORG Subject: Re: timing results for gettimeofday() (without any credential fixes) In-Reply-To: References: <15473.44699.351070.683358@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer writes: > Oh Sh*t > I just realised that the changes that we made to i386/i386/trap.c > need to be duplicated into the other architectures. (DOH!) > > Here is the diff we made for i386: Thanks.. based on Matt's patches, I looked at the cvs logs for i386/trap.c and already came up with that for alpha. I am still compiling it as we speak, as I'd like to test it prior to comitting ;) Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 19:20:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id 8092837B404 for ; Mon, 18 Feb 2002 19:20:09 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020219032009.SMYU2951.rwcrmhc53.attbi.com@InterJet.elischer.org>; Tue, 19 Feb 2002 03:20:09 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id TAA54590; Mon, 18 Feb 2002 19:02:26 -0800 (PST) Date: Mon, 18 Feb 2002 19:02:24 -0800 (PST) From: Julian Elischer To: Jeff Roberson Cc: arch@freebsd.org Subject: Re: Prioritized bio patches. In-Reply-To: <20020218214128.C12686-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG this is kind of cool Can you give any figures showing realtive effects on (say) buildworld time when there is NICE work also on the system? (I guess you must have doen it for a reason, so maybe you can give us some examples of the result) regards, julian On Mon, 18 Feb 2002, Jeff Roberson wrote: > I have developed an extension to the bio queue interface to support > priorities. It may be useful in place of the nice value related sleep > that is currently in bioqdisksort. There are 6 queues in each bio queue > head. The first is the default sorted queue. It is used if the bio > priority (bio_prio) is BIO_PRIO_DEFAULT. Then there are 5 unsorted queues > for use with different nice levels. > > The priority is automatically picked at insert time based on the process > nice value. Un-nice processes still use the default IO queue to prevent > starvation. The lower priority IO is not sorted because it may be > interrupted at any time by higher priority io, thus defeating the sort. > This also saves a bit of structure bloat and processing time. > > The main advantage over the current solution is that, if the system has > spare IO cycles, nice processes can run at full speed. Also, this was > implemented in such a way that it is completely transparent to users of > the bioq interfaces. Since cam keeps a deep queue of IOs it may want to > adjust it's priorities for the IO as well. This would prevent a possible > delay for high priority IO while cam works through a queue full of low > priority IO. As it is though the response time should be reasonable. > > This patch does not introduce any higher priority levels. It would be > easy to do, but I don't see any demand for it right now. The patch is > available at http://www.chesapeake.net/~jroberson/prio.patch > > Thanks, > Jeff > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 19:21:25 2002 Delivered-To: freebsd-arch@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 763A837B405 for ; Mon, 18 Feb 2002 19:21:21 -0800 (PST) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id WAA23965; Mon, 18 Feb 2002 22:21:21 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g1J3KoX44712; Mon, 18 Feb 2002 22:20:50 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15473.50322.930355.850554@grasshopper.cs.duke.edu> Date: Mon, 18 Feb 2002 22:20:50 -0500 (EST) To: Julian Elischer Cc: arch@FreeBSD.ORG Subject: Re: timing results for gettimeofday() (without any credential fixes) In-Reply-To: References: <15473.44699.351070.683358@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer writes: > Oh Sh*t > I just realised that the changes that we made to i386/i386/trap.c > need to be duplicated into the other architectures. (DOH!) OK, I just committed this for alpha. So now -current on alpha is only at a little worse than a factor of 2 slowdown from -stable. That sucks, but sucks a lot less than a 4x slowdown.. CURRENT 1CPU UP BUILD top-of-tree as of noon EST, no patches, no WITNESS, INVARIANTS: one TG running: 145000/sec CURRENT 1CPU UP BUILD top-of-tree as of noon EST (+ new alpha/trap.c), no patches, no WITNESS, no INVARIANTS: one TG running: 278000/sec -STABLE 1CPU UP BUILD, no patches one TG running: 625000/sec I need to get iprobe working on -current. Sigh. So little time. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 20:59: 2 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sydney.worldwide.lemis.com (sydney.lemis.com [192.109.197.198]) by hub.freebsd.org (Postfix) with ESMTP id B002437B405; Mon, 18 Feb 2002 20:58:56 -0800 (PST) Received: (from grog@localhost) by sydney.worldwide.lemis.com (8.11.6/8.9.3) id g1J2Sfh02958; Tue, 19 Feb 2002 12:58:41 +1030 (CST) (envelope-from grog) Date: Tue, 19 Feb 2002 12:58:41 +1030 From: Greg Lehey To: Matthew Dillon Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: buildworld comparison stable vs current Message-ID: <20020219125840.B2835@sydney.worldwide.lemis.com> References: <200202170818.g1H8ID067573@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200202170818.g1H8ID067573@apollo.backplane.com>; from dillon@apollo.backplane.com on Sun, Feb 17, 2002 at 12:18:13AM -0800 Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sunday, 17 February 2002 at 0:18:13 -0800, Matthew Dillon wrote: > Here are the results of a buildworld test building the -current > source tree on a SMP stable box and on a SMP current box. There was > some interest at BSDCon for a comparison just to see where we were. > Everything matches up except time and context switches, the context > switches I presume is because interrupts require context switches > in -current. > > The time difference is 1800 seconds (stable) verses 2219 seconds (current), > making stable 1.23 times faster then current at the moment. I consider > this a fairly good number for where we are, and to be expected > considering the lack of optimizations in current. Have you done any profiling? Also, how many CPUs are there? As I said on Friday, I really think we should be doing more performance measurements, including measuring what's going on at the lock level. Greg -- See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 21:34:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 07B6F37B404; Mon, 18 Feb 2002 21:34:08 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1J5Y7s58322; Mon, 18 Feb 2002 21:34:07 -0800 (PST) (envelope-from dillon) Date: Mon, 18 Feb 2002 21:34:07 -0800 (PST) From: Matthew Dillon Message-Id: <200202190534.g1J5Y7s58322@apollo.backplane.com> To: Greg Lehey Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: buildworld comparison stable vs current References: <200202170818.g1H8ID067573@apollo.backplane.com> <20020219125840.B2835@sydney.worldwide.lemis.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Have you done any profiling? Also, how many CPUs are there? : :As I said on Friday, I really think we should be doing more :performance measurements, including measuring what's going on at the :lock level. : :Greg The test boxes are DELL2550's, so 2xCPU (1.1GHz pentium III's), ECC memory, SCSI drives. A couple of things have become obvious during my own testing: * Context switching for every interrupt is expensive stable: 2467550 voluntary context switches (buildworld -j 10) current: 23879443 voluntary context switches (buildworld -j 10) buildworld was causing 10761 context switches per second on the current box, which is one context switch every 100 uS which is a serious burden. * Mutexes are expensive calls. Looking at the getuid() stats with and without Giant in userret: unpatched userret, kern.giant.ucred=1 (default) 1 process: 683K patched userret, kern.giant.ucred=1 (default) 1 process: 739K Here we have a single mutex being locked and unlocked adds 8% of overhead to the system call. This is one reason why the new ucred stuff and the timecounter modules are so nice, because they manages to entirely do away with most mutex operations in the critical path. We need to do more of that. * Context switching when a mutex is contested is extremely expensive. With two processes dueling for Giant I observed 400K calls/sec on one occassion and 200K calls/sec with the same exact setup on another occassion, which I tracked down to Giant being constantly contested and sleeping instead of spinning on the second occassion. What was really worrying here was that the results were non deterministic. Sometimes I would get the higher run rate, sometimes I would get the lower run rate, with no rhyme or reason. In anycase, just to clarify, those original numbers, 1800 vs 2219 seconds, were wrong. -current was running with default malloc options of 'J'. The correct numbers are 1800 vs 2097. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 22:27: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id E9D0337B405; Mon, 18 Feb 2002 22:26:43 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1J6QhG58642; Mon, 18 Feb 2002 22:26:43 -0800 (PST) (envelope-from dillon) Date: Mon, 18 Feb 2002 22:26:43 -0800 (PST) From: Matthew Dillon Message-Id: <200202190626.g1J6QhG58642@apollo.backplane.com> To: Robert Watson Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: John Balwdin's proc-locking patch References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :For review, John Baldwin's proc locking patch. : :Robert N M Watson FreeBSD Core Team, TrustedBSD Project :robert@fledge.watson.org NAI Labs, Safeport Network Services :... : :I've put up a diff of the jhb_proc branch here: :http://people.freebsd.org/~jake/jhb_proc.diff : :Jake Problem areas: * PROC locking for proc_rwmem(), aka: ... + PROC_LOCK(td->td_proc); return proc_rwmem(td->td_proc, &uio); This is rather odd. proc_rwmem() apparently now needs to have the proc locked on entry and will unlock it on return. Our VOP code did a lot of this sort of thing and the VOP code was massively confusing. So confusing, in fact, that there are *STILL* a couple of VOP's we haven't been able to unwind. At the very least proc_rwmem() should be documented and contain proper assertions to prevent confusion, but I'd prefer it if the proc lock could also be either internalized or externalized rather then the mix we have now. * Complexity in some of the ucred patches. For example, in kern/kern_descrip.c. The patch itself may be fine, but I would never want to see this committed at the same time everything else is. * Lack of use of Giant wrappers, such as in getuid() (though that is a fairly simple routine). The whole point of using Giant wrappers is to allow piecemeal commits which are able to test subsystem mutexes (e.g. with Witness turned on) without having to let go of Giant, thus keeping the system stable for other developers until all the piecemeal commits have been done and to allow the committer to test turning on and off Giant for a subsytem dynamically, at run-time. General comments: This patch is big, 310 KBytes, and touches a huge number of files. There are patches for half a dozen subsystems here. Now I can see something like Julian's first KSE patch needing to be separated out and committed all in one go because we have no other choice, but 85% of this particular patch by my reading could have been committed directly to the main tree in a piecemeal fashion without harming -current in the least. It is totally unecessary to accumulate so many easily-made-incremental changes off-tree and, in fact, I believe it to be detrimental to the project. It locks up large areas of code for long periods of time and creates both a synchronization hassle (even with P4) and a debugging nightmare when finally committed due to the sheer number of changes involved. - This is not a good development philosophy. I'm sorry, it just isn't. And here I am not blaming John so much as I am blaming the whole P4 philosophy. I may commit things faster, but I commit much smaller, more easily tested (and instrumented where necessary) chunks. Forward progress is made steadily. The commits go in faster because it takes far less time to test each one and if a problem arises, it is usually pretty damn obvious what piece is to blame. In our entire history there are only a handful of things that could be said to legitimately require a separate branch to develop. I can only think of two right off the bat: CAM and KSEs. In anycase, the areas of the patch related to what I was testing are almost identical, except I include instrumented Giant code and John simply removes Giant entirely. I prefer my code, frankly. I do not think it is a good idea to remove Giant entirely for even the simpler ucred/proc-related system calls at this time. That is my position. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 18 23: 4:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id 3D81237B416; Mon, 18 Feb 2002 23:04:28 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id 16E9CAE32B; Mon, 18 Feb 2002 23:04:28 -0800 (PST) Date: Mon, 18 Feb 2002 23:04:28 -0800 From: Alfred Perlstein To: Matthew Dillon Cc: Robert Watson , arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: John Balwdin's proc-locking patch Message-ID: <20020219070428.GF12136@elvis.mu.org> References: <200202190626.g1J6QhG58642@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200202190626.g1J6QhG58642@apollo.backplane.com> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Matthew Dillon [020218 22:27] wrote: > It is totally unecessary to accumulate so many easily-made-incremental > changes off-tree and, in fact, I believe it to be detrimental to the > project. It locks up large areas of code for long periods of time > and creates both a synchronization hassle (even with P4) and a > debugging nightmare when finally committed due to the sheer number > of changes involved. [snip] > > That is my position. I've felt the same way for a while. -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 0:34:16 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 80B1737B41A; Tue, 19 Feb 2002 00:34:07 -0800 (PST) Received: from peter3.wemm.org ([12.232.27.13]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020219083407.UTPL2626.rwcrmhc51.attbi.com@peter3.wemm.org>; Tue, 19 Feb 2002 08:34:07 +0000 Received: from overcee.wemm.org (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id g1J8Y6s36011; Tue, 19 Feb 2002 00:34:06 -0800 (PST) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.wemm.org (Postfix) with ESMTP id B351E3A9A; Tue, 19 Feb 2002 00:34:06 -0800 (PST) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Alfred Perlstein Cc: Matthew Dillon , Robert Watson , arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: John Balwdin's proc-locking patch In-Reply-To: <20020219070428.GF12136@elvis.mu.org> Date: Tue, 19 Feb 2002 00:34:06 -0800 From: Peter Wemm Message-Id: <20020219083406.B351E3A9A@overcee.wemm.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Alfred Perlstein wrote: > * Matthew Dillon [020218 22:27] wrote: > > > It is totally unecessary to accumulate so many easily-made-incremental > > changes off-tree and, in fact, I believe it to be detrimental to the > > project. It locks up large areas of code for long periods of time > > and creates both a synchronization hassle (even with P4) and a > > debugging nightmare when finally committed due to the sheer number > > of changes involved. > [snip] > > > > That is my position. > > I've felt the same way for a while. Nobody is disputing that. It would have been better if John had committed it piecemeal, but it didn't work out that way. I doubt he'll fall for this trap again. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 0:51:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 41E2737B402; Tue, 19 Feb 2002 00:51:39 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id TAA21747; Tue, 19 Feb 2002 19:51:34 +1100 Date: Tue, 19 Feb 2002 19:51:34 +1100 (EST) From: Bruce Evans X-X-Sender: To: Matthew Dillon Cc: Greg Lehey , , Subject: Re: buildworld comparison stable vs current In-Reply-To: <200202190534.g1J5Y7s58322@apollo.backplane.com> Message-ID: <20020219194134.V1374-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 18 Feb 2002, Matthew Dillon wrote: > The test boxes are DELL2550's, so 2xCPU (1.1GHz pentium III's), > ECC memory, SCSI drives. > > A couple of things have become obvious during my own testing: > > * Context switching for every interrupt is expensive > > stable: 2467550 voluntary context switches (buildworld -j 10) > current: 23879443 voluntary context switches (buildworld -j 10) These mostly aren't for interrupts (unless they are for IPIs). I get: 393580 voluntary context switches 355294 involuntary context switches for UP on an Athlon1600 (for a makeworld which took 1573 seconds in -current), i.e., only 1/60 as many voluntary context switches. I think most of them are for context switches to idle because Giant is held. The idle process is more of a mistake than I first thought. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 1:12:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id B69A137B400; Tue, 19 Feb 2002 01:12:50 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1J9CnB59744; Tue, 19 Feb 2002 01:12:49 -0800 (PST) (envelope-from dillon) Date: Tue, 19 Feb 2002 01:12:49 -0800 (PST) From: Matthew Dillon Message-Id: <200202190912.g1J9CnB59744@apollo.backplane.com> To: Bruce Evans Cc: Greg Lehey , , Subject: Re: buildworld comparison stable vs current References: <20020219194134.V1374-100000@gamplex.bde.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :> stable: 2467550 voluntary context switches (buildworld -j 10) :> current: 23879443 voluntary context switches (buildworld -j 10) : :These mostly aren't for interrupts (unless they are for IPIs). I get: : : 393580 voluntary context switches : 355294 involuntary context switches : :for UP on an Athlon1600 (for a makeworld which took 1573 seconds in :-current), i.e., only 1/60 as many voluntary context switches. I think :most of them are for context switches to idle because Giant is held. :The idle process is more of a mistake than I first thought. : :Bruce Ah! Ok, that makes a lot more sense. There is work in progress to patch mutex sleep locks to spin in this situation. My attempt today didn't work out too well but apparently John has some code to do it as well that hasn't been incorporated. I believe it will help the problem but I'm not sure if it will solve it. I think there may be something else going on because the context switch rate goes from 230/sec on an idle box to almost 2000/sec if I am running a single copy of a syscall intensive program, like TG (gettimeofday() loop) or TU (getuid() loop). Not even two processes... just one will do it. A for(;;); loop does not cause any increase. I'm at a loss at the moment. In anycase, it's definitely costly. I don't think the idle process is directly responsible but it could be getting switched to as a side effect of something else. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 2:56:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.nsu.ru (mx.nsu.ru [193.124.215.71]) by hub.freebsd.org (Postfix) with ESMTP id AA9AA37B404; Tue, 19 Feb 2002 02:56:08 -0800 (PST) Received: from regency.nsu.ru ([193.124.210.26] helo=cytherea.weblab.nsu.ru) by mail.nsu.ru with esmtp (Exim 3.20 #1) id 16d7wJ-0006Hi-00; Tue, 19 Feb 2002 16:56:03 +0600 Received: (from danfe@localhost) by cytherea.weblab.nsu.ru (8.11.6/8.11.6) id g1JAuU965842; Tue, 19 Feb 2002 16:56:30 +0600 (NOVT) (envelope-from danfe) Date: Tue, 19 Feb 2002 16:56:30 +0600 From: Alexey Dokuchaev To: arch@freebsd.org Cc: ipfw@freebsd.org Subject: Improvements to ipfw code (followup) Message-ID: <20020219165630.A62749@cytherea.weblab.nsu.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, Back in 1997, an email was sent to hackers@ about some substantial firewall code improvements, along with a patch, by Julian Assange . A PR (misc/2386) was then filled, but marked 'closed' shortly after submission due to 'Misfiled PR' reason. It seems to never raise any interest afterwards, despite the fact that this work definitely worth considering. I will forward original mail at the end for those who's interested. My particular interest in this comes from a fact that uid/gid-based IPFW filtering only works for outgoing connections, which is a neat thing of course. However, to be able to provide any service, I need to allow incoming connections as well, and this is where I got somewhat disappointed: I cannot control who's bind()'ing to whatever port (if outside setup connections are allowed), and if, say, for whatever reason my cvsupd (or ircd, or quaked) exits, any malicious user process can issue bind() to the [freed] unprivileged port. One might say this is not a big deal, since servers tend to restart themselves in case of any failure, however, for example, FTP passive mode requires setup connections allowed in certain port range, and I really want only ftp user to be able to bind() to those ports. At present, there is no way in IPFW to open ports for specific user/group only, while Julian's patch seems to solve the problem. Time to revise this stuff again? :-) The URL Julian gives in his email is no longer valid, but his patches are in PR misc/2386, and also can be found at ftp://regency.nsu.ru/tmp/ipfw.diff. Sincerely, Alexey Dokuchaev ------ Forwarded message ------ Date: Tue, 7 Jan 1997 07:01:16 +1100 (EST) From: proff@suburbia.net To: hackers@freebsd.org, security@freebsd.org Subject: new firewall code [uid/gid/bind() etc] Message-ID: <19970106200116.16168.qmail@suburbia.net> I tried posting the patches but, at 55k, it seems majordumbo has (silently) rejected them. You may find them at: ftp://suburbia.net/tmp/ipfw.diff My "socket credentials" patches allow you to: punch wormholes, or restrict access to the IPPORT_RESERVED space, or restrict access to bind() altogether based on: (a) uid (b) gid (including secondary groups) (c) port (d) protocol (e) interface And more importantly: Restrict access to packets being sent/received on any socket based on: (a) the packet (per normal ipfw rules) (b) uid (c) gid (including secondary groups) The former permits constructs like: /* let uid sendmail bind to port 25 */ # ipfw add accept wormhole on tcp from any 25 to any uid sendmail bind /* only let inetd bind - we presume inetd still needs to run as root for uid switching when forking off clients */ # addgroup inetd # chgrp inetd /usr/sbin/inetd # chmod 2700 /usr/sbin/inetd # killall inetd # ipfw add accept all from any to any bind gid inetd uid root # /* default policy is to deny bind */ /* keep those without security clearance out of secret network */ # ipfw add accept all from any to any via ed0 gid secret # ipfw add deny all from any to any via ed0 gid any Loging has also been enhanced: # ipfw add 60000 accept log all from any to any bind /* example of named starting up */ ipfw: 5000 Allow TCP 0.0.0.0:53 0.0.0.0:0 uid 67 gid 0 pid 1280 bind ipfw: 5000 Allow UDP 203.4.184.222:53 0.0.0.0:0 via ed0 uid 67 gid 0 pid 1280 bind ipfw: 5000 Allow UDP 203.4.184.217:53 0.0.0.0:0 via ppp0 uid 67 gid 0 pid 1280 bind ipfw: 5000 Allow UDP 127.0.0.1:53 0.0.0.0:0 via lo0 uid 67 gid 0 pid 1280 bind ipfw: 5000 Allow UDP 0.0.0.0:53 0.0.0.0:0 uid 67 gid 0 pid 1280 bind Cheers, Julian ------ End of forwarded message ------ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 7: 5: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 57EA137B419; Tue, 19 Feb 2002 07:04:51 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g1JF4hD01145; Tue, 19 Feb 2002 10:04:44 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 19 Feb 2002 10:04:43 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org Reply-To: Robert Watson To: Matthew Dillon Cc: arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: John Balwdin's proc-locking patch In-Reply-To: <200202190626.g1J6QhG58642@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thanks for the technical comments -- I'm sure John will appreciate them once he's connected again. I'll respond briefly to the more philosophical comments. On Mon, 18 Feb 2002, Matthew Dillon wrote: > > * Complexity in some of the ucred patches. For example, in > kern/kern_descrip.c. The patch itself may be fine, but I would > never want to see this committed at the same time everything else > is. I'm not sure that was his intent. I'd imagine he'd bring it in in phases, much the same as anyone would with a large and complex patch. That said, apparently he's been running it on several i386, Alpha, etc, boxes for the past couple of weeks as burn-in. I'd like to see specific components of it brought in earlier, including the ucred components, and signal-related work, for the obvious reasons. > * Lack of use of Giant wrappers, such as in getuid() (though that > is a fairly simple routine). The whole point of using Giant > wrappers is to allow piecemeal commits which are able to test > subsystem mutexes (e.g. with Witness turned on) without having > to let go of Giant, thus keeping the system stable for other > developers until all the piecemeal commits have been done and > to allow the committer to test turning on and off Giant for a > subsytem dynamically, at run-time. As I've said already, I'm fine with you going ahead and committing the instrumented Giant locking. It will make the process less painful for all involved. > General comments: This patch is big, 310 KBytes, and touches a huge > number of files. There are patches for half a dozen subsystems here. > Now I can see something like Julian's first KSE patch needing to be > separated out and committed all in one go because we have no other > choice, but 85% of this particular patch by my reading could have been > committed directly to the main tree in a piecemeal fashion without > harming -current in the least. Probably true, and we should encourage John to do so in the future. That said, you need to understand that this presentation of the patch was presumably not done in the form he would consider ideal, it was done so as to provide broad exposure while he is briefly unavailable. Part of the benefit to John's approach is that he was able to get a whole system view, understanding how locking interacted with the system at all relevant points, rather than selecting a locking strategy and discovering the implications gradually, possibly having to change it later. As a result, he appears to have a tree which is pretty much correctly locked, with substantial testing. > It is totally unecessary to accumulate so many easily-made-incremental > changes off-tree and, in fact, I believe it to be detrimental to the > project. It locks up large areas of code for long periods of time > and creates both a synchronization hassle (even with P4) and a > debugging nightmare when finally committed due to the sheer number > of changes involved. I agree that he should move things in more agressively; that doesn't invalidate the work he has done, however. That said, the technique of maintaining work in p4 and gradually migrating it into the main tree can be a very effective one. It allows you to use a lighter weight committing process to store works-in-progress without interfering with the main tree. For example, in the TrustedBSD branches, we can maintain version control for things we're not sure we even want to commit, but do want to experiment with. We can get version control from an early point in the work, where fundamental assumptions are changing. For example, we're doing a lot of work on the TrustedBSD branches that we'll only migrate in slowly, and in chunks, at a later date. We are still getting a grasp of the full implications of the work, and it would be premature to bring much of it in when we know there will be substantial changes. > This is not a good development philosophy. I'm sorry, it just isn't. > And here I am not blaming John so much as I am blaming the whole > P4 philosophy. I may commit things faster, but I commit much smaller, > more easily tested (and instrumented where necessary) chunks. Forward > progress is made steadily. The commits go in faster because it takes > far less time to test each one and if a problem arises, it is usually > pretty damn obvious what piece is to blame. Not all work can be done quickly and in small commits. Sometimes it's necessary to experiment, or work over a longer period of time. That's one reason why KSE has been done so effectively in a branch: having it in p4 provides more opportunities for collaboration, early review, etc. Julian could get feedback on his work during the development process, rather than as it hit the main tree. Using P4 allowed him to track the main tree efficiently while he did this--something that CVS has always made very difficult. > In our entire history there are only a handful of things that could > be said to legitimately require a separate branch to develop. I > can only think of two right off the bat: CAM and KSEs. Part of that is that we haven't had effective tools to do that in the past, and that we're still learning how to use P4 as a tool. That doesn't invalidate the model: in fact, I'd say that CAM and KSE both provide a strong rationale to use the model, due to their success. For a variety of reasons, less work has been done on SMPng than originally planned: you can blame the economy, specific business decisions, lack of time where time was expected, etc, etc. However, it can still have a structured and organized development process. A quick glance at the SMPng web page suggests that we have exactly that: http://www.FreeBSD.org/smp/ > In anycase, the areas of the patch related to what I was testing > are almost identical, except I include instrumented Giant code > and John simply removes Giant entirely. I prefer my code, frankly. > I do not think it is a good idea to remove Giant entirely for even > the simpler ucred/proc-related system calls at this time. > > That is my position. Then commit your instrumented versions of the giant locking. I specifically excluded that from the list of changes I asked you not to commit, precisely because I agree with many if not all of the technical points you've made. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 8:32:22 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailserver.cli.di.unipi.it (crudelia.cli.di.unipi.it [131.114.11.37]) by hub.freebsd.org (Postfix) with ESMTP id 5758D37B400 for ; Tue, 19 Feb 2002 08:32:18 -0800 (PST) Organization: Centro di Calcolo - Dipartimento di Informatica di Pisa - Italy Received: from fire.cli.di.unipi.it (fire-ext.cli.di.unipi.it [131.114.11.52]) by mailserver.cli.di.unipi.it (8.11.6/8.11.2) with SMTP id g1JGWGY19725 for ; Tue, 19 Feb 2002 17:32:16 +0100 Received: (qmail 32665 invoked by uid 7794); 19 Feb 2002 16:32:13 -0000 Received: from well13.cli.di.unipi.it(131.114.11.223) via SMTP by hub.FreeBSD.org, id smtpda32656; Tue Feb 19 17:32:10 2002 Received: (from divizio@localhost) by well13.cli.di.unipi.it (8.11.6/8.11.2) id g1JGW6w15844 for arch@FreeBSD.ORG; Tue, 19 Feb 2002 17:32:06 +0100 Date: Tue, 19 Feb 2002 17:32:06 +0100 From: Luca Di Vizio To: arch@FreeBSD.ORG Subject: Improvements to "mtree" Message-ID: <20020219163205.GA15833@well13.cli.di.unipi.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi all, Is there a project to substitute mtree with something better? For example if I exclude bind from the buildworld process, mtree creates anyway /etc/named ... Wouldn'it better to have something like "conditional installation"? Sorry for the bad english and the bad idea :) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 8:40:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id B522037B428; Tue, 19 Feb 2002 08:40:00 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g1JGd9D02838; Tue, 19 Feb 2002 11:39:09 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 19 Feb 2002 11:39:08 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Alexey Dokuchaev Cc: arch@freebsd.org, ipfw@freebsd.org Subject: Re: Improvements to ipfw code (followup) In-Reply-To: <20020219165630.A62749@cytherea.weblab.nsu.ru> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Many of these look interesting. However, it's worth noting that most of them are broken with SSH port forwarding, due to sshd binding ports as root, as opposed to as the authenticated credential. This has presented a problem for us for the MAC code also, and requires substantial re-working of the sshd code. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services On Tue, 19 Feb 2002, Alexey Dokuchaev wrote: > Hello, > > Back in 1997, an email was sent to hackers@ about some substantial firewall code improvements, > along with a patch, by Julian Assange . A PR (misc/2386) was then > filled, but marked 'closed' shortly after submission due to 'Misfiled PR' reason. It seems to > never raise any interest afterwards, despite the fact that this work definitely worth considering. > > I will forward original mail at the end for those who's interested. My particular interest in > this comes from a fact that uid/gid-based IPFW filtering only works for outgoing connections, > which is a neat thing of course. However, to be able to provide any service, I need to allow > incoming connections as well, and this is where I got somewhat disappointed: I cannot control > who's bind()'ing to whatever port (if outside setup connections are allowed), and if, say, for > whatever reason my cvsupd (or ircd, or quaked) exits, any malicious user process can issue bind() > to the [freed] unprivileged port. One might say this is not a big deal, since servers tend to > restart themselves in case of any failure, however, for example, FTP passive mode requires setup > connections allowed in certain port range, and I really want only ftp user to be able to bind() > to those ports. At present, there is no way in IPFW to open ports for specific user/group only, > while Julian's patch seems to solve the problem. > > Time to revise this stuff again? :-) > > The URL Julian gives in his email is no longer valid, but his patches are in PR misc/2386, and > also can be found at ftp://regency.nsu.ru/tmp/ipfw.diff. > > Sincerely, > Alexey Dokuchaev > > ------ Forwarded message ------ > Date: Tue, 7 Jan 1997 07:01:16 +1100 (EST) > From: proff@suburbia.net > To: hackers@freebsd.org, security@freebsd.org > Subject: new firewall code [uid/gid/bind() etc] > Message-ID: <19970106200116.16168.qmail@suburbia.net> > > I tried posting the patches but, at 55k, it seems majordumbo has > (silently) rejected them. You may find them at: > > ftp://suburbia.net/tmp/ipfw.diff > > My "socket credentials" patches allow you to: > > punch wormholes, or restrict access to the IPPORT_RESERVED space, or > restrict access to bind() altogether based on: > > (a) uid > (b) gid (including secondary groups) > (c) port > (d) protocol > (e) interface > > And more importantly: > > Restrict access to packets being sent/received on any socket based on: > > (a) the packet (per normal ipfw rules) > (b) uid > (c) gid (including secondary groups) > > The former permits constructs like: > > /* let uid sendmail bind to port 25 */ > # ipfw add accept wormhole on tcp from any 25 to any uid sendmail bind > > /* only let inetd bind - we presume inetd still needs to run as root > for uid switching when forking off clients */ > > # addgroup inetd > # chgrp inetd /usr/sbin/inetd > # chmod 2700 /usr/sbin/inetd > # killall inetd > # ipfw add accept all from any to any bind gid inetd uid root > # /* default policy is to deny bind */ > > /* keep those without security clearance out of secret network */ > # ipfw add accept all from any to any via ed0 gid secret > # ipfw add deny all from any to any via ed0 gid any > > Loging has also been enhanced: > > # ipfw add 60000 accept log all from any to any bind > /* example of named starting up */ > > ipfw: 5000 Allow TCP 0.0.0.0:53 0.0.0.0:0 uid 67 gid 0 pid 1280 bind > ipfw: 5000 Allow UDP 203.4.184.222:53 0.0.0.0:0 via ed0 uid 67 gid 0 pid 1280 bind > ipfw: 5000 Allow UDP 203.4.184.217:53 0.0.0.0:0 via ppp0 uid 67 gid 0 pid 1280 bind > ipfw: 5000 Allow UDP 127.0.0.1:53 0.0.0.0:0 via lo0 uid 67 gid 0 pid 1280 bind > ipfw: 5000 Allow UDP 0.0.0.0:53 0.0.0.0:0 uid 67 gid 0 pid 1280 bind > > Cheers, > Julian > > ------ End of forwarded message ------ > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 8:41:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 3C3BB37B402; Tue, 19 Feb 2002 08:40:39 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g1JGe4D02855; Tue, 19 Feb 2002 11:40:04 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 19 Feb 2002 11:40:03 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Alexey Dokuchaev Cc: arch@freebsd.org, ipfw@freebsd.org Subject: Re: Improvements to ipfw code (followup) In-Reply-To: <20020219165630.A62749@cytherea.weblab.nsu.ru> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Just as a slight follow-up I should have included in my earlier e-mail: the merging of ucred and pcred should make this patch now be able to support real and saved uids/gids as well as effective uids/gids, meaning that it can be used to also restrict setuid applications such as ping. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services On Tue, 19 Feb 2002, Alexey Dokuchaev wrote: > Hello, > > Back in 1997, an email was sent to hackers@ about some substantial firewall code improvements, > along with a patch, by Julian Assange . A PR (misc/2386) was then > filled, but marked 'closed' shortly after submission due to 'Misfiled PR' reason. It seems to > never raise any interest afterwards, despite the fact that this work definitely worth considering. > > I will forward original mail at the end for those who's interested. My particular interest in > this comes from a fact that uid/gid-based IPFW filtering only works for outgoing connections, > which is a neat thing of course. However, to be able to provide any service, I need to allow > incoming connections as well, and this is where I got somewhat disappointed: I cannot control > who's bind()'ing to whatever port (if outside setup connections are allowed), and if, say, for > whatever reason my cvsupd (or ircd, or quaked) exits, any malicious user process can issue bind() > to the [freed] unprivileged port. One might say this is not a big deal, since servers tend to > restart themselves in case of any failure, however, for example, FTP passive mode requires setup > connections allowed in certain port range, and I really want only ftp user to be able to bind() > to those ports. At present, there is no way in IPFW to open ports for specific user/group only, > while Julian's patch seems to solve the problem. > > Time to revise this stuff again? :-) > > The URL Julian gives in his email is no longer valid, but his patches are in PR misc/2386, and > also can be found at ftp://regency.nsu.ru/tmp/ipfw.diff. > > Sincerely, > Alexey Dokuchaev > > ------ Forwarded message ------ > Date: Tue, 7 Jan 1997 07:01:16 +1100 (EST) > From: proff@suburbia.net > To: hackers@freebsd.org, security@freebsd.org > Subject: new firewall code [uid/gid/bind() etc] > Message-ID: <19970106200116.16168.qmail@suburbia.net> > > I tried posting the patches but, at 55k, it seems majordumbo has > (silently) rejected them. You may find them at: > > ftp://suburbia.net/tmp/ipfw.diff > > My "socket credentials" patches allow you to: > > punch wormholes, or restrict access to the IPPORT_RESERVED space, or > restrict access to bind() altogether based on: > > (a) uid > (b) gid (including secondary groups) > (c) port > (d) protocol > (e) interface > > And more importantly: > > Restrict access to packets being sent/received on any socket based on: > > (a) the packet (per normal ipfw rules) > (b) uid > (c) gid (including secondary groups) > > The former permits constructs like: > > /* let uid sendmail bind to port 25 */ > # ipfw add accept wormhole on tcp from any 25 to any uid sendmail bind > > /* only let inetd bind - we presume inetd still needs to run as root > for uid switching when forking off clients */ > > # addgroup inetd > # chgrp inetd /usr/sbin/inetd > # chmod 2700 /usr/sbin/inetd > # killall inetd > # ipfw add accept all from any to any bind gid inetd uid root > # /* default policy is to deny bind */ > > /* keep those without security clearance out of secret network */ > # ipfw add accept all from any to any via ed0 gid secret > # ipfw add deny all from any to any via ed0 gid any > > Loging has also been enhanced: > > # ipfw add 60000 accept log all from any to any bind > /* example of named starting up */ > > ipfw: 5000 Allow TCP 0.0.0.0:53 0.0.0.0:0 uid 67 gid 0 pid 1280 bind > ipfw: 5000 Allow UDP 203.4.184.222:53 0.0.0.0:0 via ed0 uid 67 gid 0 pid 1280 bind > ipfw: 5000 Allow UDP 203.4.184.217:53 0.0.0.0:0 via ppp0 uid 67 gid 0 pid 1280 bind > ipfw: 5000 Allow UDP 127.0.0.1:53 0.0.0.0:0 via lo0 uid 67 gid 0 pid 1280 bind > ipfw: 5000 Allow UDP 0.0.0.0:53 0.0.0.0:0 uid 67 gid 0 pid 1280 bind > > Cheers, > Julian > > ------ End of forwarded message ------ > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 9:55: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from eumenes.hosting.pacbell.net (eumenes.hosting.pacbell.net [216.100.98.24]) by hub.freebsd.org (Postfix) with ESMTP id 3CE4E37B402; Tue, 19 Feb 2002 09:54:55 -0800 (PST) Received: from misha1 (adsl-64-172-38-74.dsl.snfc21.pacbell.net [64.172.38.74]) by eumenes.hosting.pacbell.net id MAA13437; Tue, 19 Feb 2002 12:54:52 -0500 (EST) [ConcentricHost SMTP Relay 1.14] Message-ID: <004f01c1b96b$a37d1f50$b210a8c0@netscreen5> Reply-To: "mike varga" From: "mike varga" To: "Michael Smith" Cc: References: <200202170813.g1H8DAV01750@mass.dis.org> Subject: Re: reverse vtophys() Date: Tue, 19 Feb 2002 09:34:04 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The concept is valid in Linux, there is a function that does what I am asking. bus_to_virtual(). The page is locked in memory. Therefor there is one valid page backed by the physical address that corresponds to the virtual address that I am interested in retrieving? ----- Original Message ----- From: "Michael Smith" To: "mike varga" Cc: Sent: Sunday, February 17, 2002 12:13 AM Subject: Re: reverse vtophys() > > > > Is there a function that exists=20 > > that converts a physical address to=20 > > a process's virtual address? > > No; the concept isn't valid. > > - A given physical address may be mapped in more than one virtual > location within the process' address space. > > - There is an unavoidable race with unwired memory where a physical > page may "move" as it is used to back more than a single physical > page, resulting in a bad answer. > > Perhaps you should explain what it is that you're trying to do, rather > than narrowing down to what you think is the solution... > > -- > To announce that there must be no criticism of the president, > or that we are to stand by the president, right or wrong, is not > only unpatriotic and servile, but is morally treasonable to > the American public. - Theodore Roosevelt > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 13:27:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from orthanc.ab.ca (orthanc.ab.ca [216.123.203.186]) by hub.freebsd.org (Postfix) with ESMTP id 9B03737B41A for ; Tue, 19 Feb 2002 13:27:54 -0800 (PST) Received: from orthanc.ab.ca (localhost.orthanc.ab.ca [127.0.0.1]) by orthanc.ab.ca (8.11.6/8.11.6) with ESMTP id g1JLRie56618; Tue, 19 Feb 2002 14:27:44 -0700 (MST) (envelope-from lyndon@orthanc.ab.ca) Message-Id: <200202192127.g1JLRie56618@orthanc.ab.ca> From: Lyndon Nerenberg Organization: The Frobozz Magic Homing Pigeon Company To: Luca Di Vizio Cc: arch@FreeBSD.ORG Subject: Re: Improvements to "mtree" In-reply-to: Your message of "Tue, 19 Feb 2002 17:32:06 +0100." <20020219163205.GA15833@well13.cli.di.unipi.it> X-Mailer: mh-e 5.0.92; MH 6.8.4; Emacs 21.1 Date: Tue, 19 Feb 2002 14:27:44 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "Luca" == Luca Di Vizio writes: Luca> Hi all, Is there a project to substitute mtree with something Luca> better? For example if I exclude bind from the buildworld Luca> process, mtree creates anyway /etc/named ... Wouldn'it better Luca> to have something like "conditional installation"? Create seperate mtree files for the components under make.conf NO_* control. You can then wrap the Makefile mtree invocations for the NO_* components in conditionals: # fragment from /usr/src/etc/Makefile MTREE= BSD.include.dist BSD.local.dist BSD.root.dist BSD.usr.dist \ BSD.var.dist BSD.x11.dist BSD.x11-4.dist .if !defined(NO_UUCP) MTREE+= BSD.uucp.dist .endif .if !defined(NO_SENDMAIL) MTREE+= BSD.sendmail.dist .endif .if !defined(NO_NAMED) MTREE+= BSD.named.dist .endif --lyndon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 14:21:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by hub.freebsd.org (Postfix) with ESMTP id 934E337B402 for ; Tue, 19 Feb 2002 14:21:50 -0800 (PST) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id g1JMLkL52532; Tue, 19 Feb 2002 17:21:46 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Tue, 19 Feb 2002 17:21:46 -0500 (EST) From: Jeff Roberson To: Julian Elischer Cc: arch@freebsd.org Subject: Re: Prioritized bio patches. (Updated patch) In-Reply-To: Message-ID: <20020219171504.T12686-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG First of all, I updated the patch. When I merged it in from our sources I missed a one line change that fixed a race condition. I also changed the priority level of NORMAL to 6 so that I could avoid all of the -1's to index the low priority queue. Secondly, I ran a simple test of a kernel compile. The test system has one disk. I did a dd of /dev/zero to a file in a users home directory with a nice of 20 while doing a kernel compile. The original compile took 11 minutes and 32 seconds. The compile with the dd going took 15 minutes and 12 seconds. I originally did this work for VOD server. The idea being that the VOD data was guaranteed and the rest of the system would just have to wait. This was not based on process nice values. Each sub system had a hard coded priority, that in some cases correlated to a different sorting algorithm. When I saw the background fsck work I realized that this could be beneficial to everyone if it was tied to nice. Jeff On Mon, 18 Feb 2002, Julian Elischer wrote: > this is kind of cool > > Can you give any figures showing realtive effects on (say) buildworld > time when there is NICE work also on the system? > (I guess you must have doen it for a reason, so maybe you can give us some > examples of the result) > > regards, julian > > > On Mon, 18 Feb 2002, Jeff Roberson wrote: > > > I have developed an extension to the bio queue interface to support > > priorities. It may be useful in place of the nice value related sleep > > that is currently in bioqdisksort. There are 6 queues in each bio queue > > head. The first is the default sorted queue. It is used if the bio > > priority (bio_prio) is BIO_PRIO_DEFAULT. Then there are 5 unsorted queues > > for use with different nice levels. > > > > The priority is automatically picked at insert time based on the process > > nice value. Un-nice processes still use the default IO queue to prevent > > starvation. The lower priority IO is not sorted because it may be > > interrupted at any time by higher priority io, thus defeating the sort. > > This also saves a bit of structure bloat and processing time. > > > > The main advantage over the current solution is that, if the system has > > spare IO cycles, nice processes can run at full speed. Also, this was > > implemented in such a way that it is completely transparent to users of > > the bioq interfaces. Since cam keeps a deep queue of IOs it may want to > > adjust it's priorities for the IO as well. This would prevent a possible > > delay for high priority IO while cam works through a queue full of low > > priority IO. As it is though the response time should be reasonable. > > > > This patch does not introduce any higher priority levels. It would be > > easy to do, but I don't see any demand for it right now. The patch is > > available at http://www.chesapeake.net/~jroberson/prio.patch > > > > Thanks, > > Jeff > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-arch" in the body of the message > > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 14:40:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 214D937B404 for ; Tue, 19 Feb 2002 14:40:09 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020219224008.LOBK2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Tue, 19 Feb 2002 22:40:08 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA58957; Tue, 19 Feb 2002 14:32:07 -0800 (PST) Date: Tue, 19 Feb 2002 14:32:06 -0800 (PST) From: Julian Elischer To: Jeff Roberson Cc: arch@freebsd.org Subject: Re: Prioritized bio patches. (Updated patch) In-Reply-To: <20020219171504.T12686-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 19 Feb 2002, Jeff Roberson wrote: > > First of all, I updated the patch. When I merged it in from our sources I > missed a one line change that fixed a race condition. I also changed the > priority level of NORMAL to 6 so that I could avoid all of the -1's to > index the low priority queue. > > Secondly, I ran a simple test of a kernel compile. The test system has > one disk. I did a dd of /dev/zero to a file in a users home directory > with a nice of 20 while doing a kernel compile. The original compile took > 11 minutes and 32 seconds. The compile with the dd going took 15 minutes > and 12 seconds. What did it take with the dd going and without your changes? How did it work out for the VOD system? > > I originally did this work for VOD server. The idea being that the VOD > data was guaranteed and the rest of the system would just have to wait. > This was not based on process nice values. Each sub system had a hard > coded priority, that in some cases correlated to a different sorting > algorithm. When I saw the background fsck work I realized that this could > be beneficial to everyone if it was tied to nice. > > Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 15: 2:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 0709937B405; Tue, 19 Feb 2002 15:02:10 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1JN29164102; Tue, 19 Feb 2002 15:02:09 -0800 (PST) (envelope-from dillon) Date: Tue, 19 Feb 2002 15:02:09 -0800 (PST) From: Matthew Dillon Message-Id: <200202192302.g1JN29164102@apollo.backplane.com> To: Robert Watson Cc: Julian Elischer , arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: that ucred invariant stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :My understanding was that John saw this primarily as a debugging aid. No :doubt: : :#ifdef OPTIONS_NULL_TD_UCRED : ... :#else : ... :#endif : :would be appropriate for general use once we're sure we're handling :td_ucred fine, especially given the performance difference. : :Robert N M Watson FreeBSD Core Team, TrustedBSD Project :robert@fledge.watson.org NAI Labs, Safeport Network Services Julian, should I go ahead and do this or would you like to? I already have the patches ready, I just have to change a bunch of #if 0's into #ifdef OPTIONS_TD_UCRED_DEBUG, for all architectures. I suggest something like OPTIONS_TD_UCRED_DEBUG. Turning on this option would clear out the ucred in the thread after each syscall (effectively the INVARIANTS code that is there now). Not specifying this option would give us the cache behavior (what Julian's code was intended to do). -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 15:40:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id C8AFF37B41B; Tue, 19 Feb 2002 15:40:19 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020219234019.RNRO2951.rwcrmhc53.attbi.com@InterJet.elischer.org>; Tue, 19 Feb 2002 23:40:19 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA59172; Tue, 19 Feb 2002 15:27:33 -0800 (PST) Date: Tue, 19 Feb 2002 15:27:31 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: Robert Watson , arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: that ucred invariant stuff. In-Reply-To: <200202192302.g1JN29164102@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG yes please I could also do it.. my prefered behaviour however is to just rip it out.. I'm happy to just wait a day or two till jhb isback on line to hear him say it.. I can live with slow behaviour for 2 days and you already have it zapped in your site right? We also need to do it in the other architectures. /sys/*/*/trap.c On Tue, 19 Feb 2002, Matthew Dillon wrote: > > :My understanding was that John saw this primarily as a debugging aid. No > :doubt: > : > :#ifdef OPTIONS_NULL_TD_UCRED > : ... > :#else > : ... > :#endif > : > :would be appropriate for general use once we're sure we're handling > :td_ucred fine, especially given the performance difference. > : > :Robert N M Watson FreeBSD Core Team, TrustedBSD Project > :robert@fledge.watson.org NAI Labs, Safeport Network Services > > Julian, should I go ahead and do this or would you like to? I already > have the patches ready, I just have to change a bunch of #if 0's into > #ifdef OPTIONS_TD_UCRED_DEBUG, for all architectures. > > I suggest something like OPTIONS_TD_UCRED_DEBUG. Turning on this > option would clear out the ucred in the thread after each syscall > (effectively the INVARIANTS code that is there now). Not specifying > this option would give us the cache behavior (what Julian's code was > intended to do). > > -Matt > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 15:56: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from hotmail.com (oe19.law14.hotmail.com [64.4.20.123]) by hub.freebsd.org (Postfix) with ESMTP id 6304637B400 for ; Tue, 19 Feb 2002 15:56:01 -0800 (PST) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Tue, 19 Feb 2002 15:56:01 -0800 X-Originating-IP: [198.178.8.81] From: "David Jellison" To: Subject: intrest in MIPS 10,000 CPU? support Date: Tue, 19 Feb 2002 15:45:40 -0800 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0005_01C1B95C.7B87ED40" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Message-ID: X-OriginalArrivalTime: 19 Feb 2002 23:56:01.0273 (UTC) FILETIME=[FBC12690:01C1B9A0] Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. ------=_NextPart_000_0005_01C1B95C.7B87ED40 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Is there any one working an support for SGI INDIGO 2 with the 10000 cpu = and SolidImpact Graphics? Any help would be very welcome. Thank you David ------=_NextPart_000_0005_01C1B95C.7B87ED40 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Is there any one working an support for = SGI INDIGO=20 2 with the 10000 cpu and SolidImpact Graphics?
 
Any help would be very = welcome.
Thank you
David
------=_NextPart_000_0005_01C1B95C.7B87ED40-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 16:19:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 03DE537B41A; Tue, 19 Feb 2002 16:19:03 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1K0IUA64540; Tue, 19 Feb 2002 16:18:30 -0800 (PST) (envelope-from dillon) Date: Tue, 19 Feb 2002 16:18:30 -0800 (PST) From: Matthew Dillon Message-Id: <200202200018.g1K0IUA64540@apollo.backplane.com> To: Julian Elischer Cc: Robert Watson , arch@FreeBSD.ORG, jhb@FreeBSD.ORG Subject: Re: that ucred invariant stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :yes please : :I could also do it.. : :my prefered behaviour however is to just rip it out.. :I'm happy to just wait a day or two till jhb isback on line to hear him :say it.. I can live with slow behaviour for 2 days and you already have it :zapped in your site right? : :We also need to do it in the other architectures. :/sys/*/*/trap.c I have it #if 0'd everywhere in my tree. It looks like Andrew and you have synced up the other architectures to your i386/i386/trap.c (i.e. addition of #ifdef INVARIANTS). I'm willing to wait for John to weigh in, on the off chance that he will allow it to simply be removed permanently. I'm not happy about the length of time it is taking, though. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 16:24:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from espresso.q9media.com (espresso.q9media.com [216.254.138.122]) by hub.freebsd.org (Postfix) with ESMTP id CC3C537B402 for ; Tue, 19 Feb 2002 16:24:25 -0800 (PST) Received: (from mike@localhost) by espresso.q9media.com (8.11.6/8.11.6) id g1K0Jva79346; Tue, 19 Feb 2002 19:19:57 -0500 (EST) (envelope-from mike) Date: Tue, 19 Feb 2002 19:19:57 -0500 From: Mike Barcroft To: Bruce Evans Cc: Jake Burkholder , Thomas Moestl , freebsd-arch@FreeBSD.ORG Subject: Re: adding more endian conversion and bus space functions Message-ID: <20020219191957.H5526@espresso.q9media.com> References: <20020112115513.L39321@locore.ca> <20020113230455.K709-100000@gamplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020113230455.K709-100000@gamplex.bde.org>; from bde@zeta.org.au on Sun, Jan 13, 2002 at 11:21:08PM +1100 Organization: The FreeBSD Project Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [Sorry this is a late reply.] Bruce Evans writes: > On Sat, 12 Jan 2002, Jake Burkholder wrote: > > > I think the bus > > > space headers should not depend on any endianness support in other > > > headers except defining _[_]BYTE_ORDER. > > > > Why? I disagree. > > Because they are specialized for bus accesses and need to support many > more types of accesses than . They can easily duplicate > the small part of ntohl(), etc., that they need (if they need it), like > the i386 one already does for most of the i/o instructions in the > i386 cpufunc.h. If I understand the problem correctly, these endian functions are needed in several places, so it doesn't make sense to localize it to one subsystem. Atleast one place is already using some of these functions (). I also understand FFS will require these new functions in order to support non-native endian FSes. Best regards, Mike Barcroft To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 16:36:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id BEA0E37B400 for ; Tue, 19 Feb 2002 16:36:21 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1K0a9U64733; Tue, 19 Feb 2002 16:36:09 -0800 (PST) (envelope-from dillon) Date: Tue, 19 Feb 2002 16:36:09 -0800 (PST) From: Matthew Dillon Message-Id: <200202200036.g1K0a9U64733@apollo.backplane.com> To: Jeff Roberson Cc: Julian Elischer , arch@FreeBSD.ORG Subject: Re: Prioritized bio patches. (Updated patch) References: <20020219171504.T12686-100000@mail.chesapeake.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :First of all, I updated the patch. When I merged it in from our sources I :missed a one line change that fixed a race condition. I also changed the :priority level of NORMAL to 6 so that I could avoid all of the -1's to :index the low priority queue. : :Secondly, I ran a simple test of a kernel compile. The test system has :one disk. I did a dd of /dev/zero to a file in a users home directory :with a nice of 20 while doing a kernel compile. The original compile took :11 minutes and 32 seconds. The compile with the dd going took 15 minutes :and 12 seconds. : :I originally did this work for VOD server. The idea being that the VOD :data was guaranteed and the rest of the system would just have to wait. :... : :Jeff Jeff, this looks like really interesting stuff! Could you explain the starvation issues in more depth? e.g. if I have a nice+20 process doing disk I/O and a nice-20 process saturating the disk, is it possible for the nice-20 process to lockout the nice+20 process from doing any disk I/O? Another worry: what happens when a low priority process is holding a vnode lock while doing synchronous I/O and a high priority process wants to access the same vnode? Here I am specifically thinking about directory accesses that are incidental to a path lookup. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 16:57:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by hub.freebsd.org (Postfix) with ESMTP id 8016937B400 for ; Tue, 19 Feb 2002 16:57:26 -0800 (PST) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id g1K0vHP37459; Tue, 19 Feb 2002 19:57:17 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Tue, 19 Feb 2002 19:57:17 -0500 (EST) From: Jeff Roberson To: Matthew Dillon Cc: Julian Elischer , Subject: Re: Prioritized bio patches. (Updated patch) In-Reply-To: <200202200036.g1K0a9U64733@apollo.backplane.com> Message-ID: <20020219194142.M12686-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 19 Feb 2002, Matthew Dillon wrote: > > Jeff, this looks like really interesting stuff! Could you explain > the starvation issues in more depth? e.g. if I have a nice+20 process > doing disk I/O and a nice-20 process saturating the disk, is it possible > for the nice-20 process to lockout the nice+20 process from doing > any disk I/O? > > Another worry: what happens when a low priority process is holding a > vnode lock while doing synchronous I/O and a high priority process wants > to access the same vnode? Here I am specifically thinking about > directory accesses that are incidental to a path lookup. > > -Matt > Yes, there is a possibility that a high priority process will completely lock out the low priority process. I haven't followed recent scheduler changes, but a niced process may never run at all if there is real work no? If this is true, than the two are very similar. Anyhow, there is a definite priority inversion issue here, but the system is somewhat autobalancing. One would assume that the higher priority task is blocked waiting for the low priority task to finish. So it is not issuing any other io, which allows the low priority task to finish up. This is not entirely optimal, I agree. I can not check for priority inversion with lockmgr locks though, so there really is no obvious way out. For my application, the lockout was very desirable. With workloads such as compiling, you'll always have some free slots to send low priority io. This is due to the sequential nature of the workload though. It reads, does something with the data, and then writes. In between other things can happen. I could see how a very busy database or web server may never give nice processes a chance to finish(until late at night, perhaps). You could force all file system meta data operations to run at the normal priority. This would be good for directory operations in particular. You can still have priority inversion issues with individual file read/writes, but it would alleviate some of the problems. Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 17:38:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by hub.freebsd.org (Postfix) with ESMTP id 8FFD537B416 for ; Tue, 19 Feb 2002 17:38:17 -0800 (PST) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id g1K1cGB61138 for ; Tue, 19 Feb 2002 20:38:16 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Tue, 19 Feb 2002 20:38:16 -0500 (EST) From: Jeff Roberson To: arch@freebsd.org Subject: vnode issues & shared lock patch Message-ID: <20020219202000.M38875-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At BSDCon there were several discussions about problems with vnode locking, and vnodes in general. Some of the issues are related to code not following the vnode locking rules in various areas, and some are due to the lack of soft reference tracking. One painful example is vop_revoke. It may work or it may panic. Also, there are many reports of file system deadlocks. We came up with a list of items that could go towards addressing these issues. Namely the following: 1) Soft reference tracking through a refcount (or making use count more pervasive) 2) Full locking of the vnode structure, with well defined rules 3) Replacing the lockmgr lock with sx locks and re-evaluating kernel code which should acquire these locks. I'd like to get a group of people together who are interested or knowledgeable in this area. I already know of a few folks who have offered assistance, but I don't have a complete list. I also have a patch which allows users of the namei interface to request shared locks on leafs. This has been running in all of our kernels for 6 months without a deadlock. I have only modified stat and open to take advantage of this capability. The problem we were having is that an application would stat a file that had 60 seconds or so worth of IO pending, and it would block until that IO was finished because namei always grabs exclusive locks. The patch looks a little messy due to the inconsistencies in locking around the system. I have to do extra checking since I do upgrades and downgrades and the VOP calls don't always return vnodes in the advertised lock state. This is a good example of the problems generalized above. It is really a temporary fix for more serious problems that need to be addressed. The patch is available at http://www.chesapeake.net/~jroberson/sharedlock.patch Thanks, Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 17:51:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id E7AE937B400 for ; Tue, 19 Feb 2002 17:51:12 -0800 (PST) Received: from blossom.cjclark.org ([12.234.91.48]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020220015112.ROSX2626.rwcrmhc51.attbi.com@blossom.cjclark.org>; Wed, 20 Feb 2002 01:51:12 +0000 Received: (from cjc@localhost) by blossom.cjclark.org (8.11.6/8.11.6) id g1K1pCM17186; Tue, 19 Feb 2002 17:51:12 -0800 (PST) (envelope-from cjc) Date: Tue, 19 Feb 2002 17:51:11 -0800 From: "Crist J. Clark" To: David Jellison Cc: freebsd-arch@FreeBSD.ORG Subject: Re: intrest in MIPS 10,000 CPU? support Message-ID: <20020219175111.O48401@blossom.cjclark.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from davidjellison@hotmail.com on Tue, Feb 19, 2002 at 03:45:40PM -0800 X-URL: http://people.freebsd.org/~cjc/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Feb 19, 2002 at 03:45:40PM -0800, David Jellison wrote: > Is there any one working an support for SGI INDIGO 2 with the 10000 cpu and SolidImpact Graphics? You mean the IP28 in a "Power Indigo2 10000?" http://www.uwsg.iu.edu/sgi/resources/ip-cpu.html I _think_ NetBSD does those, http://www.netbsd.org/Ports/sgimips/ I don't think there is much interest in FreeBSD about porting to any SGI hardware. -- Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 19:19:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by hub.freebsd.org (Postfix) with SMTP id 53EE437B402 for ; Tue, 19 Feb 2002 19:18:48 -0800 (PST) Received: (qmail 16145 invoked by uid 0); 20 Feb 2002 03:18:44 -0000 Received: from pd9538e61.dip.t-dialin.net (HELO forge.local) (217.83.142.97) by mail.gmx.net (mp016-rz3) with SMTP; 20 Feb 2002 03:18:44 -0000 Received: from tmm by forge.local with local (Exim 3.34 #1) id 16dNHG-0003q1-00 for ; Wed, 20 Feb 2002 04:18:42 +0100 Date: Wed, 20 Feb 2002 04:18:42 +0100 From: Thomas Moestl To: freebsd-arch@FreeBSD.org Subject: Re: adding more endian conversion and bus space functions Message-ID: <20020220031842.GF282@crow.dom2ip.de> Mail-Followup-To: freebsd-arch@FreeBSD.org References: <20020111005207.GA7246@crow.dom2ip.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020111005207.GA7246@crow.dom2ip.de> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 2002/01/11 at 01:52:07 +0100, Thomas Moestl wrote: > I'd like to propose some additions to the kernel endian conversion and > bus space functions to aid the sparc64 port, taken from NetBSD: I have attached a diff that implements these new functions, with minor deviations from the original proposal (I have not created a sys/endian.h, but have put the code that was meant to go there into sys/param.h for now). The new functions/macros are not exported to userland right now. Libstand will now include htonl and friend for all architectures; previously, consumers of libstand would use the inline functions defined in machine/endian.h, except for alpha (ia64 had these functions in libstand previously too, but they were not used as far as I can tell). These inline functions have changed names and are not exposed to userland any more. The following files will be cvs rm'ed when this patch will be committed: sys/libkern/alpha/htonl.S sys/libkern/alpha/htons.S sys/libkern/alpha/ntohl.S sys/libkern/alpha/ntohs.S sys/libkern/ia64/htonl.S sys/libkern/ia64/htons.S sys/libkern/ia64/ntohl.S sys/libkern/ia64/ntohs.S This is tested on i386 and sparc64. If there are no objections, I will try to find testers for the other architectures soon. Please review and comment! Thanks, - thomas ==== //depot/projects/sparc64/lib/libstand/Makefile#2 - /home/tmm/p4/sparc64/lib/libstand/Makefile ==== --- /tmp/tmp.13288.0 Wed Feb 20 01:53:46 2002 +++ /home/tmm/p4/sparc64/lib/libstand/Makefile Tue Feb 19 23:48:33 2002 @@ -33,6 +33,10 @@ # private (pruned) versions of libc string functions SRCS+= strcasecmp.c +# byte order functions from libc +.PATH: ${.CURDIR}/../libc/${MACHINE_ARCH}/net +SRCS+= htons.S ntohs.S htonl.S ntohl.S + # string functions from libc .PATH: ${.CURDIR}/../libc/string .if ${MACHINE_ARCH} == "i386" || ${MACHINE_ARCH} == "powerpc" || \ @@ -50,9 +54,6 @@ strncat.c strncmp.c strncpy.c strpbrk.c strrchr.c strsep.c \ strspn.c strstr.c strtok.c swab.c -.PATH: ${.CURDIR}/../libc/alpha/net -SRCS+= htons.S ntohs.S htonl.S ntohl.S - SRCS+= __divqu.S __divq.S __divlu.S __divl.S SRCS+= __remqu.S __remq.S __remlu.S __reml.S @@ -99,9 +100,6 @@ strcmp.c strcpy.c strcspn.c strlen.c \ strncat.c strncmp.c strncpy.c strpbrk.c strrchr.c strsep.c \ strspn.c strstr.c strtok.c swab.c - -.PATH: ${.CURDIR}/../libc/ia64/net -SRCS+= htons.S ntohs.S htonl.S ntohl.S .PATH: ${.CURDIR}/../libc/ia64/gen SRCS+= __divdi3.S __divsi3.S __moddi3.S __modsi3.S ==== //depot/projects/sparc64/sys/alpha/include/bus.h#1 - /home/tmm/p4/sparc64/sys/alpha/include/bus.h ==== --- /tmp/tmp.13288.1 Wed Feb 20 01:53:47 2002 +++ /home/tmm/p4/sparc64/sys/alpha/include/bus.h Wed Feb 20 01:37:00 2002 @@ -366,6 +366,70 @@ (t)->ab_ops->abo_barrier(t, (h)+(o), l, f) /* + * Stream accesses are the same as normal accesses on alpha; there are no + * supported bus systems with an endianess different from the host one. + */ +#define bus_space_read_stream_1(t, h, o) bus_space_read_1((t), (h), (o)) +#define bus_space_read_stream_2(t, h, o) bus_space_read_2((t), (h), (o)) +#define bus_space_read_stream_4(t, h, o) bus_space_read_4((t), (h), (o)) + +#define bus_space_read_multi_stream_1(t, h, o, a, c) \ + bus_space_read_multi_1((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_2(t, h, o, a, c) \ + bus_space_read_multi_2((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_4(t, h, o, a, c) \ + bus_space_read_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_write_stream_1(t, h, o, v) \ + bus_space_write_1((t), (h), (o), (v)) +#define bus_space_write_stream_2(t, h, o, v) \ + bus_space_write_2((t), (h), (o), (v)) +#define bus_space_write_stream_4(t, h, o, v) \ + bus_space_write_4((t), (h), (o), (v)) + +#define bus_space_write_multi_stream_1(t, h, o, a, c) \ + bus_space_write_multi_1((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_2(t, h, o, a, c) \ + bus_space_write_multi_2((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_4(t, h, o, a, c) \ + bus_space_write_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_set_multi_stream_1(t, h, o, v, c) \ + bus_space_set_multi_1((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_2(t, h, o, v, c) \ + bus_space_set_multi_2((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_4(t, h, o, v, c) \ + bus_space_set_multi_4((t), (h), (o), (v), (c)) + +#define bus_space_read_region_stream_1(t, h, o, a, c) \ + bus_space_read_region_1((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_2(t, h, o, a, c) \ + bus_space_read_region_2((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_4(t, h, o, a, c) \ + bus_space_read_region_4((t), (h), (o), (a), (c)) + +#define bus_space_write_region_stream_1(t, h, o, a, c) \ + bus_space_write_region_1((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_2(t, h, o, a, c) \ + bus_space_write_region_2((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_4(t, h, o, a, c) \ + bus_space_write_region_4((t), (h), (o), (a), (c)) + +#define bus_space_set_region_stream_1(t, h, o, v, c) \ + bus_space_set_region_1((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_2(t, h, o, v, c) \ + bus_space_set_region_2((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_4(t, h, o, v, c) \ + bus_space_set_region_4((t), (h), (o), (v), (c)) + +#define bus_space_copy_region_stream_1(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_1((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_2(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_2((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_4(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_4((t), (h1), (o1), (h2), (o2), (c)) + +/* * Flags used in various bus DMA methods. */ #define BUS_DMA_WAITOK 0x00 /* safe to sleep (pseudo-flag) */ ==== //depot/projects/sparc64/sys/alpha/include/endian.h#3 - /home/tmm/p4/sparc64/sys/alpha/include/endian.h ==== --- /tmp/tmp.13288.2 Wed Feb 20 01:53:47 2002 +++ /home/tmm/p4/sparc64/sys/alpha/include/endian.h Tue Feb 19 22:12:46 2002 @@ -56,6 +56,15 @@ #define PDP_ENDIAN 3412 /* LSB first in word, MSW first in long */ #define BYTE_ORDER LITTLE_ENDIAN + +#ifdef _KERNEL +__uint16_t __bswap16(__uint16_t); +__uint32_t __bswap32(__uint32_t); + +#define bswap16(x) __bswap16(x) +#define bswap32(x) __bswap32(x) +#endif + #endif /* !_POSIX_SOURCE */ #endif /* !_MACHINE_ENDIAN_H_ */ ==== //depot/projects/sparc64/sys/conf/files.alpha#8 - /home/tmm/p4/sparc64/sys/conf/files.alpha ==== --- /tmp/tmp.13288.3 Wed Feb 20 01:53:47 2002 +++ /home/tmm/p4/sparc64/sys/conf/files.alpha Tue Feb 19 20:13:51 2002 @@ -215,9 +215,7 @@ isa/syscons_isa.c optional sc isa/vga_isa.c optional vga kern/subr_diskmbr.c standard -libkern/alpha/htonl.S standard -libkern/alpha/htons.S standard -libkern/alpha/ntohl.S standard -libkern/alpha/ntohs.S standard +libkern/alpha/bswap16.S standard +libkern/alpha/bswap32.S standard libkern/bcmp.c standard libkern/ffs.c standard ==== //depot/projects/sparc64/sys/conf/files.ia64#9 - /home/tmm/p4/sparc64/sys/conf/files.ia64 ==== --- /tmp/tmp.13288.4 Wed Feb 20 01:53:47 2002 +++ /home/tmm/p4/sparc64/sys/conf/files.ia64 Tue Feb 19 20:14:56 2002 @@ -99,10 +99,8 @@ isa/syscons_isa.c optional sc isa/vga_isa.c optional vga kern/subr_diskmbr.c standard -libkern/ia64/htonl.S standard -libkern/ia64/htons.S standard -libkern/ia64/ntohl.S standard -libkern/ia64/ntohs.S standard +libkern/ia64/bswap16.S standard +libkern/ia64/bswap32.S standard libkern/ia64/__divsi3.S standard libkern/ia64/__modsi3.S standard libkern/ia64/__udivsi3.S standard ==== //depot/projects/sparc64/sys/dev/iir/iir.h#1 - /home/tmm/p4/sparc64/sys/dev/iir/iir.h ==== --- /tmp/tmp.13288.5 Wed Feb 20 01:53:48 2002 +++ /home/tmm/p4/sparc64/sys/dev/iir/iir.h Wed Feb 20 01:16:50 2002 @@ -372,10 +372,8 @@ #define GDT_SCRATCH_SZ 3072 /* 3KB scratch buffer */ /* macros */ -#define htole32(v) (v) -#define htole16(v) (v) -#define letoh32(v) (v) -#define letoh16(v) (v) +#define letoh32(v) le32toh(v) +#define letoh16(v) le16toh(v) /* Map minor numbers to device identity */ #define LUN_MASK 0x0007 ==== //depot/projects/sparc64/sys/dev/usb/ohci.c#7 - /home/tmm/p4/sparc64/sys/dev/usb/ohci.c ==== --- /tmp/tmp.13288.6 Wed Feb 20 01:53:48 2002 +++ /home/tmm/p4/sparc64/sys/dev/usb/ohci.c Wed Feb 20 00:40:01 2002 @@ -96,20 +96,6 @@ #define DPRINTFN(n,x) #endif -/* - * The OHCI controller is little endian, so on big endian machines - * the data strored in memory needs to be swapped. - */ -#if defined(__FreeBSD__) -#if BYTE_ORDER == BIG_ENDIAN -#define htole32(x) (bswap32(x)) -#define le32toh(x) (bswap32(x)) -#else -#define htole32(x) (x) -#define le32toh(x) (x) -#endif -#endif - struct ohci_pipe; Static ohci_soft_ed_t *ohci_alloc_sed(ohci_softc_t *); ==== //depot/projects/sparc64/sys/dev/usb/uhci.c#9 - /home/tmm/p4/sparc64/sys/dev/usb/uhci.c ==== --- /tmp/tmp.13288.7 Wed Feb 20 01:53:49 2002 +++ /home/tmm/p4/sparc64/sys/dev/usb/uhci.c Wed Feb 20 00:39:54 2002 @@ -112,7 +112,7 @@ * The UHCI controller is little endian, so on big endian machines * the data strored in memory needs to be swapped. */ -#if defined(__FreeBSD__) || defined(__OpenBSD__) +#if defined(__OpenBSD__) #if BYTE_ORDER == BIG_ENDIAN #define htole32(x) (bswap32(x)) #define le32toh(x) (bswap32(x)) ==== //depot/projects/sparc64/sys/dev/usb/usb_port.h#4 - /home/tmm/p4/sparc64/sys/dev/usb/usb_port.h ==== ==== //depot/projects/sparc64/sys/dev/wi/if_wi.c#13 - /home/tmm/p4/sparc64/sys/dev/wi/if_wi.c ==== --- /tmp/tmp.13288.8 Wed Feb 20 01:53:49 2002 +++ /home/tmm/p4/sparc64/sys/dev/wi/if_wi.c Wed Feb 20 00:34:30 2002 @@ -125,11 +125,9 @@ #endif /* - * The following is for compatibility with NetBSD, but should really be - * brought in from NetBSD en toto. + * The following is for compatibility with NetBSD. */ -#define le16toh(a) (a) -#define LE16TOH(a) +#define LE16TOH(a) ((a) = le16toh((a))) static void wi_intr __P((void *)); static void wi_reset __P((struct wi_softc *)); ==== //depot/projects/sparc64/sys/i386/include/bus.h#2 - /home/tmm/p4/sparc64/sys/i386/include/bus.h ==== --- /tmp/tmp.13288.9 Wed Feb 20 01:53:49 2002 +++ /home/tmm/p4/sparc64/sys/i386/include/bus.h Wed Feb 20 01:36:38 2002 @@ -43,4 +43,68 @@ #endif #include +/* + * Stream accesses are the same as normal accesses on i386/pc98; there are no + * supported bus systems with an endianess different from the host one. + */ +#define bus_space_read_stream_1(t, h, o) bus_space_read_1((t), (h), (o)) +#define bus_space_read_stream_2(t, h, o) bus_space_read_2((t), (h), (o)) +#define bus_space_read_stream_4(t, h, o) bus_space_read_4((t), (h), (o)) + +#define bus_space_read_multi_stream_1(t, h, o, a, c) \ + bus_space_read_multi_1((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_2(t, h, o, a, c) \ + bus_space_read_multi_2((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_4(t, h, o, a, c) \ + bus_space_read_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_write_stream_1(t, h, o, v) \ + bus_space_write_1((t), (h), (o), (v)) +#define bus_space_write_stream_2(t, h, o, v) \ + bus_space_write_2((t), (h), (o), (v)) +#define bus_space_write_stream_4(t, h, o, v) \ + bus_space_write_4((t), (h), (o), (v)) + +#define bus_space_write_multi_stream_1(t, h, o, a, c) \ + bus_space_write_multi_1((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_2(t, h, o, a, c) \ + bus_space_write_multi_2((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_4(t, h, o, a, c) \ + bus_space_write_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_set_multi_stream_1(t, h, o, v, c) \ + bus_space_set_multi_1((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_2(t, h, o, v, c) \ + bus_space_set_multi_2((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_4(t, h, o, v, c) \ + bus_space_set_multi_4((t), (h), (o), (v), (c)) + +#define bus_space_read_region_stream_1(t, h, o, a, c) \ + bus_space_read_region_1((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_2(t, h, o, a, c) \ + bus_space_read_region_2((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_4(t, h, o, a, c) \ + bus_space_read_region_4((t), (h), (o), (a), (c)) + +#define bus_space_write_region_stream_1(t, h, o, a, c) \ + bus_space_write_region_1((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_2(t, h, o, a, c) \ + bus_space_write_region_2((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_4(t, h, o, a, c) \ + bus_space_write_region_4((t), (h), (o), (a), (c)) + +#define bus_space_set_region_stream_1(t, h, o, v, c) \ + bus_space_set_region_1((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_2(t, h, o, v, c) \ + bus_space_set_region_2((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_4(t, h, o, v, c) \ + bus_space_set_region_4((t), (h), (o), (v), (c)) + +#define bus_space_copy_region_stream_1(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_1((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_2(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_2((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_4(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_4((t), (h1), (o1), (h2), (o2), (c)) + #endif /* _I386_BUS_H_ */ ==== //depot/projects/sparc64/sys/i386/include/endian.h#6 - /home/tmm/p4/sparc64/sys/i386/include/endian.h ==== --- /tmp/tmp.13288.10 Wed Feb 20 01:53:49 2002 +++ /home/tmm/p4/sparc64/sys/i386/include/endian.h Tue Feb 19 22:13:17 2002 @@ -58,12 +58,11 @@ #define BYTE_ORDER LITTLE_ENDIAN #endif /* ! _POSIX_SOURCE */ +#ifdef _KERNEL #ifdef __GNUC__ -__BEGIN_DECLS - static __inline __uint32_t -__htonl(__uint32_t __x) +__bswap32(__uint32_t __x) { #if defined(_KERNEL) && (defined(I486_CPU) || defined(I586_CPU) || defined(I686_CPU)) && !defined(I386_CPU) __asm ("bswap %0" : "+r" (__x)); @@ -77,28 +76,16 @@ } static __inline __uint16_t -__htons(__uint16_t __x) +__bswap16(__uint16_t __x) { __asm ("xchgb %h0, %b0" : "+q" (__x)); return __x; } -static __inline __uint32_t -__ntohl(__uint32_t __x) -{ - - return (__htonl(__x)); -} - -static __inline __uint16_t -__ntohs(__uint16_t __x) -{ - - return (__htons(__x)); -} - -__END_DECLS +#define bswap16(x) __bswap16(x) +#define bswap32(x) __bswap32(x) +#endif /* _KERNEL */ #endif /* __GNUC__ */ ==== //depot/projects/sparc64/sys/ia64/include/bus.h#1 - /home/tmm/p4/sparc64/sys/ia64/include/bus.h ==== --- /tmp/tmp.13288.11 Wed Feb 20 01:53:49 2002 +++ /home/tmm/p4/sparc64/sys/ia64/include/bus.h Wed Feb 20 01:42:54 2002 @@ -1000,6 +1000,70 @@ #endif } +/* + * Stream accesses are the same as normal accesses on ia64; there are no + * supported bus systems with an endianess different from the host one. + */ +#define bus_space_read_stream_1(t, h, o) bus_space_read_1((t), (h), (o)) +#define bus_space_read_stream_2(t, h, o) bus_space_read_2((t), (h), (o)) +#define bus_space_read_stream_4(t, h, o) bus_space_read_4((t), (h), (o)) + +#define bus_space_read_multi_stream_1(t, h, o, a, c) \ + bus_space_read_multi_1((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_2(t, h, o, a, c) \ + bus_space_read_multi_2((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_4(t, h, o, a, c) \ + bus_space_read_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_write_stream_1(t, h, o, v) \ + bus_space_write_1((t), (h), (o), (v)) +#define bus_space_write_stream_2(t, h, o, v) \ + bus_space_write_2((t), (h), (o), (v)) +#define bus_space_write_stream_4(t, h, o, v) \ + bus_space_write_4((t), (h), (o), (v)) + +#define bus_space_write_multi_stream_1(t, h, o, a, c) \ + bus_space_write_multi_1((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_2(t, h, o, a, c) \ + bus_space_write_multi_2((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_4(t, h, o, a, c) \ + bus_space_write_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_set_multi_stream_1(t, h, o, v, c) \ + bus_space_set_multi_1((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_2(t, h, o, v, c) \ + bus_space_set_multi_2((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_4(t, h, o, v, c) \ + bus_space_set_multi_4((t), (h), (o), (v), (c)) + +#define bus_space_read_region_stream_1(t, h, o, a, c) \ + bus_space_read_region_1((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_2(t, h, o, a, c) \ + bus_space_read_region_2((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_4(t, h, o, a, c) \ + bus_space_read_region_4((t), (h), (o), (a), (c)) + +#define bus_space_write_region_stream_1(t, h, o, a, c) \ + bus_space_write_region_1((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_2(t, h, o, a, c) \ + bus_space_write_region_2((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_4(t, h, o, a, c) \ + bus_space_write_region_4((t), (h), (o), (a), (c)) + +#define bus_space_set_region_stream_1(t, h, o, v, c) \ + bus_space_set_region_1((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_2(t, h, o, v, c) \ + bus_space_set_region_2((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_4(t, h, o, v, c) \ + bus_space_set_region_4((t), (h), (o), (v), (c)) + +#define bus_space_copy_region_stream_1(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_1((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_2(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_2((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_4(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_4((t), (h1), (o1), (h2), (o2), (c)) + #endif /* defined(_MACHINE_BUS_PIO_H_) || defined(_MACHINE_BUS_MEMIO_H_) */ #if 0 /* Cause a link error for bus_space_copy_8 */ ==== //depot/projects/sparc64/sys/ia64/include/endian.h#5 - /home/tmm/p4/sparc64/sys/ia64/include/endian.h ==== --- /tmp/tmp.13288.12 Wed Feb 20 01:53:49 2002 +++ /home/tmm/p4/sparc64/sys/ia64/include/endian.h Tue Feb 19 22:12:11 2002 @@ -59,10 +59,9 @@ #define BYTE_ORDER LITTLE_ENDIAN #endif /* !_POSIX_SOURCE */ +#ifdef _KERNEL #ifdef __GNUC__ -__BEGIN_DECLS - static __inline __uint64_t __uint8_swap_uint64(__uint64_t __x) { @@ -73,35 +72,29 @@ } static __inline __uint32_t -__htonl(__uint32_t __x) -{ - - return (__uint8_swap_uint64(__x) >> 32); -} - -static __inline __uint16_t -__htons(__uint16_t __x) -{ - - return (__uint8_swap_uint64(__x) >> 48); -} - -static __inline __uint32_t -__ntohl(__uint32_t __x) +__bswap32(__uint32_t __x) { return (__uint8_swap_uint64(__x) >> 32); } static __inline __uint16_t -__ntohs(__uint16_t __x) +__bswap16(__uint16_t __x) { return (__uint8_swap_uint64(__x) >> 48); } -__END_DECLS +#define bswap64(x) __uint8_swap_uint64(x) +#endif +#else /* __GNUC__ */ +__uint16_t __bswap16(__uint16_t); +__uint32_t __bswap32(__uint32_t); #endif /* __GNUC__ */ + +#define bswap16(x) __bswap16(x) +#define bswap32(x) __bswap32(x) +#endif /* _KERNEL */ #endif /* !_MACHINE_ENDIAN_H_ */ ==== //depot/projects/sparc64/sys/libkern/alpha/byte_swap_2.S#2 - /home/tmm/p4/sparc64/sys/libkern/alpha/byte_swap_2.S ==== --- /tmp/tmp.13288.14 Wed Feb 20 01:53:50 2002 +++ /home/tmm/p4/sparc64/sys/libkern/alpha/byte_swap_2.S Tue Feb 19 20:06:20 2002 @@ -30,8 +30,8 @@ #include -#if !defined(ALIAS) || !defined(NAME) -#error ALIAS or NAME not defined +#ifndef NAME +#error NAME not defined #endif /* @@ -39,7 +39,6 @@ * * Argument is an unsigned 2-byte integer (u_int16_t). */ -XLEAF(ALIAS, 1) LEAF(NAME, 1) /* a0 contains 0x0123 */ extbl a0, 0, t0 /* t0 = 0x 23 */ extbl a0, 1, t1 /* t1 = 0x 01 */ ==== //depot/projects/sparc64/sys/libkern/alpha/byte_swap_4.S#2 - /home/tmm/p4/sparc64/sys/libkern/alpha/byte_swap_4.S ==== --- /tmp/tmp.13288.15 Wed Feb 20 01:53:50 2002 +++ /home/tmm/p4/sparc64/sys/libkern/alpha/byte_swap_4.S Tue Feb 19 20:06:52 2002 @@ -30,8 +30,8 @@ #include -#if !defined(ALIAS) || !defined(NAME) -#error ALIAS or NAME not defined +#ifndef NAME +#error NAME not defined #endif /* @@ -39,7 +39,6 @@ * * Argument is an unsigned 4-byte integer (u_int32_t). */ -XLEAF(ALIAS, 1) LEAF(NAME, 1) /* a0 contains 0x01234567 */ extbl a0, 0, t0 /* t0 = 0x 67 */ extbl a0, 1, t1 /* t1 = 0x 45 */ ==== //depot/projects/sparc64/sys/libkern/ia64/byte_swap_2.S#3 - /home/tmm/p4/sparc64/sys/libkern/ia64/byte_swap_2.S ==== --- /tmp/tmp.13288.16 Wed Feb 20 01:53:50 2002 +++ /home/tmm/p4/sparc64/sys/libkern/ia64/byte_swap_2.S Tue Feb 19 20:04:55 2002 @@ -30,8 +30,8 @@ #include -#if !defined(ALIAS) || !defined(NAME) -#error ALIAS or NAME not defined +#ifndef NAME +#error NAME not defined #endif /* @@ -39,7 +39,6 @@ * * Argument is an unsigned 2-byte integer (u_int16_t). */ -WEAK_ALIAS(ALIAS, NAME) ENTRY(NAME, 1) mux1 r16=in0,@rev ;; ==== //depot/projects/sparc64/sys/libkern/ia64/byte_swap_4.S#3 - /home/tmm/p4/sparc64/sys/libkern/ia64/byte_swap_4.S ==== --- /tmp/tmp.13288.17 Wed Feb 20 01:53:50 2002 +++ /home/tmm/p4/sparc64/sys/libkern/ia64/byte_swap_4.S Tue Feb 19 20:05:25 2002 @@ -30,8 +30,8 @@ #include -#if !defined(ALIAS) || !defined(NAME) -#error ALIAS or NAME not defined +#ifndef NAME +#error NAME not defined #endif /* @@ -39,7 +39,6 @@ * * Argument is an unsigned 4-byte integer (u_int32_t). */ -WEAK_ALIAS(ALIAS, NAME) ENTRY(NAME, 1) mux1 r16=in0,@rev ;; ==== //depot/projects/sparc64/sys/powerpc/include/endian.h#3 - /home/tmm/p4/sparc64/sys/powerpc/include/endian.h ==== --- /tmp/tmp.13288.18 Wed Feb 20 01:53:51 2002 +++ /home/tmm/p4/sparc64/sys/powerpc/include/endian.h Tue Feb 19 21:18:12 2002 @@ -60,18 +60,6 @@ #ifndef _KERNEL #include - -__BEGIN_DECLS -__uint32_t __htonl __P((__uint32_t)); -__uint16_t __htons __P((__uint16_t)); -__uint32_t __ntohl __P((__uint32_t)); -__uint16_t __ntohs __P((__uint16_t)); -__END_DECLS #endif /* _KERNEL */ - -#define __htonl(x) (x) -#define __htons(x) (x) -#define __ntohl(x) (x) -#define __ntohs(x) (x) #endif /* !_MACHINE_ENDIAN_H_ */ ==== //depot/projects/sparc64/sys/sparc64/include/endian.h#8 - /home/tmm/p4/sparc64/sys/sparc64/include/endian.h ==== --- /tmp/tmp.13288.19 Wed Feb 20 01:53:51 2002 +++ /home/tmm/p4/sparc64/sys/sparc64/include/endian.h Tue Feb 19 20:38:23 2002 @@ -57,59 +57,4 @@ #define BYTE_ORDER BIG_ENDIAN #endif /* !_POSIX_SOURCE */ -#define __htonl(x) (x) -#define __htons(x) (x) -#define __ntohl(x) (x) -#define __ntohs(x) (x) - -__BEGIN_DECLS -__uint16_t bswap16 __P((__uint16_t)); -__uint32_t bswap32 __P((__uint32_t)); -__uint64_t bswap64 __P((__uint64_t)); -__END_DECLS - -#ifdef _KERNEL -/* XXX: these three macros should be moved! */ -static __inline __uint16_t -__swap16(__uint16_t x) -{ - return ((x >> 8) | ((x << 8) & 0xff00U)); -} - -static __inline __uint32_t -__swap32(__uint32_t x) -{ - return ((x >> 24) | ((x >> 8) & 0xff00U) | ((x << 8) & 0xff0000U) | - ((x << 24) & 0xff000000U)); -} - -static __inline __uint64_t -__swap64(__uint64_t x) -{ - return ((x >> 56) | ((x >> 40) & 0xff00UL) | ((x >> 24) & 0xff0000UL) | - ((x >> 8) & 0xff000000UL) | ((x << 8) & 0xff00000000UL) | - ((x << 24) & 0xff0000000000UL) | ((x << 40) & 0xff000000000000UL) | - ((x << 56))); -} - -#define htobe16(x) (x) -#define htobe32(x) (x) -#define htobe64(x) (x) -#define htole16(x) __swap16((x)) -#define htole32(x) __swap32((x)) -#define htole64(x) __swap64((x)) - -#define be16toh(x) (x) -#define be32toh(x) (x) -#define be64toh(x) (x) -#define le16toh(x) __swap16((x)) -#define le32toh(x) __swap32((x)) -#define le64toh(x) __swap64((x)) - -#define bswap16(x) __swap16((x)) -#define bswap32(x) __swap32((x)) -#define bswap64(x) __swap64((x)) - -#endif - #endif /* !_MACHINE_ENDIAN_H_ */ ==== //depot/projects/sparc64/sys/sys/imgact_aout.h#3 - /home/tmm/p4/sparc64/sys/sys/imgact_aout.h ==== --- /tmp/tmp.13288.20 Wed Feb 20 01:53:51 2002 +++ /home/tmm/p4/sparc64/sys/sys/imgact_aout.h Wed Feb 20 00:51:10 2002 @@ -50,13 +50,13 @@ ((mag) & 0xffff) ) #define N_GETMAGIC_NET(ex) \ - (__ntohl((ex).a_midmag) & 0xffff) + (ntohl((ex).a_midmag) & 0xffff) #define N_GETMID_NET(ex) \ - ((__ntohl((ex).a_midmag) >> 16) & 0x03ff) + ((ntohl((ex).a_midmag) >> 16) & 0x03ff) #define N_GETFLAG_NET(ex) \ - ((__ntohl((ex).a_midmag) >> 26) & 0x3f) + ((ntohl((ex).a_midmag) >> 26) & 0x3f) #define N_SETMAGIC_NET(ex,mag,mid,flag) \ - ( (ex).a_midmag = __htonl( (((flag)&0x3f)<<26) | (((mid)&0x03ff)<<16) \ + ( (ex).a_midmag = htonl( (((flag)&0x3f)<<26) | (((mid)&0x03ff)<<16) \ | (((mag)&0xffff)) ) ) #define N_ALIGN(ex,x) \ ==== //depot/projects/sparc64/sys/sys/mchain.h#2 - /home/tmm/p4/sparc64/sys/sys/mchain.h ==== --- /tmp/tmp.13288.21 Wed Feb 20 01:53:51 2002 +++ /home/tmm/p4/sparc64/sys/sys/mchain.h Tue Feb 19 22:22:06 2002 @@ -36,6 +36,28 @@ #include +#ifdef _KERNEL + +/* + * XXX: remove these defines and change the function calls in the code. Use + * it unconditionally if (when) the extended byte order functions become + * available in user space. + */ +#define htoles(x) htole16((x)) +#define letohs(x) le16toh((x)) +#define htolel(x) htole32((x)) +#define letohl(x) le32toh((x)) +#define htoleq(x) htole64((int64_t)(x)) +#define letohq(x) le64toh((int64_t)(x)) + +#define htobes(x) htobe16((x)) +#define betohs(x) be16toh((x)) +#define htobel(x) htobe32((x)) +#define betohl(x) be32toh((x)) +#define htobeq(x) htobe64((int64_t)(x)) +#define betohq(x) be64toh((int64_t)(x)) + +#else /* * This macros probably belongs to the endian.h */ @@ -78,6 +100,7 @@ #define letohl(x) ((u_int32_t)(x)) */ #endif /* (BYTE_ORDER == LITTLE_ENDIAN) */ +#endif /* _KERNEL */ #ifdef _KERNEL ==== //depot/projects/sparc64/sys/sys/param.h#10 - /home/tmm/p4/sparc64/sys/sys/param.h ==== --- /tmp/tmp.13288.22 Wed Feb 20 01:53:51 2002 +++ /home/tmm/p4/sparc64/sys/sys/param.h Wed Feb 20 01:20:02 2002 @@ -188,28 +188,110 @@ #define MAX(a,b) (((a)>(b))?(a):(b)) #endif +#ifdef _KERNEL +/* + * Extended byte order support functions, for kernel use only currently. + * First, generic implementation of the byte swapping functions for those + * architectures that do not have optimized variants of each. + */ + +#ifndef bswap16 +static __inline __uint16_t +__bswap16(__uint16_t x) +{ + return ((x >> 8) | ((x << 8) & 0xff00U)); +} +#define bswap16(x) __bswap16(x) +#endif + +#ifndef bswap32 +static __inline __uint32_t +__bswap32(__uint32_t x) +{ + return ((x >> 24) | ((x >> 8) & 0xff00U) | ((x << 8) & 0xff0000U) | + ((x << 24) & 0xff000000U)); +} +#define bswap32(x) __bswap32(x) +#endif + +#ifndef bswap64 +static __inline __uint64_t +__bswap64(__uint64_t x) +{ + return ((x >> 56) | ((x >> 40) & 0xff00UL) | ((x >> 24) & 0xff0000UL) | + ((x >> 8) & 0xff000000UL) | ((x << 8) & 0xff00000000UL) | + ((x << 24) & 0xff0000000000UL) | ((x << 40) & 0xff000000000000UL) | + ((x << 56))); +} +#define bswap64(x) __bswap64(x) +#endif + +#ifndef _BYTEORDER_FUNC_DEFINED +#define _BYTEORDER_FUNC_DEFINED +#if BYTE_ORDER == LITTLE_ENDIAN +#define htobe16(x) bswap16((x)) +#define htobe32(x) bswap32((x)) +#define htobe64(x) bswap64((x)) +#define htole16(x) ((__uint16_t)(x)) +#define htole32(x) ((__uint32_t)(x)) +#define htole64(x) ((__uint64_t)(x)) + +#define be16toh(x) bswap16((x)) +#define be32toh(x) bswap32((x)) +#define be64toh(x) bswap64((x)) +#define le16toh(x) ((__uint16_t)(x)) +#define le32toh(x) ((__uint32_t)(x)) +#define le64toh(x) ((__uint64_t)(x)) +#else /* BYTE_ORDER */ +#define htobe16(x) ((__uint16_t)(x)) +#define htobe32(x) ((__uint32_t)(x)) +#define htobe64(x) ((__uint64_t)(x)) +#define htole16(x) bswap16((x)) +#define htole32(x) bswap32((x)) +#define htole64(x) bswap64((x)) + +#define be16toh(x) ((__uint16_t)(x)) +#define be32toh(x) ((__uint32_t)(x)) +#define be64toh(x) ((__uint64_t)(x)) +#define le16toh(x) bswap16((x)) +#define le32toh(x) bswap32((x)) +#define le64toh(x) bswap64((x)) +#endif /* BYTE_ORDER */ + +#define htonl(x) htobe32((x)) +#define htons(x) htobe16((x)) +#define ntohl(x) be32toh((x)) +#define ntohs(x) be16toh((x)) +#endif /* _BYTEORDER_FUNC_DEFINED */ + +#else /* _KERNEL */ /* - * Kernel exposed versions of byteorder(3) functions. - * * XXX this section should only be defined in the kernel, but some userland * software utilizes it. */ #ifndef _BYTEORDER_FUNC_DEFINED #define _BYTEORDER_FUNC_DEFINED +__BEGIN_DECLS +__uint32_t __htonl __P((__uint32_t)); +__uint16_t __htons __P((__uint16_t)); +__uint32_t __ntohl __P((__uint32_t)); +__uint16_t __ntohs __P((__uint16_t)); +__END_DECLS #define htonl(x) __htonl(x) #define htons(x) __htons(x) #define ntohl(x) __ntohl(x) #define ntohs(x) __ntohs(x) -#endif +#endif /* BYTEORDER_FUNC_DEFINED */ +#endif /* _KERNEL */ /* * XXX deprecated uppercase variants for byteorder(3) functions. */ #ifndef _POSIX_SOURCE -#define NTOHL(x) ((x) = __ntohl(x)) -#define NTOHS(x) ((x) = __ntohs(x)) -#define HTONL(x) ((x) = __htonl(x)) -#define HTONS(x) ((x) = __htons(x)) +#define NTOHL(x) ((x) = ntohl(x)) +#define NTOHS(x) ((x) = ntohs(x)) +#define HTONL(x) ((x) = htonl(x)) +#define HTONS(x) ((x) = htons(x)) #endif /* _POSIX_SOURCE */ /* --- /dev/null Wed Feb 20 01:47:07 2002 +++ /home/tmm/p4/sparc64/sys/libkern/alpha/bswap16.S Tue Feb 19 20:08:13 2002 @@ -0,0 +1,35 @@ +/* + * Copyright (c) 1996 Carnegie-Mellon University. + * All rights reserved. + * + * Author: Chris G. Demetriou + * + * Permission to use, copy, modify and distribute this software and + * its documentation is hereby granted, provided that both the copyright + * notice and this permission notice appear in all copies of the + * software, derivative works or modified versions, and any portions + * thereof, and that both notices appear in supporting documentation. + * + * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" + * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND + * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. + * + * Carnegie Mellon requests users of this software to return to + * + * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU + * School of Computer Science + * Carnegie Mellon University + * Pittsburgh PA 15213-3890 + * + * any improvements or extensions that they make and grant Carnegie the + * rights to redistribute these changes. + * + * from: NetBSD: htons.S,v 1.1 1996/04/17 22:36:54 cgd + * from: src/sys/libkern/alpha/htons.S,v 1.3 2002/02/18 20:35:21 + * + * $FreeBSD$ + */ + +#define NAME __bswap16 + +#include --- /dev/null Wed Feb 20 01:47:07 2002 +++ /home/tmm/p4/sparc64/sys/libkern/alpha/bswap32.S Tue Feb 19 20:09:52 2002 @@ -0,0 +1,35 @@ +/* + * Copyright (c) 1996 Carnegie-Mellon University. + * All rights reserved. + * + * Author: Chris G. Demetriou + * + * Permission to use, copy, modify and distribute this software and + * its documentation is hereby granted, provided that both the copyright + * notice and this permission notice appear in all copies of the + * software, derivative works or modified versions, and any portions + * thereof, and that both notices appear in supporting documentation. + * + * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" + * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND + * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. + * + * Carnegie Mellon requests users of this software to return to + * + * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU + * School of Computer Science + * Carnegie Mellon University + * Pittsburgh PA 15213-3890 + * + * any improvements or extensions that they make and grant Carnegie the + * rights to redistribute these changes. + * + * from: NetBSD: htonl.S,v 1.1 1996/04/17 22:36:52 cgd + * from: src/sys/libkern/alpha/htonl.S,v 1.3 2002/02/18 20:35:21 + * + * $FreeBSD$ + */ + +#define NAME __bswap32 + +#include --- /dev/null Wed Feb 20 01:47:07 2002 +++ /home/tmm/p4/sparc64/sys/libkern/ia64/bswap16.S Tue Feb 19 20:11:47 2002 @@ -0,0 +1,35 @@ +/* + * Copyright (c) 1996 Carnegie-Mellon University. + * All rights reserved. + * + * Author: Chris G. Demetriou + * + * Permission to use, copy, modify and distribute this software and + * its documentation is hereby granted, provided that both the copyright + * notice and this permission notice appear in all copies of the + * software, derivative works or modified versions, and any portions + * thereof, and that both notices appear in supporting documentation. + * + * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" + * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND + * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. + * + * Carnegie Mellon requests users of this software to return to + * + * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU + * School of Computer Science + * Carnegie Mellon University + * Pittsburgh PA 15213-3890 + * + * any improvements or extensions that they make and grant Carnegie the + * rights to redistribute these changes. + * + * from: NetBSD: htons.S,v 1.1 1996/04/17 22:36:54 cgd + * from: src/sys/libkern/ia64/htons.S,v 1.2 2002/02/18 20:35:21 + * + * $FreeBSD$ + */ + +#define NAME __bswap16 + +#include --- /dev/null Wed Feb 20 01:47:07 2002 +++ /home/tmm/p4/sparc64/sys/libkern/ia64/bswap32.S Tue Feb 19 20:13:03 2002 @@ -0,0 +1,35 @@ +/* + * Copyright (c) 1996 Carnegie-Mellon University. + * All rights reserved. + * + * Author: Chris G. Demetriou + * + * Permission to use, copy, modify and distribute this software and + * its documentation is hereby granted, provided that both the copyright + * notice and this permission notice appear in all copies of the + * software, derivative works or modified versions, and any portions + * thereof, and that both notices appear in supporting documentation. + * + * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" + * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND + * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. + * + * Carnegie Mellon requests users of this software to return to + * + * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU + * School of Computer Science + * Carnegie Mellon University + * Pittsburgh PA 15213-3890 + * + * any improvements or extensions that they make and grant Carnegie the + * rights to redistribute these changes. + * + * from: NetBSD: htonl.S,v 1.1 1996/04/17 22:36:52 cgd + * from: src/sys/libkern/ia64/htonl.S,v 1.2 2002/02/18 20:35:21 + * + * $FreeBSD$ + */ + +#define NAME __bswap32 + +#include --- freebsd/sys/dev/usb/usb_port.h Fri Feb 8 01:54:08 2002 +++ /home/tmm/p4/sparc64/sys/dev/usb/usb_port.h Wed Feb 20 00:43:17 2002 @@ -305,7 +305,6 @@ /* XXX Change this when FreeBSD has memset */ #define memcpy(d, s, l) bcopy((s),(d),(l)) #define memset(d, v, l) bzero((d),(l)) -#define bswap32(x) swap32(x) #define kthread_create1(f, s, p, a0, a1) \ kthread_create((f), (s), (p), RFHIGHPID, (a0), (a1)) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 19:39:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.nsu.ru (mx.nsu.ru [193.124.215.71]) by hub.freebsd.org (Postfix) with ESMTP id C4A0037B404; Tue, 19 Feb 2002 19:39:14 -0800 (PST) Received: from regency.nsu.ru ([193.124.210.26] helo=cytherea.weblab.nsu.ru) by mail.nsu.ru with esmtp (Exim 3.20 #1) id 16dNav-0003yV-00; Wed, 20 Feb 2002 09:39:01 +0600 Received: (from danfe@localhost) by cytherea.weblab.nsu.ru (8.11.6/8.11.6) id g1K3dXG80972; Wed, 20 Feb 2002 09:39:33 +0600 (NOVT) (envelope-from danfe) Date: Wed, 20 Feb 2002 09:39:33 +0600 From: Alexey Dokuchaev To: Robert Watson Cc: arch@freebsd.org, ipfw@freebsd.org Subject: Re: Improvements to ipfw code (followup) Message-ID: <20020220093933.A78191@cytherea.weblab.nsu.ru> References: <20020219165630.A62749@cytherea.weblab.nsu.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from rwatson@freebsd.org on Tue, Feb 19, 2002 at 11:40:03AM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Feb 19, 2002 at 11:40:03AM -0500, Robert Watson wrote: > Just as a slight follow-up I should have included in my earlier e-mail: > the merging of ucred and pcred should make this patch now be able to > support real and saved uids/gids as well as effective uids/gids, meaning > that it can be used to also restrict setuid applications such as ping. Cool! Right now I am cleaning up this 5-year old patch to catch up with current IPFW code, fixing possible bugs, and separating optimizations and features stuff for easier reviewing and testing. I like the idea of supporting real and saved uids/gids as well as effective ones, I think I will include this functionality as soon as I get the whole thing working with current -CURRENT. Regs, Alexey Dokuchaev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 20:18:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 9898D37B405; Tue, 19 Feb 2002 20:18:15 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.3/8.11.3) with ESMTP id g1K4IEi91800; Tue, 19 Feb 2002 21:18:14 -0700 (MST) (envelope-from imp@village.org) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.11.6/8.11.6) with ESMTP id g1K4IDL76005; Tue, 19 Feb 2002 21:18:13 -0700 (MST) (envelope-from imp@village.org) Date: Tue, 19 Feb 2002 21:18:01 -0700 (MST) Message-Id: <20020219.211801.45873766.imp@village.org> To: mike.varga@cavium.com Cc: msmith@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: reverse vtophys() From: "M. Warner Losh" In-Reply-To: <004f01c1b96b$a37d1f50$b210a8c0@netscreen5> References: <200202170813.g1H8DAV01750@mass.dis.org> <004f01c1b96b$a37d1f50$b210a8c0@netscreen5> X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <004f01c1b96b$a37d1f50$b210a8c0@netscreen5> "mike varga" writes: : The concept is valid in Linux, : there is a function that does : what I am asking. : bus_to_virtual(). : : The page is locked : in memory. : Therefor there is one valid page : backed by the physical : address that corresponds : to the virtual address that : I am interested in retrieving? If you have used the bus_space functions, you could use rman_get_virtual(r), where r is a struct resource * that you got back from bus_alloc_resource(). Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 21:40:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id CA15337B400; Tue, 19 Feb 2002 21:40:09 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020220054009.YPSJ2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Wed, 20 Feb 2002 05:40:09 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id VAA60515; Tue, 19 Feb 2002 21:38:15 -0800 (PST) Date: Tue, 19 Feb 2002 21:38:14 -0800 (PST) From: Julian Elischer To: "M. Warner Losh" Cc: mike.varga@cavium.com, msmith@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: reverse vtophys() In-Reply-To: <20020219.211801.45873766.imp@village.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mike, How do you know the page is locked in memory? but more importantly, how do you know it is only mapped to a single address? we ar moving toward keeping a lot of pages unmapped and for DMA devices the need never be mapped for IO. they need onlybe mapped when copying data to serspace or when mmapping them. We'd need to know a much more specific reason to be able to give you the answer. I may have missed some of your posts. sorry if you already gave this information. Julian On Tue, 19 Feb 2002, M. Warner Losh wrote: > In message: <004f01c1b96b$a37d1f50$b210a8c0@netscreen5> > "mike varga" writes: > : The concept is valid in Linux, > : there is a function that does > : what I am asking. > : bus_to_virtual(). > : > : The page is locked > : in memory. > : Therefor there is one valid page > : backed by the physical > : address that corresponds > : to the virtual address that > : I am interested in retrieving? > > If you have used the bus_space functions, you could use > rman_get_virtual(r), where r is a struct resource * that you got back > from bus_alloc_resource(). > > Warner > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 19 22:44:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from smtp017.mail.yahoo.com (smtp017.mail.yahoo.com [216.136.174.114]) by hub.freebsd.org (Postfix) with SMTP id BB00237B41A for ; Tue, 19 Feb 2002 22:43:22 -0800 (PST) Received: from l228ppp190.ksc.net.th (HELO thailifetime) (203.155.228.190) by smtp.mail.vip.sc5.yahoo.com with SMTP; 20 Feb 2002 06:43:17 -0000 Message-ID: <01d601c1b9d9$37eceb80$82e29bcb@thailifetime> From: "faststep_3" To: Subject: Work From Home Free Information Date: Wed, 20 Feb 2002 13:36:25 +0700 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_01D3_01C1BA13.97B8CC20" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. ------=_NextPart_000_01D3_01C1BA13.97B8CC20 Content-Type: text/plain; charset="windows-874" Content-Transfer-Encoding: quoted-printable Work From Home Free Information WHo DO YOU KNOW IN .....INDIA - CHAINA? USA-Europe-Africa-Asia:-Taiwan-Hongkong Korea-Japan-Malasia-Pakistan-Bangdesh Multibillion Dollar International Company Expanding Rapidly, needs your help EARN US$ 500-$1500 A MONTH PART TIME $1,000-$5,000 A MONTH FULL - TIME TEL.6627520011 / 6691112270 E-mail;muimint@yahoo.com www.thailifetime.com/somporn www.herbalife.com Work from your home -country all around the world =E0=C7=C5=D2 10-15 =B9=D2=B7=D5 ... =C1=D5=A4=D8=B3=A4=E8=D2 =CB=D2=A1=B7=E8=D2=B9=A1=D3=C5=D1=A7=CB=D2=A7=D2=B9=B9=CD=A1=E0=C7=C5=D2 = =CB=C3=D7=CD=C3=D2=C2=E4=B4=E9=E0=CA=C3=D4=C1 =E4=C1=E8=B5=E9=CD=A7=CB=D2=CD=D5=A1=B5=E8=CD=E4=BB = =B7=E8=D2=B9=CA=D2=C1=D2=C3=B6=B7=D3=C3=D2=C2=E4=B4=E9=B6=D6=A7 =B9=CD=A1=E0=C7=C5=D2 5,000-30,000 =BA=D2=B7/=E0=B4=D7=CD=B9 =E0=B5=E7=C1=E0=C7=C5=D2 30,000-110,000 =BA=D2=B7/=E0=B4=D7=CD=B9 =CA=D2=C1=D2=C3=B6=E0=C5=D7=CD=A1=E0=C7=C5=D2=B7=D3=A7=D2=B9, = =B7=D3=A7=D2=B9=B7=D5=E8=BA=E9=D2=B9=CB=C3=D7=CD=CD=D4=B9=E0=B5=CD=C3=EC=E0= =B9=E7=B5 =B9=D1=A1=B8=D8=C3=A1=D4=A8 =E1=BE=B7=C2=EC =BE=C2=D2=BA=D2=C5 = =E0=C0=CA=D1=AA=A1=C3 =CA=B6=D2=BB=B9=D4=A1 =C7=D4=C8=C7=A1=C3 = =B7=B9=D2=C2=A4=C7=D2=C1 =B9=D1=A1=BA=D1=AD=AA=D5 =BE=B9=D1=A1=A7=D2=B9=BA=C3=D4=C9=D1=B7 =A2=E9=D2=C3=D2=AA=A1=D2=C3 = =B9=D1=A1=C8=D6=A1=C9=D2 =B9=D1=A1=A4=CD=C1=BE=D4=C7=E0=B5=CD=C3=EC = =B9=D1=A1=A2=D2=C2=BB=C3=D0=A1=D1=B9 =B5=D4=B4=B5=E8=CD =A4=D8=B3=CA=C1=BE=C3 66 91112 270 , 66 2752 00 = 11 www.thailifetime.com/somporn=20 *=CB=D2=A1=B7=E8=D2=B9=E3=AA=E9=CD=D4=B9=E0=B5=CD=C3=EC=E0=B9=E7=B5=E4=B4= =E9 =E0=C3=D2=A8=D0=BE=D4=A8=D2=C3=B3=D2=E0=BB=E7=B9=BE=D4=E0=C8=C9 ------=_NextPart_000_01D3_01C1BA13.97B8CC20 Content-Type: text/html; charset="windows-874" Content-Transfer-Encoding: quoted-printable
Work From Home Free = Information
 
WHo DO YOU KNOW IN .....INDIA -=20 CHAINA?
USA-Europe-Africa-Asia:-Taiwan-Hongkong
Korea-Japan-Malasia= -Pakistan-Bangdesh
Multibillion=20 Dollar International Company
Expanding Rapidly, needs your = help
 
EARN US$ 500-$1500 A MONTH PART=20 TIME
$1,000-$5,000 A MONTH FULL - TIME
 
TEL.6627520011    =20 /   6691112270
E-mail;muimint@yahoo.com
www.thailifetime.com/somporn=       =20 www.herbalife.com
 
Work from your home -country all around = the=20 world
 

=E0=C7=C5=D2 10-15 =B9=D2=B7=D5 ... = =C1=D5=A4=D8=B3=A4=E8=D2
=CB=D2=A1=B7=E8=D2=B9=A1=D3=C5=D1=A7=CB=D2=A7= =D2=B9=B9=CD=A1=E0=C7=C5=D2=20 =CB=C3=D7=CD=C3=D2=C2=E4=B4=E9=E0=CA=C3=D4=C1
=E4=C1=E8=B5=E9=CD=A7=CB= =D2=CD=D5=A1=B5=E8=CD=E4=BB = =B7=E8=D2=B9=CA=D2=C1=D2=C3=B6=B7=D3=C3=D2=C2=E4=B4=E9=B6=D6=A7
=B9=CD= =A1=E0=C7=C5=D2=20 5,000-30,000 =BA=D2=B7/=E0=B4=D7=CD=B9
=E0=B5=E7=C1=E0=C7=C5=D2 = 30,000-110,000=20 =BA=D2=B7/=E0=B4=D7=CD=B9
=CA=D2=C1=D2=C3=B6=E0=C5=D7=CD=A1=E0=C7=C5=D2= =B7=D3=A7=D2=B9, = =B7=D3=A7=D2=B9=B7=D5=E8=BA=E9=D2=B9=CB=C3=D7=CD=CD=D4=B9=E0=B5=CD=C3=EC=E0= =B9=E7=B5
=B9=D1=A1=B8=D8=C3=A1=D4=A8=20 =E1=BE=B7=C2=EC =BE=C2=D2=BA=D2=C5 =E0=C0=CA=D1=AA=A1=C3 = =CA=B6=D2=BB=B9=D4=A1 =C7=D4=C8=C7=A1=C3 =B7=B9=D2=C2=A4=C7=D2=C1 = =B9=D1=A1=BA=D1=AD=AA=D5
=BE=B9=D1=A1=A7=D2=B9=BA=C3=D4=C9=D1=B7 = =A2=E9=D2=C3=D2=AA=A1=D2=C3=20 =B9=D1=A1=C8=D6=A1=C9=D2 =B9=D1=A1=A4=CD=C1=BE=D4=C7=E0=B5=CD=C3=EC = =B9=D1=A1=A2=D2=C2=BB=C3=D0=A1=D1=B9
 
=B5=D4=B4=B5=E8=CD =A4=D8=B3=CA=C1=BE=C3   66 91112 = 270  ,  66 2752 00 11
www.thailifetime.com/somporn= =20
*=CB=D2=A1=B7=E8=D2=B9=E3=AA=E9=CD=D4=B9=E0=B5=CD=C3=EC=E0=B9=E7=B5=E4= =B4=E9 = =E0=C3=D2=A8=D0=BE=D4=A8=D2=C3=B3=D2=E0=BB=E7=B9=BE=D4=E0=C8=C9 ------=_NextPart_000_01D3_01C1BA13.97B8CC20-- _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 1:35: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from avocet.prod.itd.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by hub.freebsd.org (Postfix) with ESMTP id E678A37B41D; Wed, 20 Feb 2002 01:34:45 -0800 (PST) Received: from pool0143.cvx22-bradley.dialup.earthlink.net ([209.179.198.143] helo=mindspring.com) by avocet.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16dT9B-0000bc-00; Wed, 20 Feb 2002 01:34:45 -0800 Message-ID: <3C736DAB.4B6B4CEA@mindspring.com> Date: Wed, 20 Feb 2002 01:34:35 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Crist J. Clark" Cc: David Jellison , freebsd-arch@FreeBSD.ORG Subject: Re: intrest in MIPS 10,000 CPU? support References: <20020219175111.O48401@blossom.cjclark.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Crist J. Clark" wrote: > > On Tue, Feb 19, 2002 at 03:45:40PM -0800, David Jellison wrote: > > Is there any one working an support for SGI INDIGO 2 with the > > 10000 cpu and SolidImpact Graphics? > > You mean the IP28 in a "Power Indigo2 10000?" > > http://www.uwsg.iu.edu/sgi/resources/ip-cpu.html > > I _think_ NetBSD does those, > > http://www.netbsd.org/Ports/sgimips/ > > I don't think there is much interest in FreeBSD about porting to any > SGI hardware. For $299 I can get a 733MHz machine with 20G of disk, 128M of RAM, etc. etc.. I can get an Alpha for pretty cheap, but I can only stick 2G of RAM in it because of FreeBSD limitations that are Alpha specific. What's the ballpark on a cheap 64 bit MIPS box? The only reason I would go for MIPS or Itanium or other 64 bit machine would be to break the 4G of physical RAM barrier. I think the MIPS 10000 is going to take a long time to reach a casual hacker price point. That said, there is apparently an unmerged FreeBSD MIPS port that boots multiuser; this was reported on at the BSDCon. It looks to be a difficult merge to get it in the tree, and I'm not sure if it's 32 bit or 64 bit (I think it may be 32, but there are some 64 bit embedded systems, like SIBytes, so I could be wrong on that). You should contact the people involved on the correct mailing list, if you are interested in helping with the peeing on the code to make it smell like FreeBSD. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 2:18:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id AA72937B400 for ; Wed, 20 Feb 2002 02:18:33 -0800 (PST) Received: from pool0143.cvx22-bradley.dialup.earthlink.net ([209.179.198.143] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16dTpY-0003wv-00; Wed, 20 Feb 2002 02:18:32 -0800 Message-ID: <3C7377EE.5E151D84@mindspring.com> Date: Wed, 20 Feb 2002 02:18:22 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Roberson Cc: arch@freebsd.org Subject: Re: vnode issues & shared lock patch References: <20020219202000.M38875-100000@mail.chesapeake.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jeff Roberson wrote: > I also have a patch which allows users of the namei interface to request > shared locks on leafs. This has been running in all of our kernels for 6 > months without a deadlock. I have only modified stat and open to take > advantage of this capability. The problem we were having is that an > application would stat a file that had 60 seconds or so worth of IO > pending, and it would block until that IO was finished because namei > always grabs exclusive locks. > > The patch looks a little messy due to the inconsistencies in locking > around the system. I have to do extra checking since I do upgrades and > downgrades and the VOP calls don't always return vnodes in the advertised > lock state. This is a good example of the problems generalized above. It > is really a temporary fix for more serious problems that need to be > addressed. Other than some unnecessary whit space changes in namei.h, this looks like good code. My personal preference would be for SIX locks, which should be significantly more concurrent than SX locks in the create and delete cases, but signalling intent might be enough harder that it would take much more of an impact. This is probably the best compromise for testing purposes. I'm curious what happens if you are, for example, untar'ring a ports tree, and you rm -rf it a couple of times during the process. I would expect perhaps a panic in the cache code (does a name cache entry need to act as a shared reference?) -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 6:26:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 9345F37B402 for ; Wed, 20 Feb 2002 06:26:38 -0800 (PST) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id JAA15144; Wed, 20 Feb 2002 09:26:38 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id g1KEQ8r62078; Wed, 20 Feb 2002 09:26:08 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15475.45568.4884.470350@grasshopper.cs.duke.edu> Date: Wed, 20 Feb 2002 09:26:08 -0500 (EST) To: Terry Lambert Cc: freebsd-arch@FreeBSD.ORG Subject: Re: intrest in MIPS 10,000 CPU? support In-Reply-To: <3C736DAB.4B6B4CEA@mindspring.com> References: <20020219175111.O48401@blossom.cjclark.org> <3C736DAB.4B6B4CEA@mindspring.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert writes: > I can get an Alpha for pretty cheap, but I can only stick 2G > of RAM in it because of FreeBSD limitations that are Alpha > specific. <...> > The only reason I would go for MIPS or Itanium or other 64 > bit machine would be to break the 4G of physical RAM barrier. Alpha actually has a rather nice framework for doing scatter/gather mapping in the pci chipset for DMA for address limited devices. This is how ISA dma is done on all alpha platforms (except for UP1x00, which uses an AMD 751 pee-cee chipset which has no such support and does bounce buffering) The limitations are FreeBSD specific, not Alpha specific. The limitation is all the various & sundry drivers which do: dma_addr = vtophys(va); Rather than using busdma. I think Bill Paul posted a patch for establishing a busdma framework for network drivers last year. I wonder what happened to that..? Of course, to get 4GB or more in a box, you're talking some fairly serious hardware (rawhide or ds20/dp264 family). Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 12:17:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from hotmail.com (oe47.law14.hotmail.com [64.4.20.19]) by hub.freebsd.org (Postfix) with ESMTP id 21E0537B416; Wed, 20 Feb 2002 12:17:21 -0800 (PST) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 20 Feb 2002 12:17:20 -0800 X-Originating-IP: [198.178.8.81] From: "David Jellison" To: "Terry Lambert" , "Crist J. Clark" Cc: References: <20020219175111.O48401@blossom.cjclark.org> <3C736DAB.4B6B4CEA@mindspring.com> Subject: Re: intrest in MIPS 10,000 CPU? support Date: Wed, 20 Feb 2002 12:18:55 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2600.0000 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Message-ID: X-OriginalArrivalTime: 20 Feb 2002 20:17:20.0981 (UTC) FILETIME=[99DD3850:01C1BA4B] Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Regarding "What's the ballpark on a cheap 64 bit MIPS box?" I picked one up for $50. with 192Mb ram and HD sled WO drive. I think that MIPS 10000 is going to take a casual hacker price point. ----- Original Message ----- From: "Terry Lambert" To: "Crist J. Clark" Cc: "David Jellison" ; Sent: Wednesday, February 20, 2002 1:34 AM Subject: Re: intrest in MIPS 10,000 CPU? support > "Crist J. Clark" wrote: > > > > On Tue, Feb 19, 2002 at 03:45:40PM -0800, David Jellison wrote: > > > Is there any one working an support for SGI INDIGO 2 with the > > > 10000 cpu and SolidImpact Graphics? > > > > You mean the IP28 in a "Power Indigo2 10000?" > > > > http://www.uwsg.iu.edu/sgi/resources/ip-cpu.html > > > > I _think_ NetBSD does those, > > > > http://www.netbsd.org/Ports/sgimips/ > > > > I don't think there is much interest in FreeBSD about porting to any > > SGI hardware. > > For $299 I can get a 733MHz machine with 20G of disk, 128M of > RAM, etc. etc.. > > I can get an Alpha for pretty cheap, but I can only stick 2G > of RAM in it because of FreeBSD limitations that are Alpha > specific. > > What's the ballpark on a cheap 64 bit MIPS box? > > The only reason I would go for MIPS or Itanium or other 64 > bit machine would be to break the 4G of physical RAM barrier. > > I think the MIPS 10000 is going to take a long time to reach > a casual hacker price point. > > That said, there is apparently an unmerged FreeBSD MIPS port > that boots multiuser; this was reported on at the BSDCon. It > looks to be a difficult merge to get it in the tree, and I'm > not sure if it's 32 bit or 64 bit (I think it may be 32, but > there are some 64 bit embedded systems, like SIBytes, so I > could be wrong on that). > > You should contact the people involved on the correct mailing > list, if you are interested in helping with the peeing on the > code to make it smell like FreeBSD. > > -- Terry > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 12:45:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from swan.prod.itd.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by hub.freebsd.org (Postfix) with ESMTP id A0F6E37B437; Wed, 20 Feb 2002 12:45:29 -0800 (PST) Received: from pool0067.cvx40-bradley.dialup.earthlink.net ([216.244.42.67] helo=mindspring.com) by swan.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16ddcB-0002NL-00; Wed, 20 Feb 2002 12:45:24 -0800 Message-ID: <3C740AD9.C1DD2C61@mindspring.com> Date: Wed, 20 Feb 2002 12:45:13 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Jellison Cc: "Crist J. Clark" , freebsd-arch@FreeBSD.ORG Subject: Re: intrest in MIPS 10,000 CPU? support References: <20020219175111.O48401@blossom.cjclark.org> <3C736DAB.4B6B4CEA@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG David Jellison wrote: > > Regarding "What's the ballpark on a cheap 64 bit MIPS box?" > I picked one up for $50. with 192Mb ram and HD sled WO drive. > I think that MIPS 10000 is going to take a casual hacker price point. OK, now you have to tell us where. 8-). The comment I made on the MIPS port of FreeBSD already existing for a 4.x version is still valid; has anyone considered posting to platforms@freebsd.org and asking about MIPS? The Developer Summit the day after BSDCon indicated that there was a MIPS port available, but I didn't get much detail; for $50, I'd be willing to buy one of these things (of course!). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 14:27:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail12.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by hub.freebsd.org (Postfix) with ESMTP id 4213D37B402 for ; Wed, 20 Feb 2002 14:27:21 -0800 (PST) Received: (qmail 11331 invoked from network); 20 Feb 2002 22:27:20 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.90.117.57]) (envelope-sender ) by mail12.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 20 Feb 2002 22:27:20 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200202160500.g1G50j145638@apollo.backplane.com> Date: Wed, 20 Feb 2002 17:27:21 -0500 (EST) From: John Baldwin To: Matthew Dillon Subject: RE: gettimeofday() and copyout(). Is copyout() MPSAFE on non-i3 Cc: jake@locore.ca, peter@wemm.org, phk@critter.freebsd.dk, arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 16-Feb-02 Matthew Dillon wrote: > Poul indicated today that the microtime() call that gettimeofday() uses > is MPSAFE. copyout() on I386 is MPSAFE, and from my perusal of the > other archs it appears to be MPSAFE for alpha, ia64, and sparc64 > as well. > > This would seem to indicate that Giant can be removed from the > gettimeofday() system call. I would like those familiar with the > other archs to verify that copyout() is MPSAFE on them. I am testing > Giant removal for this syscall on i386 now. The tz stuff still needs Giant, or perhaps some sort of lock for that variable. > -Matt > Matthew Dillon > -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 14:28: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by hub.freebsd.org (Postfix) with ESMTP id 499A037B41E for ; Wed, 20 Feb 2002 14:27:25 -0800 (PST) Received: (qmail 4846 invoked from network); 20 Feb 2002 22:27:23 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.90.117.57]) (envelope-sender ) by mail6.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 20 Feb 2002 22:27:23 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Wed, 20 Feb 2002 17:27:26 -0500 (EST) From: John Baldwin To: Julian Elischer Subject: RE: that ucred invariant stuff. Cc: dillon@freebsd.org, arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 18-Feb-02 Julian Elischer wrote: > > John, do you REALLY want that invariant stuff to clear ucreds in user > space. Matt and I discussed it and we'd really prefer to just shoot it. > I'm not sure what it gives you but I'm planning on having a flag on teh > thread that says when the thread is supposed to be in user mode > so maybe you can test that instead if you suspect that you're accessing > a thread presently in userland. If you want to add KASSERT()'s to _every_ place we use td_ucred to test that flag, then go for it. :) That would be an equivalent change once the td_ucred stuff goes in. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 14:29:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by hub.freebsd.org (Postfix) with ESMTP id 9EA6E37B420 for ; Wed, 20 Feb 2002 14:27:28 -0800 (PST) Received: (qmail 4883 invoked from network); 20 Feb 2002 22:27:27 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.90.117.57]) (envelope-sender ) by mail6.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 20 Feb 2002 22:27:27 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020219083406.B351E3A9A@overcee.wemm.org> Date: Wed, 20 Feb 2002 17:27:30 -0500 (EST) From: John Baldwin To: Peter Wemm Subject: Re: John Balwdin's proc-locking patch Cc: arch@FreeBSD.ORG, Robert Watson , Matthew Dillon , Alfred Perlstein Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 19-Feb-02 Peter Wemm wrote: > Alfred Perlstein wrote: >> * Matthew Dillon [020218 22:27] wrote: >> >> > It is totally unecessary to accumulate so many easily-made-incremental >> > changes off-tree and, in fact, I believe it to be detrimental to the >> > project. It locks up large areas of code for long periods of time >> > and creates both a synchronization hassle (even with P4) and a >> > debugging nightmare when finally committed due to the sheer number >> > of changes involved. >> [snip] >> > >> > That is my position. >> >> I've felt the same way for a while. > > Nobody is disputing that. It would have been better if John had committed > it piecemeal, but it didn't work out that way. I doubt he'll fall for this > trap again. Actually, I tried to do it piecemeal and ended up with a kernel that wouldn't boot. :( -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 17:10:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 1F2A637B400; Wed, 20 Feb 2002 17:10:49 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1L1Amw91277; Wed, 20 Feb 2002 17:10:48 -0800 (PST) (envelope-from dillon) Date: Wed, 20 Feb 2002 17:10:48 -0800 (PST) From: Matthew Dillon Message-Id: <200202210110.g1L1Amw91277@apollo.backplane.com> To: John Baldwin Cc: Peter Wemm , arch@FreeBSD.org, Robert Watson , Alfred Perlstein Subject: Re: John Balwdin's proc-locking patch References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :>> > :>> > That is my position. :>> :>> I've felt the same way for a while. :> :> Nobody is disputing that. It would have been better if John had committed :> it piecemeal, but it didn't work out that way. I doubt he'll fall for this :> trap again. : :Actually, I tried to do it piecemeal and ended up with a kernel that wouldn't :boot. :( : :-- : :John Baldwin <>< http://www.FreeBSD.org/~jhb/ :"Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ Well... try again. If something ought to be compatible in a piecemeal commit and isn't, then something unexpected happened and you need to find out what it was. Doing a full-on commit for something like the proc lock patch is far too dangerous. It's just too large a patch set and we know from experience (cam, softupdates, etc...) that having a small handful of people testing a large private patch is not going to find all the bugs. This is why I introduced the mtx_lock_giant() family of calls in the first place .. to allow pushdown work to be committed piecemeal without turning Giant off, so developers working on unrelated subsystems don't get unexpected timing changes or crashes if, for example, you forget to PROC_LOCK a particular piece of code. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 17:12:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 0C44337B41A; Wed, 20 Feb 2002 17:12:09 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1L1C8091297; Wed, 20 Feb 2002 17:12:08 -0800 (PST) (envelope-from dillon) Date: Wed, 20 Feb 2002 17:12:08 -0800 (PST) From: Matthew Dillon Message-Id: <200202210112.g1L1C8091297@apollo.backplane.com> To: John Baldwin Cc: Julian Elischer , arch@FreeBSD.ORG Subject: Re: RE: that ucred invariant stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :On 18-Feb-02 Julian Elischer wrote: :> :> John, do you REALLY want that invariant stuff to clear ucreds in user :> space. Matt and I discussed it and we'd really prefer to just shoot it. :> I'm not sure what it gives you but I'm planning on having a flag on teh :> thread that says when the thread is supposed to be in user mode :> so maybe you can test that instead if you suspect that you're accessing :> a thread presently in userland. : :If you want to add KASSERT()'s to _every_ place we use td_ucred to test that :flag, then go for it. :) That would be an equivalent change once the td_ucred :stuff goes in. : :-- : :John Baldwin <>< http://www.FreeBSD.org/~jhb/ :"Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ Well, if you don't want to simply remove it then we need to create a kernel option for it, because it is seriously getting in the way of testing the way it is now. So should we make a kernel option and if so what do you want the default state to be? -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 19:30:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from swan.prod.itd.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by hub.freebsd.org (Postfix) with ESMTP id D701C37B404; Wed, 20 Feb 2002 19:29:51 -0800 (PST) Received: from pool0050.cvx22-bradley.dialup.earthlink.net ([209.179.198.50] helo=mindspring.com) by swan.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16djvP-0006eT-00; Wed, 20 Feb 2002 19:29:40 -0800 Message-ID: <3C746999.7F4C6B88@mindspring.com> Date: Wed, 20 Feb 2002 19:29:29 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: John Baldwin Cc: Peter Wemm , arch@FreeBSD.ORG, Robert Watson , Matthew Dillon , Alfred Perlstein Subject: Re: John Balwdin's proc-locking patch References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG John Baldwin wrote: > Actually, I tried to do it piecemeal and ended up with a kernel that wouldn't > boot. :( I've done the same thing. There's a lot of code that, if it's broken into bite-sized chunks, loses its relevence. Such code is only valid when you look at "the big picture". Bite-sized chunks are almost always too small to build a big picture. The idea that you can get from any point A to any point B in evolutionary steps is really, really, broken. At best, you get punctiuated equilibrium, and have to take hits on the big things as ...well, big hits of a lot of code at once. Historically, the routing code rewrite orphaned ISODE and X.25, the CAM code orphaned many IDE controllers and the AHA 1540/1542 and 1510, etc.. Disruptive technology is the price of progress, and trying to make the transition nice and safe always is why companies like Shugart aren't the leading disk manufacturers today. Personally, I'd rather take the pain of a John Dyson unifying the VM or a Julian Elisher pounding KSE's into a square hole, or Kirk McKusick designing a UFS2, than trying to figure out how to cross the Grand Canyon with "baby steps". And it's not like the source control system couldn't let you back out the changes in a month, should it turn out they were an incredibly bad idea. ...if they were an incredibly good idea, on the other hand, you're just that much farther ahead. I'd like to see the code committed, so that it can be pounded on, and I'd like to see Jonathan Lemon's sysctl for the NETISR removal committed and on by default, and I'd like to see the kernel preemption patches _in_, and I'd like to see Luigi's polling changes become niversal. And those are just the obvious examples that leap to mind. A six month delay in all this work means a six month delay in everything that anyone could possibly think to build on top of it, and _that_ would be the real loss. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 19:44:39 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 4074737B400; Wed, 20 Feb 2002 19:44:35 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1L3iXr92334; Wed, 20 Feb 2002 19:44:33 -0800 (PST) (envelope-from dillon) Date: Wed, 20 Feb 2002 19:44:33 -0800 (PST) From: Matthew Dillon Message-Id: <200202210344.g1L3iXr92334@apollo.backplane.com> To: Terry Lambert Cc: John Baldwin , Peter Wemm , arch@FreeBSD.org, Robert Watson , Alfred Perlstein Subject: Re: John Balwdin's proc-locking patch References: <3C746999.7F4C6B88@mindspring.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is not contrary to my comments. I am not saying that everything can be done piecemeal... in fact, I have said quite specifically that there are many things that CAN'T be done piecemeal. But that has absolutely no relevance to my complaint, because things like ucred CAN be done piecemeal. I went through John's proc patchset and I believe the number I came up with was that 85% of it was trivially piecemeal-able. My complaint, very specifically, is that P4 is being over-used and that is creating severe interference for everyone outside of P4. Yes, that means me specifically. And also Alfred. A good chunk of what I've seen so far doesn't belong in P4 at all. We are at a point in -current where great progress can be made, but if we stick to P4 it just isn't going to happen on a timescale that anyone other then a snail will be happy with. -Matt Matthew Dillon :John Baldwin wrote: :> Actually, I tried to do it piecemeal and ended up with a kernel that wouldn't :> boot. :( : :I've done the same thing. : :There's a lot of code that, if it's broken into bite-sized :chunks, loses its relevence. Such code is only valid when :you look at "the big picture". Bite-sized chunks are almost :always too small to build a big picture. : :The idea that you can get from any point A to any point B in :evolutionary steps is really, really, broken. At best, you :get punctiuated equilibrium, and have to take hits on the :big things as ...well, big hits of a lot of code at once. : :Historically, the routing code rewrite orphaned ISODE and :X.25, the CAM code orphaned many IDE controllers and the :AHA 1540/1542 and 1510, etc.. : :Disruptive technology is the price of progress, and trying :to make the transition nice and safe always is why companies :like Shugart aren't the leading disk manufacturers today. : :Personally, I'd rather take the pain of a John Dyson unifying :the VM or a Julian Elisher pounding KSE's into a square hole, :or Kirk McKusick designing a UFS2, than trying to figure out :how to cross the Grand Canyon with "baby steps". : :And it's not like the source control system couldn't let you :back out the changes in a month, should it turn out they were :an incredibly bad idea. ...if they were an incredibly good :idea, on the other hand, you're just that much farther ahead. : : :I'd like to see the code committed, so that it can be pounded :on, and I'd like to see Jonathan Lemon's sysctl for the NETISR :removal committed and on by default, and I'd like to see the :kernel preemption patches _in_, and I'd like to see Luigi's :polling changes become niversal. And those are just the obvious :examples that leap to mind. : :A six month delay in all this work means a six month delay in :everything that anyone could possibly think to build on top of :it, and _that_ would be the real loss. : :-- Terry : To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 23:40: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id 79AE837B416; Wed, 20 Feb 2002 23:40:08 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020221074008.FNYF2951.rwcrmhc53.attbi.com@InterJet.elischer.org>; Thu, 21 Feb 2002 07:40:08 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id XAA66561; Wed, 20 Feb 2002 23:25:18 -0800 (PST) Date: Wed, 20 Feb 2002 23:25:17 -0800 (PST) From: Julian Elischer To: jhb@freebsd.org Cc: arch@freebsd.org Subject: that INVARIANT/ucred freeing stuff. Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG After talking with a bunch of people there's a genral concensus that we should just remove the ucred freeing stuff at least in normal builds. I personally canr't see how it can help in debugging in a real life non tailored situation to have the ucreds zero'd and it's robbing us of upto 30% of throughput in some cases. I REALLY want to just rip it out! julian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 20 23:55:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 834BB37B402; Wed, 20 Feb 2002 23:55:36 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1L7t5F93967; Wed, 20 Feb 2002 23:55:05 -0800 (PST) (envelope-from dillon) Date: Wed, 20 Feb 2002 23:55:05 -0800 (PST) From: Matthew Dillon Message-Id: <200202210755.g1L7t5F93967@apollo.backplane.com> To: Julian Elischer Cc: jhb@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: that INVARIANT/ucred freeing stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :I REALLY want to just rip it out! : :julian Ditto. I don't mind optioning it (i.e. option to recover zeroing code, default to rip it out), but I would much prefer to simply see it gone. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 0:30:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from harrier.prod.itd.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by hub.freebsd.org (Postfix) with ESMTP id 1079537B404; Thu, 21 Feb 2002 00:30:24 -0800 (PST) Received: from pool0063.cvx40-bradley.dialup.earthlink.net ([216.244.42.63] helo=mindspring.com) by harrier.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16docP-0004ND-00; Thu, 21 Feb 2002 00:30:22 -0800 Message-ID: <3C74B013.606DBE83@mindspring.com> Date: Thu, 21 Feb 2002 00:30:11 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: jhb@freebsd.org, arch@freebsd.org Subject: Re: that INVARIANT/ucred freeing stuff. References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer wrote: > After talking with a bunch of people there's a genral concensus that we > should just remove the ucred freeing stuff at least in normal builds. I > personally canr't see how it can help in debugging in a real life non > tailored situation to have the ucreds zero'd and it's robbing us of upto > 30% of throughput in some cases. > > I REALLY want to just rip it out! If this code had been there when I first found the cred reference overflow bug and fixed it, it would have taken me another week to find the problem, I think. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 0:40:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hub.freebsd.org (Postfix) with ESMTP id AD74A37B404; Thu, 21 Feb 2002 00:40:07 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020221084007.LXHW1147.rwcrmhc52.attbi.com@InterJet.elischer.org>; Thu, 21 Feb 2002 08:40:07 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id AAA66782; Thu, 21 Feb 2002 00:23:07 -0800 (PST) Date: Thu, 21 Feb 2002 00:23:05 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: jhb@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: that INVARIANT/ucred freeing stuff. In-Reply-To: <200202210755.g1L7t5F93967@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG If he REALLY wants it to be zero, then we can just hold it in another field, and zero the one that is referenced..i.e.on leaving the kernel: td->td_held_ucred == td->td_ucred; td->td_ucred = NULL; and on entering the kernel: td->td_ucred = td->td_held_ucred; td->td_held_ucred = NULL; if (td->td_ucred != p->p_ucred) cred_update_thread(td); On Wed, 20 Feb 2002, Matthew Dillon wrote: > > :I REALLY want to just rip it out! > : > :julian > > Ditto. I don't mind optioning it (i.e. option to recover zeroing code, > default to rip it out), but I would much prefer to simply see it gone. > > -Matt > Matthew Dillon > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 6:52:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id 5B75E37B402 for ; Thu, 21 Feb 2002 06:52:39 -0800 (PST) Received: (qmail 19069 invoked from network); 21 Feb 2002 14:52:38 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.90.117.117]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 21 Feb 2002 14:52:38 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Thu, 21 Feb 2002 09:52:41 -0500 (EST) From: John Baldwin To: Julian Elischer Subject: RE: that INVARIANT/ucred freeing stuff. Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 21-Feb-02 Julian Elischer wrote: > > After talking with a bunch of people there's a genral concensus that we > should just remove the ucred freeing stuff at least in normal builds. I > personally canr't see how it can help in debugging in a real life non > tailored situation to have the ucreds zero'd and it's robbing us of upto > 30% of throughput in some cases. > > I REALLY want to just rip it out! Well, benchmarking debugging code usually isn't a very useful exercise. Why not go for some more optimizations and change KASSSERT() to not do anything for INVARIANTS either? That should add back in some performance as well. Seriously, getting Giant to do the free is probably biting you and doesn't need to happen 99% of the time. You can push down Giant into crfree() around the call to free() and see if that helps. Who needs debugging anyway, right? We should all just be staring at the code and "seeing" bugs. > julian -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 14: 5: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id ADE5937B402; Thu, 21 Feb 2002 14:04:57 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1LM4vQ09988; Thu, 21 Feb 2002 14:04:57 -0800 (PST) (envelope-from dillon) Date: Thu, 21 Feb 2002 14:04:57 -0800 (PST) From: Matthew Dillon Message-Id: <200202212204.g1LM4vQ09988@apollo.backplane.com> To: John Baldwin Cc: Julian Elischer , arch@FreeBSD.ORG Subject: Re: RE: that INVARIANT/ucred freeing stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG We learned a lesson with DIAGNOSTIC. You are repeating the same mistakes DIAGNOSTIC had which resulted in DIAGNOSTIC becoming essentially unusable. INVARIANTS != DIAGNOSTIC. If you want to resurrect DIAGNOSTIC you are welcome to throw this ucred-clearing code under that option. INVARIANTS is not supposed to change algorithms or optimizations wholesale. It is simply supposed to add additional sanity checks to existing algorithms and optimizations. So when the decision was made to implement td_ucred as an optimization, that means that is what we need to use. We should not go lobotomizing it for INVARIANTS. It's that simple. It is nothing even close to the absurdness of removing KASSERT for INVARIANTS that you are trying to contrast it against. What I recommend you do, John, is to add KASSERT()s at a few critical points in the code to assert that td_ucred is not being used out of context. We ... meaning Julian and myself mainly, see absolutely no point in lobotomizing an optimization that has such a huge effect on performance just because INVARIANTS is turned on. That is not the purpose of INVARIANTS. We want that code *GONE*. We would like you to acquiesce to this request. -Matt :Well, benchmarking debugging code usually isn't a very useful exercise. Why :not go for some more optimizations and change KASSSERT() to not do anything for :INVARIANTS either? That should add back in some performance as well. : : : :Seriously, getting Giant to do the free is probably biting you and doesn't need :to happen 99% of the time. You can push down Giant into crfree() around the :call to free() and see if that helps. Who needs debugging anyway, right? We :should all just be staring at the code and "seeing" bugs. : :> julian : :-- : :John Baldwin <>< http://www.FreeBSD.org/~jhb/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 17: 6:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 0DC5637B400 for ; Thu, 21 Feb 2002 17:06:15 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1M168p12213; Thu, 21 Feb 2002 17:06:08 -0800 (PST) (envelope-from dillon) Date: Thu, 21 Feb 2002 17:06:08 -0800 (PST) From: Matthew Dillon Message-Id: <200202220106.g1M168p12213@apollo.backplane.com> To: Jeff Roberson Cc: Julian Elischer , Subject: Re: Prioritized bio patches. (Updated patch) References: <20020219194142.M12686-100000@mail.chesapeake.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I was thinking potentially of some sort of priority stealing mechanism similar to the mechanism that exists for mutexes. It would not be easy to implement though. We would have to deal with vnode locks as well as buffer locks. Priority inversion can be a serious problem, especially in more heavily loaded systems. I don't think it would be safe to commit a bio priority mechanism without dealing with the problem. Another possibility is to guarentee that the I/O request will eventually go out by slowly bumping up its priority in the queue, so it doesn't get stuck indefinintely. -Matt Matthew Dillon :Yes, there is a possibility that a high priority process will completely :lock out the low priority process. I haven't followed recent scheduler :changes, but a niced process may never run at all if there is real work :no? If this is true, than the two are very similar. Anyhow, there is a :definite priority inversion issue here, but the system is somewhat :autobalancing. One would assume that the higher priority task is blocked :waiting for the low priority task to finish. So it is not issuing any :other io, which allows the low priority task to finish up. This is not :entirely optimal, I agree. I can not check for priority inversion with :lockmgr locks though, so there really is no obvious way out. : :For my application, the lockout was very desirable. With workloads such :as compiling, you'll always have some free slots to send low priority io. :This is due to the sequential nature of the workload though. It reads, :does something with the data, and then writes. In between other things :can happen. I could see how a very busy database or web server may never :give nice processes a chance to finish(until late at night, perhaps). : :You could force all file system meta data operations to run at the normal :priority. This would be good for directory operations in particular. You :can still have priority inversion issues with individual file :read/writes, but it would alleviate some of the problems. : : :Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 17:22:32 2002 Delivered-To: freebsd-arch@freebsd.org Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by hub.freebsd.org (Postfix) with ESMTP id 6328F37B402 for ; Thu, 21 Feb 2002 17:22:26 -0800 (PST) Received: from isi.edu (0v7d3jg8gvnriu2a@hbo.isi.edu [128.9.160.75]) by boreas.isi.edu (8.11.6/8.11.2) with ESMTP id g1M1MHQ02075; Thu, 21 Feb 2002 17:22:17 -0800 (PST) Message-ID: <3C759D49.2070003@isi.edu> Date: Thu, 21 Feb 2002 17:22:17 -0800 From: Lars Eggert User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.8) Gecko/20020213 X-Accept-Language: en-us, de-de MIME-Version: 1.0 To: Matthew Dillon Cc: Jeff Roberson , Julian Elischer , arch@FreeBSD.ORG Subject: Re: Prioritized bio patches. (Updated patch) References: <20020219194142.M12686-100000@mail.chesapeake.net> <200202220106.g1M168p12213@apollo.backplane.com> Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="------------ms010607060108030403080906" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a cryptographically signed message in MIME format. --------------ms010607060108030403080906 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Matthew Dillon wrote: > Priority inversion can be a serious problem, especially in more > heavily loaded systems. I don't think it would be safe to commit > a bio priority mechanism without dealing with the problem. > > Another possibility is to guarentee that the I/O request will > eventually go out by slowly bumping up its priority in the queue, > so it doesn't get stuck indefinintely. I'm looking at something similar as part of my thesis. The idea is to provide a way to use idle resource capacity (on all major resources) without interfering with regular processing. (The second part of my thesis is using this free capacity speculatively to improve performance. In some sense, it's the OS equivalent of speculative execution on CPUs.) For that, you'd want starvation and strict priorities. Priority inversion is avoided by having regular processes issue only regular-class resource requests, and idle-time processes only issue low-priority requests. I'm interested in Jeff's patches, and just started to look if I can port them to -STABLE. (Can't move to -CURRENT because I depend on ALTQ for part of my network stack idle-time hacks.) Lars -- Lars Eggert Information Sciences Institute http://www.isi.edu/larse/ University of Southern California --------------ms010607060108030403080906 Content-Type: application/x-pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIInzCC ArUwggIeoAMCAQICAwWBRzANBgkqhkiG9w0BAQIFADCBkjELMAkGA1UEBhMCWkExFTATBgNV BAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMQ8wDQYDVQQKEwZUaGF3dGUx HTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYDVQQDEx9QZXJzb25hbCBGcmVl bWFpbCBSU0EgMjAwMC44LjMwMB4XDTAxMDgyNDE2NDAwMFoXDTAyMDgyNDE2NDAwMFowVDEP MA0GA1UEBBMGRWdnZXJ0MQ0wCwYDVQQqEwRMYXJzMRQwEgYDVQQDEwtMYXJzIEVnZ2VydDEc MBoGCSqGSIb3DQEJARYNbGFyc2VAaXNpLmVkdTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkC gYEA0AvLBsD78nxcUHeHkaMgl3b4qYPnfgbf8Lh+HQP8RgGMRG/Yb+vTpkGezlwt9pkJxiD1 1uZDy4CNNJUu3gKxKSb+zRV70O+lkwwftuHoLHoH4xwo3LcQ2LGDpd+I95tUN4dfJ3TmeEcU SF50dC/SuUI4w8AlhXQ8IxrhgdayTpECAwEAAaNWMFQwKgYFK2UBBAEEITAfAgEAMBowGAIB BAQTTDJ1TXlmZkJOVWJOSkpjZFoyczAYBgNVHREEETAPgQ1sYXJzZUBpc2kuZWR1MAwGA1Ud EwEB/wQCMAAwDQYJKoZIhvcNAQECBQADgYEAheZhn0pQA8zI7U2K1ZIAl11j0a1DKxnp3GtT vOUrGRB3WvYxidvdZ1kizhEsWeXU81TkNDH0DaRqtOEeu6Q2OhB+jeKEqY7IDAJE4/fI0e+d 6PnG1hd+vEvYmsKHkmzBhPc94XUOKNWO+qVNP2NGyNI3QIDy5wX4fdcOo1S34r4wggK1MIIC HqADAgECAgMFgUcwDQYJKoZIhvcNAQECBQAwgZIxCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxX ZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEPMA0GA1UEChMGVGhhd3RlMR0wGwYD VQQLExRDZXJ0aWZpY2F0ZSBTZXJ2aWNlczEoMCYGA1UEAxMfUGVyc29uYWwgRnJlZW1haWwg UlNBIDIwMDAuOC4zMDAeFw0wMTA4MjQxNjQwMDBaFw0wMjA4MjQxNjQwMDBaMFQxDzANBgNV BAQTBkVnZ2VydDENMAsGA1UEKhMETGFyczEUMBIGA1UEAxMLTGFycyBFZ2dlcnQxHDAaBgkq hkiG9w0BCQEWDWxhcnNlQGlzaS5lZHUwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANAL ywbA+/J8XFB3h5GjIJd2+KmD534G3/C4fh0D/EYBjERv2G/r06ZBns5cLfaZCcYg9dbmQ8uA jTSVLt4CsSkm/s0Ve9DvpZMMH7bh6Cx6B+McKNy3ENixg6XfiPebVDeHXyd05nhHFEhedHQv 0rlCOMPAJYV0PCMa4YHWsk6RAgMBAAGjVjBUMCoGBStlAQQBBCEwHwIBADAaMBgCAQQEE0wy dU15ZmZCTlViTkpKY2RaMnMwGAYDVR0RBBEwD4ENbGFyc2VAaXNpLmVkdTAMBgNVHRMBAf8E AjAAMA0GCSqGSIb3DQEBAgUAA4GBAIXmYZ9KUAPMyO1NitWSAJddY9GtQysZ6dxrU7zlKxkQ d1r2MYnb3WdZIs4RLFnl1PNU5DQx9A2karThHrukNjoQfo3ihKmOyAwCROP3yNHvnej5xtYX frxL2JrCh5JswYT3PeF1DijVjvqlTT9jRsjSN0CA8ucF+H3XDqNUt+K+MIIDKTCCApKgAwIB AgIBDDANBgkqhkiG9w0BAQQFADCB0TELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4g Q2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMRowGAYDVQQKExFUaGF3dGUgQ29uc3VsdGluZzEo MCYGA1UECxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEkMCIGA1UEAxMbVGhh d3RlIFBlcnNvbmFsIEZyZWVtYWlsIENBMSswKQYJKoZIhvcNAQkBFhxwZXJzb25hbC1mcmVl bWFpbEB0aGF3dGUuY29tMB4XDTAwMDgzMDAwMDAwMFoXDTAyMDgyOTIzNTk1OVowgZIxCzAJ BgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEP MA0GA1UEChMGVGhhd3RlMR0wGwYDVQQLExRDZXJ0aWZpY2F0ZSBTZXJ2aWNlczEoMCYGA1UE AxMfUGVyc29uYWwgRnJlZW1haWwgUlNBIDIwMDAuOC4zMDCBnzANBgkqhkiG9w0BAQEFAAOB jQAwgYkCgYEA3jMypmPHCSVFPtJueCdngcXaiBmClw7jRCmKYzUqbXA8+tyu9+50bzC8M5B/ +TRxoKNtmPHDT6Jl2w36S/HW3WGl+YXNVZo1Gp2Sdagnrthy+boC9tewkd4c6avgGAOofENC UFGHgzzwObSbVIoTh/+zm51JZgAtCYnslGvpoWkCAwEAAaNOMEwwKQYDVR0RBCIwIKQeMBwx GjAYBgNVBAMTEVByaXZhdGVMYWJlbDEtMjk3MBIGA1UdEwEB/wQIMAYBAf8CAQAwCwYDVR0P BAQDAgEGMA0GCSqGSIb3DQEBBAUAA4GBAHMbbyZli/8VNEtZYortRL5Jx+gNu4+5DWomKmKE H7iHY3QcbbfPGlORS+HN5jjZ7VD0Omw0kqzmkpxuwSMBwgmn70uuct0GZ/VQby5YuLYLwVBX tewc1+8XttWIm7eiiBrtOVs5fTT8tpYYJU1q9J3Fw5EvqZa4BTxS/N3pYgNIMYICpjCCAqIC AQEwgZowgZIxCzAJBgNVBAYTAlpBMRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcT CUNhcGUgVG93bjEPMA0GA1UEChMGVGhhd3RlMR0wGwYDVQQLExRDZXJ0aWZpY2F0ZSBTZXJ2 aWNlczEoMCYGA1UEAxMfUGVyc29uYWwgRnJlZW1haWwgUlNBIDIwMDAuOC4zMAIDBYFHMAkG BSsOAwIaBQCgggFhMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8X DTAyMDIyMjAxMjIxN1owIwYJKoZIhvcNAQkEMRYEFIB6jbyi8gHpKN93dcGeQYf/jWvhMFIG CSqGSIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMC AgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGtBgsqhkiG9w0BCRACCzGBnaCBmjCBkjEL MAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3du MQ8wDQYDVQQKEwZUaGF3dGUxHTAbBgNVBAsTFENlcnRpZmljYXRlIFNlcnZpY2VzMSgwJgYD VQQDEx9QZXJzb25hbCBGcmVlbWFpbCBSU0EgMjAwMC44LjMwAgMFgUcwDQYJKoZIhvcNAQEB BQAEgYAAmfBAzjAzv3GKa5/E64zmsoi7Cr5kgt3nJzXIZk4sNjSneBt8fofGX38xH2M4lu0M olsm7kTHKXNRIiPdAy14EAsR0184TMqNhfL4MMk0kQ4cf2DXLiyQG0qNWj86Ux/nEoKZ2pGI Jq6IDg3rbLKhqLopNQ5f8g3Hsqz6ucmY/QAAAAAAAA== --------------ms010607060108030403080906-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 19:35:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by hub.freebsd.org (Postfix) with SMTP id 5F89C37B400 for ; Thu, 21 Feb 2002 19:34:59 -0800 (PST) Received: (qmail 30681 invoked by uid 0); 22 Feb 2002 03:34:55 -0000 Received: from pd9e16b60.dip.t-dialin.net (HELO forge.local) (217.225.107.96) by mail.gmx.net (mp009-rz3) with SMTP; 22 Feb 2002 03:34:55 -0000 Received: from tmm by forge.local with local (Exim 3.34 #1) id 16e6U4-0001xA-00 for ; Fri, 22 Feb 2002 04:34:56 +0100 Date: Fri, 22 Feb 2002 04:34:55 +0100 From: Thomas Moestl To: freebsd-arch@FreeBSD.org Subject: Re: adding more endian conversion and bus space functions Message-ID: <20020222033455.GA289@crow.dom2ip.de> Mail-Followup-To: freebsd-arch@FreeBSD.org References: <20020111005207.GA7246@crow.dom2ip.de> <20020220031842.GF282@crow.dom2ip.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020220031842.GF282@crow.dom2ip.de> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 2002/02/20 at 04:18:42 +0100, Thomas Moestl wrote: > On Fri, 2002/01/11 at 01:52:07 +0100, Thomas Moestl wrote: > > I'd like to propose some additions to the kernel endian conversion and > > bus space functions to aid the sparc64 port, taken from NetBSD: > > I have attached a diff that implements these new functions, with minor > deviations from the original proposal I've attached an updated version of the patch, which implements some helpful suggestions by Mike Barcroft to reduce name space pollution when the new functions will be exposed to userland, as well as some cleanups. This is tested on i386, alpha (buildworld, I hope to get the results of a GENERIC kernel build tomorrow) and sparc64. I'd like to commit it soon if I can get somebody to test it on ia64, hopefully during the weekend. - thomas ==== //depot/projects/sparc64/lib/libstand/Makefile#2 - /home/tmm/p4/sparc64/lib/libstand/Makefile ==== --- /tmp/tmp.8072.0 Thu Feb 21 19:06:07 2002 +++ /home/tmm/p4/sparc64/lib/libstand/Makefile Tue Feb 19 23:48:33 2002 @@ -33,6 +33,10 @@ # private (pruned) versions of libc string functions SRCS+= strcasecmp.c +# byte order functions from libc +.PATH: ${.CURDIR}/../libc/${MACHINE_ARCH}/net +SRCS+= htons.S ntohs.S htonl.S ntohl.S + # string functions from libc .PATH: ${.CURDIR}/../libc/string .if ${MACHINE_ARCH} == "i386" || ${MACHINE_ARCH} == "powerpc" || \ @@ -50,9 +54,6 @@ strncat.c strncmp.c strncpy.c strpbrk.c strrchr.c strsep.c \ strspn.c strstr.c strtok.c swab.c -.PATH: ${.CURDIR}/../libc/alpha/net -SRCS+= htons.S ntohs.S htonl.S ntohl.S - SRCS+= __divqu.S __divq.S __divlu.S __divl.S SRCS+= __remqu.S __remq.S __remlu.S __reml.S @@ -99,9 +100,6 @@ strcmp.c strcpy.c strcspn.c strlen.c \ strncat.c strncmp.c strncpy.c strpbrk.c strrchr.c strsep.c \ strspn.c strstr.c strtok.c swab.c - -.PATH: ${.CURDIR}/../libc/ia64/net -SRCS+= htons.S ntohs.S htonl.S ntohl.S .PATH: ${.CURDIR}/../libc/ia64/gen SRCS+= __divdi3.S __divsi3.S __moddi3.S __modsi3.S ==== //depot/projects/sparc64/sys/alpha/include/bus.h#1 - /home/tmm/p4/sparc64/sys/alpha/include/bus.h ==== --- /tmp/tmp.8072.1 Thu Feb 21 19:06:08 2002 +++ /home/tmm/p4/sparc64/sys/alpha/include/bus.h Wed Feb 20 01:37:00 2002 @@ -366,6 +366,70 @@ (t)->ab_ops->abo_barrier(t, (h)+(o), l, f) /* + * Stream accesses are the same as normal accesses on alpha; there are no + * supported bus systems with an endianess different from the host one. + */ +#define bus_space_read_stream_1(t, h, o) bus_space_read_1((t), (h), (o)) +#define bus_space_read_stream_2(t, h, o) bus_space_read_2((t), (h), (o)) +#define bus_space_read_stream_4(t, h, o) bus_space_read_4((t), (h), (o)) + +#define bus_space_read_multi_stream_1(t, h, o, a, c) \ + bus_space_read_multi_1((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_2(t, h, o, a, c) \ + bus_space_read_multi_2((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_4(t, h, o, a, c) \ + bus_space_read_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_write_stream_1(t, h, o, v) \ + bus_space_write_1((t), (h), (o), (v)) +#define bus_space_write_stream_2(t, h, o, v) \ + bus_space_write_2((t), (h), (o), (v)) +#define bus_space_write_stream_4(t, h, o, v) \ + bus_space_write_4((t), (h), (o), (v)) + +#define bus_space_write_multi_stream_1(t, h, o, a, c) \ + bus_space_write_multi_1((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_2(t, h, o, a, c) \ + bus_space_write_multi_2((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_4(t, h, o, a, c) \ + bus_space_write_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_set_multi_stream_1(t, h, o, v, c) \ + bus_space_set_multi_1((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_2(t, h, o, v, c) \ + bus_space_set_multi_2((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_4(t, h, o, v, c) \ + bus_space_set_multi_4((t), (h), (o), (v), (c)) + +#define bus_space_read_region_stream_1(t, h, o, a, c) \ + bus_space_read_region_1((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_2(t, h, o, a, c) \ + bus_space_read_region_2((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_4(t, h, o, a, c) \ + bus_space_read_region_4((t), (h), (o), (a), (c)) + +#define bus_space_write_region_stream_1(t, h, o, a, c) \ + bus_space_write_region_1((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_2(t, h, o, a, c) \ + bus_space_write_region_2((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_4(t, h, o, a, c) \ + bus_space_write_region_4((t), (h), (o), (a), (c)) + +#define bus_space_set_region_stream_1(t, h, o, v, c) \ + bus_space_set_region_1((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_2(t, h, o, v, c) \ + bus_space_set_region_2((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_4(t, h, o, v, c) \ + bus_space_set_region_4((t), (h), (o), (v), (c)) + +#define bus_space_copy_region_stream_1(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_1((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_2(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_2((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_4(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_4((t), (h1), (o1), (h2), (o2), (c)) + +/* * Flags used in various bus DMA methods. */ #define BUS_DMA_WAITOK 0x00 /* safe to sleep (pseudo-flag) */ ==== //depot/projects/sparc64/sys/alpha/include/endian.h#3 - /home/tmm/p4/sparc64/sys/alpha/include/endian.h ==== --- /tmp/tmp.8072.2 Thu Feb 21 19:06:08 2002 +++ /home/tmm/p4/sparc64/sys/alpha/include/endian.h Thu Feb 21 02:03:21 2002 @@ -56,6 +56,18 @@ #define PDP_ENDIAN 3412 /* LSB first in word, MSW first in long */ #define BYTE_ORDER LITTLE_ENDIAN + +#ifdef _KERNEL +#ifndef _BSWAP16_DEFINED +#define _BSWAP16_DEFINED +__uint16_t __bswap16(__uint16_t); +#endif +#ifndef _BSWAP32_DEFINED +#define _BSWAP32_DEFINED +__uint32_t __bswap32(__uint32_t); +#endif +#endif /* _KERNEL */ + #endif /* !_POSIX_SOURCE */ #endif /* !_MACHINE_ENDIAN_H_ */ ==== //depot/projects/sparc64/sys/conf/files.alpha#8 - /home/tmm/p4/sparc64/sys/conf/files.alpha ==== --- /tmp/tmp.8072.3 Thu Feb 21 19:06:08 2002 +++ /home/tmm/p4/sparc64/sys/conf/files.alpha Tue Feb 19 20:13:51 2002 @@ -215,9 +215,7 @@ isa/syscons_isa.c optional sc isa/vga_isa.c optional vga kern/subr_diskmbr.c standard -libkern/alpha/htonl.S standard -libkern/alpha/htons.S standard -libkern/alpha/ntohl.S standard -libkern/alpha/ntohs.S standard +libkern/alpha/bswap16.S standard +libkern/alpha/bswap32.S standard libkern/bcmp.c standard libkern/ffs.c standard ==== //depot/projects/sparc64/sys/conf/files.ia64#9 - /home/tmm/p4/sparc64/sys/conf/files.ia64 ==== --- /tmp/tmp.8072.4 Thu Feb 21 19:06:08 2002 +++ /home/tmm/p4/sparc64/sys/conf/files.ia64 Tue Feb 19 20:14:56 2002 @@ -99,10 +99,8 @@ isa/syscons_isa.c optional sc isa/vga_isa.c optional vga kern/subr_diskmbr.c standard -libkern/ia64/htonl.S standard -libkern/ia64/htons.S standard -libkern/ia64/ntohl.S standard -libkern/ia64/ntohs.S standard +libkern/ia64/bswap16.S standard +libkern/ia64/bswap32.S standard libkern/ia64/__divsi3.S standard libkern/ia64/__modsi3.S standard libkern/ia64/__udivsi3.S standard ==== //depot/projects/sparc64/sys/dev/iir/iir.h#1 - /home/tmm/p4/sparc64/sys/dev/iir/iir.h ==== --- /tmp/tmp.8072.5 Thu Feb 21 19:06:09 2002 +++ /home/tmm/p4/sparc64/sys/dev/iir/iir.h Wed Feb 20 01:16:50 2002 @@ -372,10 +372,8 @@ #define GDT_SCRATCH_SZ 3072 /* 3KB scratch buffer */ /* macros */ -#define htole32(v) (v) -#define htole16(v) (v) -#define letoh32(v) (v) -#define letoh16(v) (v) +#define letoh32(v) le32toh(v) +#define letoh16(v) le16toh(v) /* Map minor numbers to device identity */ #define LUN_MASK 0x0007 ==== //depot/projects/sparc64/sys/dev/usb/ohci.c#7 - /home/tmm/p4/sparc64/sys/dev/usb/ohci.c ==== --- /tmp/tmp.8072.6 Thu Feb 21 19:06:09 2002 +++ /home/tmm/p4/sparc64/sys/dev/usb/ohci.c Wed Feb 20 00:40:01 2002 @@ -96,20 +96,6 @@ #define DPRINTFN(n,x) #endif -/* - * The OHCI controller is little endian, so on big endian machines - * the data strored in memory needs to be swapped. - */ -#if defined(__FreeBSD__) -#if BYTE_ORDER == BIG_ENDIAN -#define htole32(x) (bswap32(x)) -#define le32toh(x) (bswap32(x)) -#else -#define htole32(x) (x) -#define le32toh(x) (x) -#endif -#endif - struct ohci_pipe; Static ohci_soft_ed_t *ohci_alloc_sed(ohci_softc_t *); ==== //depot/projects/sparc64/sys/dev/usb/uhci.c#9 - /home/tmm/p4/sparc64/sys/dev/usb/uhci.c ==== --- /tmp/tmp.8072.7 Thu Feb 21 19:06:09 2002 +++ /home/tmm/p4/sparc64/sys/dev/usb/uhci.c Wed Feb 20 00:39:54 2002 @@ -112,7 +112,7 @@ * The UHCI controller is little endian, so on big endian machines * the data strored in memory needs to be swapped. */ -#if defined(__FreeBSD__) || defined(__OpenBSD__) +#if defined(__OpenBSD__) #if BYTE_ORDER == BIG_ENDIAN #define htole32(x) (bswap32(x)) #define le32toh(x) (bswap32(x)) ==== //depot/projects/sparc64/sys/dev/usb/usb_port.h#4 - /home/tmm/p4/sparc64/sys/dev/usb/usb_port.h ==== ==== //depot/projects/sparc64/sys/dev/wi/if_wi.c#13 - /home/tmm/p4/sparc64/sys/dev/wi/if_wi.c ==== --- /tmp/tmp.8072.8 Thu Feb 21 19:06:10 2002 +++ /home/tmm/p4/sparc64/sys/dev/wi/if_wi.c Wed Feb 20 00:34:30 2002 @@ -125,11 +125,9 @@ #endif /* - * The following is for compatibility with NetBSD, but should really be - * brought in from NetBSD en toto. + * The following is for compatibility with NetBSD. */ -#define le16toh(a) (a) -#define LE16TOH(a) +#define LE16TOH(a) ((a) = le16toh((a))) static void wi_intr __P((void *)); static void wi_reset __P((struct wi_softc *)); ==== //depot/projects/sparc64/sys/i386/include/bus.h#2 - /home/tmm/p4/sparc64/sys/i386/include/bus.h ==== --- /tmp/tmp.8072.9 Thu Feb 21 19:06:10 2002 +++ /home/tmm/p4/sparc64/sys/i386/include/bus.h Wed Feb 20 01:36:38 2002 @@ -43,4 +43,68 @@ #endif #include +/* + * Stream accesses are the same as normal accesses on i386/pc98; there are no + * supported bus systems with an endianess different from the host one. + */ +#define bus_space_read_stream_1(t, h, o) bus_space_read_1((t), (h), (o)) +#define bus_space_read_stream_2(t, h, o) bus_space_read_2((t), (h), (o)) +#define bus_space_read_stream_4(t, h, o) bus_space_read_4((t), (h), (o)) + +#define bus_space_read_multi_stream_1(t, h, o, a, c) \ + bus_space_read_multi_1((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_2(t, h, o, a, c) \ + bus_space_read_multi_2((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_4(t, h, o, a, c) \ + bus_space_read_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_write_stream_1(t, h, o, v) \ + bus_space_write_1((t), (h), (o), (v)) +#define bus_space_write_stream_2(t, h, o, v) \ + bus_space_write_2((t), (h), (o), (v)) +#define bus_space_write_stream_4(t, h, o, v) \ + bus_space_write_4((t), (h), (o), (v)) + +#define bus_space_write_multi_stream_1(t, h, o, a, c) \ + bus_space_write_multi_1((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_2(t, h, o, a, c) \ + bus_space_write_multi_2((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_4(t, h, o, a, c) \ + bus_space_write_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_set_multi_stream_1(t, h, o, v, c) \ + bus_space_set_multi_1((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_2(t, h, o, v, c) \ + bus_space_set_multi_2((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_4(t, h, o, v, c) \ + bus_space_set_multi_4((t), (h), (o), (v), (c)) + +#define bus_space_read_region_stream_1(t, h, o, a, c) \ + bus_space_read_region_1((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_2(t, h, o, a, c) \ + bus_space_read_region_2((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_4(t, h, o, a, c) \ + bus_space_read_region_4((t), (h), (o), (a), (c)) + +#define bus_space_write_region_stream_1(t, h, o, a, c) \ + bus_space_write_region_1((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_2(t, h, o, a, c) \ + bus_space_write_region_2((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_4(t, h, o, a, c) \ + bus_space_write_region_4((t), (h), (o), (a), (c)) + +#define bus_space_set_region_stream_1(t, h, o, v, c) \ + bus_space_set_region_1((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_2(t, h, o, v, c) \ + bus_space_set_region_2((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_4(t, h, o, v, c) \ + bus_space_set_region_4((t), (h), (o), (v), (c)) + +#define bus_space_copy_region_stream_1(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_1((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_2(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_2((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_4(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_4((t), (h1), (o1), (h2), (o2), (c)) + #endif /* _I386_BUS_H_ */ ==== //depot/projects/sparc64/sys/i386/include/endian.h#6 - /home/tmm/p4/sparc64/sys/i386/include/endian.h ==== --- /tmp/tmp.8072.10 Thu Feb 21 19:06:10 2002 +++ /home/tmm/p4/sparc64/sys/i386/include/endian.h Thu Feb 21 01:53:24 2002 @@ -58,12 +58,13 @@ #define BYTE_ORDER LITTLE_ENDIAN #endif /* ! _POSIX_SOURCE */ +#ifdef _KERNEL #ifdef __GNUC__ -__BEGIN_DECLS - +#ifndef _BSWAP32_DEFINED +#define _BSWAP32_DEFINED static __inline __uint32_t -__htonl(__uint32_t __x) +__bswap32(__uint32_t __x) { #if defined(_KERNEL) && (defined(I486_CPU) || defined(I586_CPU) || defined(I686_CPU)) && !defined(I386_CPU) __asm ("bswap %0" : "+r" (__x)); @@ -75,30 +76,20 @@ #endif return __x; } +#endif +#ifndef _BSWAP16_DEFINED +#define _BSWAP16_DEFINED static __inline __uint16_t -__htons(__uint16_t __x) +__bswap16(__uint16_t __x) { __asm ("xchgb %h0, %b0" : "+q" (__x)); return __x; } +#endif -static __inline __uint32_t -__ntohl(__uint32_t __x) -{ - - return (__htonl(__x)); -} - -static __inline __uint16_t -__ntohs(__uint16_t __x) -{ - - return (__htons(__x)); -} - -__END_DECLS +#endif /* _KERNEL */ #endif /* __GNUC__ */ ==== //depot/projects/sparc64/sys/ia64/include/bus.h#1 - /home/tmm/p4/sparc64/sys/ia64/include/bus.h ==== --- /tmp/tmp.8072.11 Thu Feb 21 19:06:10 2002 +++ /home/tmm/p4/sparc64/sys/ia64/include/bus.h Wed Feb 20 01:42:54 2002 @@ -1000,6 +1000,70 @@ #endif } +/* + * Stream accesses are the same as normal accesses on ia64; there are no + * supported bus systems with an endianess different from the host one. + */ +#define bus_space_read_stream_1(t, h, o) bus_space_read_1((t), (h), (o)) +#define bus_space_read_stream_2(t, h, o) bus_space_read_2((t), (h), (o)) +#define bus_space_read_stream_4(t, h, o) bus_space_read_4((t), (h), (o)) + +#define bus_space_read_multi_stream_1(t, h, o, a, c) \ + bus_space_read_multi_1((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_2(t, h, o, a, c) \ + bus_space_read_multi_2((t), (h), (o), (a), (c)) +#define bus_space_read_multi_stream_4(t, h, o, a, c) \ + bus_space_read_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_write_stream_1(t, h, o, v) \ + bus_space_write_1((t), (h), (o), (v)) +#define bus_space_write_stream_2(t, h, o, v) \ + bus_space_write_2((t), (h), (o), (v)) +#define bus_space_write_stream_4(t, h, o, v) \ + bus_space_write_4((t), (h), (o), (v)) + +#define bus_space_write_multi_stream_1(t, h, o, a, c) \ + bus_space_write_multi_1((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_2(t, h, o, a, c) \ + bus_space_write_multi_2((t), (h), (o), (a), (c)) +#define bus_space_write_multi_stream_4(t, h, o, a, c) \ + bus_space_write_multi_4((t), (h), (o), (a), (c)) + +#define bus_space_set_multi_stream_1(t, h, o, v, c) \ + bus_space_set_multi_1((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_2(t, h, o, v, c) \ + bus_space_set_multi_2((t), (h), (o), (v), (c)) +#define bus_space_set_multi_stream_4(t, h, o, v, c) \ + bus_space_set_multi_4((t), (h), (o), (v), (c)) + +#define bus_space_read_region_stream_1(t, h, o, a, c) \ + bus_space_read_region_1((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_2(t, h, o, a, c) \ + bus_space_read_region_2((t), (h), (o), (a), (c)) +#define bus_space_read_region_stream_4(t, h, o, a, c) \ + bus_space_read_region_4((t), (h), (o), (a), (c)) + +#define bus_space_write_region_stream_1(t, h, o, a, c) \ + bus_space_write_region_1((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_2(t, h, o, a, c) \ + bus_space_write_region_2((t), (h), (o), (a), (c)) +#define bus_space_write_region_stream_4(t, h, o, a, c) \ + bus_space_write_region_4((t), (h), (o), (a), (c)) + +#define bus_space_set_region_stream_1(t, h, o, v, c) \ + bus_space_set_region_1((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_2(t, h, o, v, c) \ + bus_space_set_region_2((t), (h), (o), (v), (c)) +#define bus_space_set_region_stream_4(t, h, o, v, c) \ + bus_space_set_region_4((t), (h), (o), (v), (c)) + +#define bus_space_copy_region_stream_1(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_1((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_2(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_2((t), (h1), (o1), (h2), (o2), (c)) +#define bus_space_copy_region_stream_4(t, h1, o1, h2, o2, c) \ + bus_space_copy_region_4((t), (h1), (o1), (h2), (o2), (c)) + #endif /* defined(_MACHINE_BUS_PIO_H_) || defined(_MACHINE_BUS_MEMIO_H_) */ #if 0 /* Cause a link error for bus_space_copy_8 */ ==== //depot/projects/sparc64/sys/ia64/include/endian.h#5 - /home/tmm/p4/sparc64/sys/ia64/include/endian.h ==== --- /tmp/tmp.8072.12 Thu Feb 21 19:06:10 2002 +++ /home/tmm/p4/sparc64/sys/ia64/include/endian.h Thu Feb 21 02:14:18 2002 @@ -59,49 +59,54 @@ #define BYTE_ORDER LITTLE_ENDIAN #endif /* !_POSIX_SOURCE */ +#ifdef _KERNEL #ifdef __GNUC__ -__BEGIN_DECLS - +#ifndef _BSWAP64_DEFINED +#define _BSWAP64_DEFINED static __inline __uint64_t -__uint8_swap_uint64(__uint64_t __x) +__bswap64(__uint64_t __x) { __uint64_t __r; __asm __volatile("mux1 %0=%1,@rev" : "=r" (__r) : "r"(__x)); return __r; } +#endif +#ifndef _BSWAP32_DEFINED +#define _BSWAP32_DEFINED static __inline __uint32_t -__htonl(__uint32_t __x) -{ - - return (__uint8_swap_uint64(__x) >> 32); -} - -static __inline __uint16_t -__htons(__uint16_t __x) -{ - - return (__uint8_swap_uint64(__x) >> 48); -} - -static __inline __uint32_t -__ntohl(__uint32_t __x) +__bswap32(__uint32_t __x) { - return (__uint8_swap_uint64(__x) >> 32); + return (__bswap64(__x) >> 32); } +#endif +#ifndef _BSWAP16_DEFINED +#define _BSWAP16_DEFINED static __inline __uint16_t -__ntohs(__uint16_t __x) +__bswap16(__uint16_t __x) { - return (__uint8_swap_uint64(__x) >> 48); + return (__bswap64(__x) >> 48); } -__END_DECLS +#endif +#else /* !__GNUC__ */ +/* XXX: use the libkern versions for now; these might go away soon. */ +#ifndef _BSWAP16_DEFINED +#define _BSWAP16_DEFINED +__uint16_t __bswap16(__uint16_t); +#endif +#ifndef _BSWAP32_DEFINED +#define _BSWAP32_DEFINED +__uint32_t __bswap32(__uint32_t); +#endif #endif /* __GNUC__ */ + +#endif /* _KERNEL */ #endif /* !_MACHINE_ENDIAN_H_ */ ==== //depot/projects/sparc64/sys/libkern/alpha/byte_swap_2.S#2 - /home/tmm/p4/sparc64/sys/libkern/alpha/byte_swap_2.S ==== --- /tmp/tmp.8072.14 Thu Feb 21 19:06:11 2002 +++ /home/tmm/p4/sparc64/sys/libkern/alpha/byte_swap_2.S Tue Feb 19 20:06:20 2002 @@ -30,8 +30,8 @@ #include -#if !defined(ALIAS) || !defined(NAME) -#error ALIAS or NAME not defined +#ifndef NAME +#error NAME not defined #endif /* @@ -39,7 +39,6 @@ * * Argument is an unsigned 2-byte integer (u_int16_t). */ -XLEAF(ALIAS, 1) LEAF(NAME, 1) /* a0 contains 0x0123 */ extbl a0, 0, t0 /* t0 = 0x 23 */ extbl a0, 1, t1 /* t1 = 0x 01 */ ==== //depot/projects/sparc64/sys/libkern/alpha/byte_swap_4.S#2 - /home/tmm/p4/sparc64/sys/libkern/alpha/byte_swap_4.S ==== --- /tmp/tmp.8072.15 Thu Feb 21 19:06:11 2002 +++ /home/tmm/p4/sparc64/sys/libkern/alpha/byte_swap_4.S Tue Feb 19 20:06:52 2002 @@ -30,8 +30,8 @@ #include -#if !defined(ALIAS) || !defined(NAME) -#error ALIAS or NAME not defined +#ifndef NAME +#error NAME not defined #endif /* @@ -39,7 +39,6 @@ * * Argument is an unsigned 4-byte integer (u_int32_t). */ -XLEAF(ALIAS, 1) LEAF(NAME, 1) /* a0 contains 0x01234567 */ extbl a0, 0, t0 /* t0 = 0x 67 */ extbl a0, 1, t1 /* t1 = 0x 45 */ ==== //depot/projects/sparc64/sys/libkern/ia64/byte_swap_2.S#3 - /home/tmm/p4/sparc64/sys/libkern/ia64/byte_swap_2.S ==== --- /tmp/tmp.8072.16 Thu Feb 21 19:06:11 2002 +++ /home/tmm/p4/sparc64/sys/libkern/ia64/byte_swap_2.S Tue Feb 19 20:04:55 2002 @@ -30,8 +30,8 @@ #include -#if !defined(ALIAS) || !defined(NAME) -#error ALIAS or NAME not defined +#ifndef NAME +#error NAME not defined #endif /* @@ -39,7 +39,6 @@ * * Argument is an unsigned 2-byte integer (u_int16_t). */ -WEAK_ALIAS(ALIAS, NAME) ENTRY(NAME, 1) mux1 r16=in0,@rev ;; ==== //depot/projects/sparc64/sys/libkern/ia64/byte_swap_4.S#3 - /home/tmm/p4/sparc64/sys/libkern/ia64/byte_swap_4.S ==== --- /tmp/tmp.8072.17 Thu Feb 21 19:06:11 2002 +++ /home/tmm/p4/sparc64/sys/libkern/ia64/byte_swap_4.S Tue Feb 19 20:05:25 2002 @@ -30,8 +30,8 @@ #include -#if !defined(ALIAS) || !defined(NAME) -#error ALIAS or NAME not defined +#ifndef NAME +#error NAME not defined #endif /* @@ -39,7 +39,6 @@ * * Argument is an unsigned 4-byte integer (u_int32_t). */ -WEAK_ALIAS(ALIAS, NAME) ENTRY(NAME, 1) mux1 r16=in0,@rev ;; ==== //depot/projects/sparc64/sys/powerpc/include/endian.h#3 - /home/tmm/p4/sparc64/sys/powerpc/include/endian.h ==== --- /tmp/tmp.8072.18 Thu Feb 21 19:06:11 2002 +++ /home/tmm/p4/sparc64/sys/powerpc/include/endian.h Tue Feb 19 21:18:12 2002 @@ -60,18 +60,6 @@ #ifndef _KERNEL #include - -__BEGIN_DECLS -__uint32_t __htonl __P((__uint32_t)); -__uint16_t __htons __P((__uint16_t)); -__uint32_t __ntohl __P((__uint32_t)); -__uint16_t __ntohs __P((__uint16_t)); -__END_DECLS #endif /* _KERNEL */ - -#define __htonl(x) (x) -#define __htons(x) (x) -#define __ntohl(x) (x) -#define __ntohs(x) (x) #endif /* !_MACHINE_ENDIAN_H_ */ ==== //depot/projects/sparc64/sys/sparc64/include/endian.h#8 - /home/tmm/p4/sparc64/sys/sparc64/include/endian.h ==== --- /tmp/tmp.8072.19 Thu Feb 21 19:06:12 2002 +++ /home/tmm/p4/sparc64/sys/sparc64/include/endian.h Tue Feb 19 20:38:23 2002 @@ -57,59 +57,4 @@ #define BYTE_ORDER BIG_ENDIAN #endif /* !_POSIX_SOURCE */ -#define __htonl(x) (x) -#define __htons(x) (x) -#define __ntohl(x) (x) -#define __ntohs(x) (x) - -__BEGIN_DECLS -__uint16_t bswap16 __P((__uint16_t)); -__uint32_t bswap32 __P((__uint32_t)); -__uint64_t bswap64 __P((__uint64_t)); -__END_DECLS - -#ifdef _KERNEL -/* XXX: these three macros should be moved! */ -static __inline __uint16_t -__swap16(__uint16_t x) -{ - return ((x >> 8) | ((x << 8) & 0xff00U)); -} - -static __inline __uint32_t -__swap32(__uint32_t x) -{ - return ((x >> 24) | ((x >> 8) & 0xff00U) | ((x << 8) & 0xff0000U) | - ((x << 24) & 0xff000000U)); -} - -static __inline __uint64_t -__swap64(__uint64_t x) -{ - return ((x >> 56) | ((x >> 40) & 0xff00UL) | ((x >> 24) & 0xff0000UL) | - ((x >> 8) & 0xff000000UL) | ((x << 8) & 0xff00000000UL) | - ((x << 24) & 0xff0000000000UL) | ((x << 40) & 0xff000000000000UL) | - ((x << 56))); -} - -#define htobe16(x) (x) -#define htobe32(x) (x) -#define htobe64(x) (x) -#define htole16(x) __swap16((x)) -#define htole32(x) __swap32((x)) -#define htole64(x) __swap64((x)) - -#define be16toh(x) (x) -#define be32toh(x) (x) -#define be64toh(x) (x) -#define le16toh(x) __swap16((x)) -#define le32toh(x) __swap32((x)) -#define le64toh(x) __swap64((x)) - -#define bswap16(x) __swap16((x)) -#define bswap32(x) __swap32((x)) -#define bswap64(x) __swap64((x)) - -#endif - #endif /* !_MACHINE_ENDIAN_H_ */ ==== //depot/projects/sparc64/sys/sys/imgact_aout.h#3 - /home/tmm/p4/sparc64/sys/sys/imgact_aout.h ==== --- /tmp/tmp.8072.20 Thu Feb 21 19:06:12 2002 +++ /home/tmm/p4/sparc64/sys/sys/imgact_aout.h Wed Feb 20 00:51:10 2002 @@ -50,13 +50,13 @@ ((mag) & 0xffff) ) #define N_GETMAGIC_NET(ex) \ - (__ntohl((ex).a_midmag) & 0xffff) + (ntohl((ex).a_midmag) & 0xffff) #define N_GETMID_NET(ex) \ - ((__ntohl((ex).a_midmag) >> 16) & 0x03ff) + ((ntohl((ex).a_midmag) >> 16) & 0x03ff) #define N_GETFLAG_NET(ex) \ - ((__ntohl((ex).a_midmag) >> 26) & 0x3f) + ((ntohl((ex).a_midmag) >> 26) & 0x3f) #define N_SETMAGIC_NET(ex,mag,mid,flag) \ - ( (ex).a_midmag = __htonl( (((flag)&0x3f)<<26) | (((mid)&0x03ff)<<16) \ + ( (ex).a_midmag = htonl( (((flag)&0x3f)<<26) | (((mid)&0x03ff)<<16) \ | (((mag)&0xffff)) ) ) #define N_ALIGN(ex,x) \ ==== //depot/projects/sparc64/sys/sys/mchain.h#2 - /home/tmm/p4/sparc64/sys/sys/mchain.h ==== --- /tmp/tmp.8072.21 Thu Feb 21 19:06:12 2002 +++ /home/tmm/p4/sparc64/sys/sys/mchain.h Tue Feb 19 22:22:06 2002 @@ -36,6 +36,28 @@ #include +#ifdef _KERNEL + +/* + * XXX: remove these defines and change the function calls in the code. Use + * it unconditionally if (when) the extended byte order functions become + * available in user space. + */ +#define htoles(x) htole16((x)) +#define letohs(x) le16toh((x)) +#define htolel(x) htole32((x)) +#define letohl(x) le32toh((x)) +#define htoleq(x) htole64((int64_t)(x)) +#define letohq(x) le64toh((int64_t)(x)) + +#define htobes(x) htobe16((x)) +#define betohs(x) be16toh((x)) +#define htobel(x) htobe32((x)) +#define betohl(x) be32toh((x)) +#define htobeq(x) htobe64((int64_t)(x)) +#define betohq(x) be64toh((int64_t)(x)) + +#else /* * This macros probably belongs to the endian.h */ @@ -78,6 +100,7 @@ #define letohl(x) ((u_int32_t)(x)) */ #endif /* (BYTE_ORDER == LITTLE_ENDIAN) */ +#endif /* _KERNEL */ #ifdef _KERNEL ==== //depot/projects/sparc64/sys/sys/param.h#10 - /home/tmm/p4/sparc64/sys/sys/param.h ==== --- /tmp/tmp.8072.22 Thu Feb 21 19:06:12 2002 +++ /home/tmm/p4/sparc64/sys/sys/param.h Thu Feb 21 18:58:34 2002 @@ -188,28 +188,92 @@ #define MAX(a,b) (((a)>(b))?(a):(b)) #endif +#ifdef _KERNEL /* - * Kernel exposed versions of byteorder(3) functions. - * - * XXX this section should only be defined in the kernel, but some userland - * software utilizes it. + * Extended byte order support functions, for kernel use only currently. + * First, generic implementation of the byte swapping functions for those + * architectures that do not have optimized variants of each. */ -#ifndef _BYTEORDER_FUNC_DEFINED -#define _BYTEORDER_FUNC_DEFINED -#define htonl(x) __htonl(x) -#define htons(x) __htons(x) -#define ntohl(x) __ntohl(x) -#define ntohs(x) __ntohs(x) +#ifndef _BSWAP16_DEFINED +#define _BSWAP16_DEFINED +static __inline __uint16_t +__bswap16(__uint16_t x) +{ + return ((x >> 8) | ((x << 8) & 0xff00U)); +} #endif +#ifndef _BSWAP32_DEFINED +#define _BSWAP32_DEFINED +static __inline __uint32_t +__bswap32(__uint32_t x) +{ + return ((x >> 24) | ((x >> 8) & 0xff00U) | ((x << 8) & 0xff0000U) | + ((x << 24) & 0xff000000U)); +} +#endif + +#ifndef _BSWAP64_DEFINED +#define _BSWAP64_DEFINED +static __inline __uint64_t +__bswap64(__uint64_t x) +{ + return ((x >> 56) | ((x >> 40) & 0xff00UL) | ((x >> 24) & 0xff0000UL) | + ((x >> 8) & 0xff000000UL) | ((x << 8) & 0xff00000000UL) | + ((x << 24) & 0xff0000000000UL) | ((x << 40) & 0xff000000000000UL) | + ((x << 56))); +} +#endif + +#define bswap16(x) __bswap16(x) +#define bswap32(x) __bswap32(x) +#define bswap64(x) __bswap64(x) + +#if BYTE_ORDER == LITTLE_ENDIAN +#define htobe16(x) bswap16((x)) +#define htobe32(x) bswap32((x)) +#define htobe64(x) bswap64((x)) +#define htole16(x) ((__uint16_t)(x)) +#define htole32(x) ((__uint32_t)(x)) +#define htole64(x) ((__uint64_t)(x)) + +#define be16toh(x) bswap16((x)) +#define be32toh(x) bswap32((x)) +#define be64toh(x) bswap64((x)) +#define le16toh(x) ((__uint16_t)(x)) +#define le32toh(x) ((__uint32_t)(x)) +#define le64toh(x) ((__uint64_t)(x)) +#else /* BYTE_ORDER != LITTLE_ENDIAN */ +#define htobe16(x) ((__uint16_t)(x)) +#define htobe32(x) ((__uint32_t)(x)) +#define htobe64(x) ((__uint64_t)(x)) +#define htole16(x) bswap16((x)) +#define htole32(x) bswap32((x)) +#define htole64(x) bswap64((x)) + +#define be16toh(x) ((__uint16_t)(x)) +#define be32toh(x) ((__uint32_t)(x)) +#define be64toh(x) ((__uint64_t)(x)) +#define le16toh(x) bswap16((x)) +#define le32toh(x) bswap32((x)) +#define le64toh(x) bswap64((x)) +#endif /* BYTE_ORDER */ + +#define htonl(x) htobe32((x)) +#define htons(x) htobe16((x)) +#define ntohl(x) be32toh((x)) +#define ntohs(x) be16toh((x)) + +#endif /* _KERNEL */ + /* * XXX deprecated uppercase variants for byteorder(3) functions. */ #ifndef _POSIX_SOURCE -#define NTOHL(x) ((x) = __ntohl(x)) -#define NTOHS(x) ((x) = __ntohs(x)) -#define HTONL(x) ((x) = __htonl(x)) -#define HTONS(x) ((x) = __htons(x)) +#define NTOHL(x) ((x) = ntohl(x)) +#define NTOHS(x) ((x) = ntohs(x)) +#define HTONL(x) ((x) = htonl(x)) +#define HTONS(x) ((x) = htons(x)) #endif /* _POSIX_SOURCE */ /* --- /dev/null Thu Feb 21 17:45:56 2002 +++ /home/tmm/p4/sparc64/sys/libkern/alpha/bswap16.S Tue Feb 19 20:08:13 2002 @@ -0,0 +1,35 @@ +/* + * Copyright (c) 1996 Carnegie-Mellon University. + * All rights reserved. + * + * Author: Chris G. Demetriou + * + * Permission to use, copy, modify and distribute this software and + * its documentation is hereby granted, provided that both the copyright + * notice and this permission notice appear in all copies of the + * software, derivative works or modified versions, and any portions + * thereof, and that both notices appear in supporting documentation. + * + * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" + * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND + * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. + * + * Carnegie Mellon requests users of this software to return to + * + * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU + * School of Computer Science + * Carnegie Mellon University + * Pittsburgh PA 15213-3890 + * + * any improvements or extensions that they make and grant Carnegie the + * rights to redistribute these changes. + * + * from: NetBSD: htons.S,v 1.1 1996/04/17 22:36:54 cgd + * from: src/sys/libkern/alpha/htons.S,v 1.3 2002/02/18 20:35:21 + * + * $FreeBSD$ + */ + +#define NAME __bswap16 + +#include --- /dev/null Thu Feb 21 17:45:56 2002 +++ /home/tmm/p4/sparc64/sys/libkern/alpha/bswap32.S Tue Feb 19 20:09:52 2002 @@ -0,0 +1,35 @@ +/* + * Copyright (c) 1996 Carnegie-Mellon University. + * All rights reserved. + * + * Author: Chris G. Demetriou + * + * Permission to use, copy, modify and distribute this software and + * its documentation is hereby granted, provided that both the copyright + * notice and this permission notice appear in all copies of the + * software, derivative works or modified versions, and any portions + * thereof, and that both notices appear in supporting documentation. + * + * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" + * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND + * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. + * + * Carnegie Mellon requests users of this software to return to + * + * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU + * School of Computer Science + * Carnegie Mellon University + * Pittsburgh PA 15213-3890 + * + * any improvements or extensions that they make and grant Carnegie the + * rights to redistribute these changes. + * + * from: NetBSD: htonl.S,v 1.1 1996/04/17 22:36:52 cgd + * from: src/sys/libkern/alpha/htonl.S,v 1.3 2002/02/18 20:35:21 + * + * $FreeBSD$ + */ + +#define NAME __bswap32 + +#include --- /dev/null Thu Feb 21 17:45:56 2002 +++ /home/tmm/p4/sparc64/sys/libkern/ia64/bswap16.S Tue Feb 19 20:11:47 2002 @@ -0,0 +1,35 @@ +/* + * Copyright (c) 1996 Carnegie-Mellon University. + * All rights reserved. + * + * Author: Chris G. Demetriou + * + * Permission to use, copy, modify and distribute this software and + * its documentation is hereby granted, provided that both the copyright + * notice and this permission notice appear in all copies of the + * software, derivative works or modified versions, and any portions + * thereof, and that both notices appear in supporting documentation. + * + * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" + * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND + * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. + * + * Carnegie Mellon requests users of this software to return to + * + * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU + * School of Computer Science + * Carnegie Mellon University + * Pittsburgh PA 15213-3890 + * + * any improvements or extensions that they make and grant Carnegie the + * rights to redistribute these changes. + * + * from: NetBSD: htons.S,v 1.1 1996/04/17 22:36:54 cgd + * from: src/sys/libkern/ia64/htons.S,v 1.2 2002/02/18 20:35:21 + * + * $FreeBSD$ + */ + +#define NAME __bswap16 + +#include --- /dev/null Thu Feb 21 17:45:56 2002 +++ /home/tmm/p4/sparc64/sys/libkern/ia64/bswap32.S Tue Feb 19 20:13:03 2002 @@ -0,0 +1,35 @@ +/* + * Copyright (c) 1996 Carnegie-Mellon University. + * All rights reserved. + * + * Author: Chris G. Demetriou + * + * Permission to use, copy, modify and distribute this software and + * its documentation is hereby granted, provided that both the copyright + * notice and this permission notice appear in all copies of the + * software, derivative works or modified versions, and any portions + * thereof, and that both notices appear in supporting documentation. + * + * CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" + * CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND + * FOR ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. + * + * Carnegie Mellon requests users of this software to return to + * + * Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU + * School of Computer Science + * Carnegie Mellon University + * Pittsburgh PA 15213-3890 + * + * any improvements or extensions that they make and grant Carnegie the + * rights to redistribute these changes. + * + * from: NetBSD: htonl.S,v 1.1 1996/04/17 22:36:52 cgd + * from: src/sys/libkern/ia64/htonl.S,v 1.2 2002/02/18 20:35:21 + * + * $FreeBSD$ + */ + +#define NAME __bswap32 + +#include --- /home/tmm/p4/freebsd/sys/dev/usb/usb_port.h Fri Feb 8 01:54:08 2002 +++ /home/tmm/p4/sparc64/sys/dev/usb/usb_port.h Wed Feb 20 00:43:17 2002 @@ -305,7 +305,6 @@ /* XXX Change this when FreeBSD has memset */ #define memcpy(d, s, l) bcopy((s),(d),(l)) #define memset(d, v, l) bzero((d),(l)) -#define bswap32(x) swap32(x) #define kthread_create1(f, s, p, a0, a1) \ kthread_create((f), (s), (p), RFHIGHPID, (a0), (a1)) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 20:37:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by hub.freebsd.org (Postfix) with ESMTP id 1BFAD37B404 for ; Thu, 21 Feb 2002 20:37:31 -0800 (PST) Received: (qmail 2683 invoked from network); 22 Feb 2002 04:37:23 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.91.136.163]) (envelope-sender ) by mail6.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 22 Feb 2002 04:37:23 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200202212204.g1LM4vQ09988@apollo.backplane.com> Date: Thu, 21 Feb 2002 23:36:59 -0500 (EST) From: John Baldwin To: Matthew Dillon Subject: Re: RE: that INVARIANT/ucred freeing stuff. Cc: arch@FreeBSD.ORG, Julian Elischer Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 21-Feb-02 Matthew Dillon wrote: > We learned a lesson with DIAGNOSTIC. You are repeating the same mistakes > DIAGNOSTIC had which resulted in DIAGNOSTIC becoming essentially > unusable. > INVARIANTS != DIAGNOSTIC. If you want to resurrect DIAGNOSTIC you are > welcome to throw this ucred-clearing code under that option. > > INVARIANTS is not supposed to change algorithms or optimizations > wholesale. It is simply supposed to add additional sanity checks to > existing algorithms and optimizations. > > So when the decision was made to implement td_ucred as an optimization, > that means that is what we need to use. We should not go lobotomizing > it for INVARIANTS. > > It's that simple. It is nothing even close to the absurdness > of removing KASSERT for INVARIANTS that you are trying to contrast it > against. > > What I recommend you do, John, is to add KASSERT()s at a few critical > points in the code to assert that td_ucred is not being used out of > context. We ... meaning Julian and myself mainly, see absolutely no > point in lobotomizing an optimization that has such a huge effect on > performance just because INVARIANTS is turned on. That is not the > purpose of INVARIANTS. We want that code *GONE*. > > We would like you to acquiesce to this request. Fine, stick it under DIAGNOSTIC (which isn't dead.) The problem is that there aren't just 5 places in the kernel that you would need to stick this assert, you would need it all over the place. But I guess no one else has looked at all the places that p_ucred is used and thought about how to ensure we don't use a bogus td_ucred. > -Matt -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 21: 7:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from espresso.q9media.com (espresso.q9media.com [216.254.138.122]) by hub.freebsd.org (Postfix) with ESMTP id 7AC4837B400 for ; Thu, 21 Feb 2002 21:07:38 -0800 (PST) Received: (from mike@localhost) by espresso.q9media.com (8.11.6/8.11.6) id g1M53GC29409; Fri, 22 Feb 2002 00:03:16 -0500 (EST) (envelope-from mike) Date: Fri, 22 Feb 2002 00:03:16 -0500 From: Mike Barcroft To: Thomas Moestl Cc: freebsd-arch@FreeBSD.org Subject: Re: adding more endian conversion and bus space functions Message-ID: <20020222000316.B19019@espresso.q9media.com> References: <20020111005207.GA7246@crow.dom2ip.de> <20020220031842.GF282@crow.dom2ip.de> <20020222033455.GA289@crow.dom2ip.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020222033455.GA289@crow.dom2ip.de>; from tmoestl@gmx.net on Fri, Feb 22, 2002 at 04:34:55AM +0100 Organization: The FreeBSD Project Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thomas Moestl writes: > I've attached an updated version of the patch, which implements some > helpful suggestions by Mike Barcroft to reduce name space pollution > when the new functions will be exposed to userland, as well as some > cleanups. This is tested on i386, alpha (buildworld, I hope to get the > results of a GENERIC kernel build tomorrow) and sparc64. I'd like to > commit it soon if I can get somebody to test it on ia64, hopefully > during the weekend. GENERIC on alpha was successful too. Best regards, Mike Barcroft To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 21:27:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by hub.freebsd.org (Postfix) with ESMTP id D8A9A37B400 for ; Thu, 21 Feb 2002 21:27:44 -0800 (PST) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id g1M5ReP52999; Fri, 22 Feb 2002 00:27:40 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Fri, 22 Feb 2002 00:27:40 -0500 (EST) From: Jeff Roberson To: Matthew Dillon Cc: Julian Elischer , Subject: Re: Prioritized bio patches. (Updated patch) In-Reply-To: <200202220106.g1M168p12213@apollo.backplane.com> Message-ID: <20020222001512.A38875-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I agree, it is definately a problem. phk also pointed out that B_ORDERED breakage. This is really only used by ffs w/o softupdates now though. I was thinking of a trivial time slicing mechanism where you could guarantee a certain number of IOs per 100 to each queue. phk also pointed me to some interesting research on the topic. The document describes anticipatory scheduling, and is very applicable to the problem at hand. It looks like it's freely redistributable so I'll post the link here. http://phk.freebsd.dk/misc/antsched.pdf On a side note, even when our system is running at peak capacity (31MB/s per disk), lower priority IO gets a chance to go through. This is partially due to our long sim queue in cam. The interrupt latency in current is so bad that large ammounts of work get cleared out at once. Since this happens before we have a chance to issue new requests, the low priority ones make it through. Anyway, thanks for the feedback. I'll consider these comments and see if I can come up with a more optimal solution (or just steal the one presented in the paper, if the work is properly licensed). Jeff On Thu, 21 Feb 2002, Matthew Dillon wrote: > I was thinking potentially of some sort of priority stealing > mechanism similar to the mechanism that exists for mutexes. > It would not be easy to implement though. We would have to > deal with vnode locks as well as buffer locks. > > Priority inversion can be a serious problem, especially in more > heavily loaded systems. I don't think it would be safe to commit > a bio priority mechanism without dealing with the problem. > > Another possibility is to guarentee that the I/O request will > eventually go out by slowly bumping up its priority in the queue, > so it doesn't get stuck indefinintely. > > -Matt > Matthew Dillon > > > > :Yes, there is a possibility that a high priority process will completely > :lock out the low priority process. I haven't followed recent scheduler > :changes, but a niced process may never run at all if there is real work > :no? If this is true, than the two are very similar. Anyhow, there is a > :definite priority inversion issue here, but the system is somewhat > :autobalancing. One would assume that the higher priority task is blocked > :waiting for the low priority task to finish. So it is not issuing any > :other io, which allows the low priority task to finish up. This is not > :entirely optimal, I agree. I can not check for priority inversion with > :lockmgr locks though, so there really is no obvious way out. > : > :For my application, the lockout was very desirable. With workloads such > :as compiling, you'll always have some free slots to send low priority io. > :This is due to the sequential nature of the workload though. It reads, > :does something with the data, and then writes. In between other things > :can happen. I could see how a very busy database or web server may never > :give nice processes a chance to finish(until late at night, perhaps). > : > :You could force all file system meta data operations to run at the normal > :priority. This would be good for directory operations in particular. You > :can still have priority inversion issues with individual file > :read/writes, but it would alleviate some of the problems. > : > : > :Jeff > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 21 21:39:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 3965637B400; Thu, 21 Feb 2002 21:39:39 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1M5ddM13773; Thu, 21 Feb 2002 21:39:39 -0800 (PST) (envelope-from dillon) Date: Thu, 21 Feb 2002 21:39:39 -0800 (PST) From: Matthew Dillon Message-Id: <200202220539.g1M5ddM13773@apollo.backplane.com> To: John Baldwin Cc: arch@FreeBSD.org, Julian Elischer Subject: Re: RE: that INVARIANT/ucred freeing stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Fine, stick it under DIAGNOSTIC (which isn't dead.) The problem is that there :aren't just 5 places in the kernel that you would need to stick this assert, :you would need it all over the place. But I guess no one else has looked at :all the places that p_ucred is used and thought about how to ensure we don't :use a bogus td_ucred. : : :John Baldwin <>< http://www.FreeBSD.org/~jhb/ Don't try to overengineer the problem. Unless you believe there is a serious problem, there is no need to put a check in every single conceivable place an error might occur. Just putting a few safety checks in a few critical places should be sufficient. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 0: 2:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 2C0EC37B402; Fri, 22 Feb 2002 00:02:15 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g1M828q82474; Fri, 22 Feb 2002 10:02:08 +0200 (EET) (envelope-from ru) Date: Fri, 22 Feb 2002 10:02:08 +0200 From: Ruslan Ermilov To: arch@FreeBSD.org Cc: Bruce Evans , Pekka Savola Subject: MAKEOBJDIRPREFIX Message-ID: <20020222080208.GB81821@sunbay.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="IJpNTDwzlM2Ie8A6" Content-Disposition: inline User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --IJpNTDwzlM2Ie8A6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi! If I am reading the POSIX specs correctly, MAKEOBJDIRPREFIX could be easily made to be honoured even if specified on a command line. : Before the makefile(s) are read, all of the make utility command : line macro definitions (except the MAKEFLAGS macro or the SHELL : macro) shall be added to the environment of make. Other : implementation-defined variables may also be added to the environment : of make. The attached patch merely moves the `objdir' initialization below the MainParseArgs() call, after all command line arguments have already been parsed. To be honest, the current behavior does not contradict to POSIX (which does not say anything about MAKEOBJDIR[PREFIX]), but the proposed behavior would help users errouneously attempting to set MAKEOBJDIRPREFIX on a command line. (It still does not work if MAKEOBJDIRPREFIX is set as a make's global.) Comments? Cheers, -- Ruslan Ermilov Sysadmin and DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age --IJpNTDwzlM2Ie8A6 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=p Index: main.c =================================================================== RCS file: /home/ncvs/src/usr.bin/make/main.c,v retrieving revision 1.49 diff -u -p -r1.49 main.c --- main.c 25 Apr 2001 14:44:41 -0000 1.49 +++ main.c 22 Feb 2002 07:34:04 -0000 @@ -563,48 +563,6 @@ main(argc, argv) else machine_cpu = "unknown"; } - - /* - * The object directory location is determined using the - * following order of preference: - * - * 1. MAKEOBJDIRPREFIX`cwd` - * 2. MAKEOBJDIR - * 3. _PATH_OBJDIR.${MACHINE} - * 4. _PATH_OBJDIR - * 5. _PATH_OBJDIRPREFIX`cwd` - * - * If one of the first two fails, use the current directory. - * If the remaining three all fail, use the current directory. - * - * Once things are initted, - * have to add the original directory to the search path, - * and modify the paths for the Makefiles apropriately. The - * current directory is also placed as a variable for make scripts. - */ - if (!(pathp = getenv("MAKEOBJDIRPREFIX"))) { - if (!(path = getenv("MAKEOBJDIR"))) { - path = _PATH_OBJDIR; - pathp = _PATH_OBJDIRPREFIX; - (void) snprintf(mdpath, MAXPATHLEN, "%s.%s", - path, machine); - if (!(objdir = chdir_verify_path(mdpath, obpath))) - if (!(objdir=chdir_verify_path(path, obpath))) { - (void) snprintf(mdpath, MAXPATHLEN, - "%s%s", pathp, curdir); - if (!(objdir=chdir_verify_path(mdpath, - obpath))) - objdir = curdir; - } - } - else if (!(objdir = chdir_verify_path(path, obpath))) - objdir = curdir; - } - else { - (void) snprintf(mdpath, MAXPATHLEN, "%s%s", pathp, curdir); - if (!(objdir = chdir_verify_path(mdpath, obpath))) - objdir = curdir; - } create = Lst_Init(FALSE); makefiles = Lst_Init(FALSE); @@ -646,10 +604,6 @@ main(argc, argv) Var_Init(); /* As well as the lists of variables for * parsing arguments */ str_init(); - if (objdir != curdir) - Dir_AddDir(dirSearchPath, curdir); - Var_Set(".CURDIR", curdir, VAR_GLOBAL); - Var_Set(".OBJDIR", objdir, VAR_GLOBAL); /* * Initialize various variables. @@ -676,6 +630,53 @@ main(argc, argv) #endif MainParseArgs(argc, argv); + + /* + * The object directory location is determined using the + * following order of preference: + * + * 1. MAKEOBJDIRPREFIX`cwd` + * 2. MAKEOBJDIR + * 3. _PATH_OBJDIR.${MACHINE} + * 4. _PATH_OBJDIR + * 5. _PATH_OBJDIRPREFIX`cwd` + * + * If one of the first two fails, use the current directory. + * If the remaining three all fail, use the current directory. + * + * Once things are initted, + * have to add the original directory to the search path, + * and modify the paths for the Makefiles apropriately. The + * current directory is also placed as a variable for make scripts. + */ + if (!(pathp = getenv("MAKEOBJDIRPREFIX"))) { + if (!(path = getenv("MAKEOBJDIR"))) { + path = _PATH_OBJDIR; + pathp = _PATH_OBJDIRPREFIX; + (void) snprintf(mdpath, MAXPATHLEN, "%s.%s", + path, machine); + if (!(objdir = chdir_verify_path(mdpath, obpath))) + if (!(objdir=chdir_verify_path(path, obpath))) { + (void) snprintf(mdpath, MAXPATHLEN, + "%s%s", pathp, curdir); + if (!(objdir=chdir_verify_path(mdpath, + obpath))) + objdir = curdir; + } + } + else if (!(objdir = chdir_verify_path(path, obpath))) + objdir = curdir; + } + else { + (void) snprintf(mdpath, MAXPATHLEN, "%s%s", pathp, curdir); + if (!(objdir = chdir_verify_path(mdpath, obpath))) + objdir = curdir; + } + + if (objdir != curdir) + Dir_AddDir(dirSearchPath, curdir); + Var_Set(".CURDIR", curdir, VAR_GLOBAL); + Var_Set(".OBJDIR", objdir, VAR_GLOBAL); /* * Be compatible if user did not specify -j and did not explicitly --IJpNTDwzlM2Ie8A6-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 1:15:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id AD7DB37B400 for ; Fri, 22 Feb 2002 01:15:35 -0800 (PST) Received: (qmail 13189 invoked from network); 22 Feb 2002 09:15:34 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.91.136.163]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 22 Feb 2002 09:15:34 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200202220539.g1M5ddM13773@apollo.backplane.com> Date: Fri, 22 Feb 2002 04:15:35 -0500 (EST) From: John Baldwin To: Matthew Dillon Subject: Re: RE: that INVARIANT/ucred freeing stuff. Cc: Julian Elischer , arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 22-Feb-02 Matthew Dillon wrote: >:Fine, stick it under DIAGNOSTIC (which isn't dead.) The problem is that >:there >:aren't just 5 places in the kernel that you would need to stick this assert, >:you would need it all over the place. But I guess no one else has looked at >:all the places that p_ucred is used and thought about how to ensure we don't >:use a bogus td_ucred. >: >: >:John Baldwin <>< http://www.FreeBSD.org/~jhb/ > > Don't try to overengineer the problem. Unless you believe there is > a serious problem, there is no need to put a check in every single > conceivable place an error might occur. Just putting a few safety checks > in a few critical places should be sufficient. I don't know where all the places we might look at a ucred wrongly are. That's why I wanted the much simpler solution of just clearing td_ucred to NULL so we had an implicit KASSERT for us in all those places. > -Matt > Matthew Dillon > -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 1:43:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from straylight.ringlet.net (discworld.nanolink.com [217.75.135.248]) by hub.freebsd.org (Postfix) with SMTP id 062E937B404 for ; Fri, 22 Feb 2002 01:43:31 -0800 (PST) Received: (qmail 1791 invoked by uid 1000); 22 Feb 2002 09:43:56 -0000 Date: Fri, 22 Feb 2002 11:43:56 +0200 From: Peter Pentchev To: Ruslan Ermilov Cc: arch@FreeBSD.org, Bruce Evans , Pekka Savola Subject: Re: MAKEOBJDIRPREFIX Message-ID: <20020222114356.B335@straylight.oblivion.bg> Mail-Followup-To: Ruslan Ermilov , arch@FreeBSD.org, Bruce Evans , Pekka Savola References: <20020222080208.GB81821@sunbay.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="ADZbWkCsHQ7r3kzd" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020222080208.GB81821@sunbay.com>; from ru@FreeBSD.org on Fri, Feb 22, 2002 at 10:02:08AM +0200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --ADZbWkCsHQ7r3kzd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Feb 22, 2002 at 10:02:08AM +0200, Ruslan Ermilov wrote: > Hi! >=20 > If I am reading the POSIX specs correctly, MAKEOBJDIRPREFIX could > be easily made to be honoured even if specified on a command line. >=20 > : Before the makefile(s) are read, all of the make utility command > : line macro definitions (except the MAKEFLAGS macro or the SHELL > : macro) shall be added to the environment of make. Other > : implementation-defined variables may also be added to the environment > : of make. >=20 > The attached patch merely moves the `objdir' initialization below > the MainParseArgs() call, after all command line arguments have > already been parsed. Looks good to me (and God knows how many times I've stumbled into this!) Wish I'd known it was that simple.. :) but thanks for actually hunting it down and fixing it! G'luck, Peter --=20 Peter Pentchev roam@ringlet.net roam@FreeBSD.org PGP key: http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint FDBA FD79 C26F 3C51 C95E DF9E ED18 B68D 1619 4553 This sentence claims to be an Epimenides paradox, but it is lying. --ADZbWkCsHQ7r3kzd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjx2EtwACgkQ7Ri2jRYZRVN13wCeKHQPkZvngo3R0y5SIY6khU4v 6R4AnAlfuxYuOYQuPXBX+G4/dZs1pyMe =OSBx -----END PGP SIGNATURE----- --ADZbWkCsHQ7r3kzd-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 4:22:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from chez.McKusick.COM (chez.mckusick.com [209.31.233.177]) by hub.freebsd.org (Postfix) with ESMTP id A172337B400 for ; Fri, 22 Feb 2002 04:22:44 -0800 (PST) Received: from chez.McKusick.COM (localhost [127.0.0.1]) by chez.McKusick.COM (8.11.6/8.11.6) with ESMTP id g1MCMiW78211 for ; Fri, 22 Feb 2002 12:22:44 GMT (envelope-from mckusick@chez.McKusick.COM) Message-Id: <200202221222.g1MCMiW78211@chez.McKusick.COM> To: arch@freebsd.org Subject: NetBSD regression testing for system calls Date: Fri, 22 Feb 2002 04:22:44 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Is this something that FreeBSD might want to consider using and/or contributing to? Kirk McKusick ------- Forwarded Message Date: Fri, 22 Feb 2002 22:45:34 +1100 From: Darren Reed To: developers@netbsd.org Subject: regression testing for system calls (thought I'd mention this given recent regression testing talks...) When NetBSD was 1.4, I started on a project to build a regression test suite for system calls. For one reason or another I got side tracked (probably in getting bugs fixed) and never got back to it. Is there any interest in including this work in the base distribution (of source code at least) ? The work can be found at: cvs.netbsd.org:~darrenr/systest.tar There's no documentation, just a Makefile. "make" and run "./stest". It must be run as root and it does play with things in /tmp with consideration for safety - ie. if you worry about symlink race conditions then do not run it on a multi-user box. The general thrust of it has been to ensure that when system calls are presented with bad parameters, they fail as expected, as well as success from good input. So far I've done 35 system calls out of 168 identified at the time. Darren p.s. I have no trouble assigning copyright for this to TNF. p.s.s. oink ------- End of Forwarded Message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 6:22:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 7164237B400 for ; Fri, 22 Feb 2002 06:22:26 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id BAA12597; Sat, 23 Feb 2002 01:22:20 +1100 Date: Sat, 23 Feb 2002 01:22:30 +1100 (EST) From: Bruce Evans X-X-Sender: To: Thomas Moestl Cc: Subject: Re: adding more endian conversion and bus space functions In-Reply-To: <20020222033455.GA289@crow.dom2ip.de> Message-ID: <20020223004643.V25362-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 22 Feb 2002, Thomas Moestl wrote: > I've attached an updated version of the patch, which implements some > helpful suggestions by Mike Barcroft to reduce name space pollution > when the new functions will be exposed to userland, as well as some > cleanups... > ==== //depot/projects/sparc64/sys/alpha/include/bus.h#1 - /home/tmm/p4/sparc64/sys/alpha/include/bus.h ==== > --- /tmp/tmp.8072.1 Thu Feb 21 19:06:08 2002 > +++ /home/tmm/p4/sparc64/sys/alpha/include/bus.h Wed Feb 20 01:37:00 2002 > @@ -366,6 +366,70 @@ > (t)->ab_ops->abo_barrier(t, (h)+(o), l, f) > > /* > + * Stream accesses are the same as normal accesses on alpha; there are no > + * supported bus systems with an endianess different from the host one. > + */ > +#define bus_space_read_stream_1(t, h, o) bus_space_read_1((t), (h), (o)) > ... I think these definitions should be in a common header. > ==== //depot/projects/sparc64/sys/i386/include/endian.h#6 - /home/tmm/p4/sparc64/sys/i386/include/endian.h ==== > --- /tmp/tmp.8072.10 Thu Feb 21 19:06:10 2002 > +++ /home/tmm/p4/sparc64/sys/i386/include/endian.h Thu Feb 21 01:53:24 2002 > @@ -58,12 +58,13 @@ > #define BYTE_ORDER LITTLE_ENDIAN > #endif /* ! _POSIX_SOURCE */ > > +#ifdef _KERNEL > #ifdef __GNUC__ > > -__BEGIN_DECLS > - > +#ifndef _BSWAP32_DEFINED > +#define _BSWAP32_DEFINED > static __inline __uint32_t > -__htonl(__uint32_t __x) > +__bswap32(__uint32_t __x) > { > #if defined(_KERNEL) && (defined(I486_CPU) || defined(I586_CPU) || defined(I686_CPU)) && !defined(I386_CPU) > __asm ("bswap %0" : "+r" (__x)); Can _BSWAP32_DEFINED be already defined here? I think only this file should define it (for i386's), and the multiple-inclusion protection prevents it being redefined. I think it was a mistake to change these from macros to inline functions. With macros, you can just define the macros and not need an extra definition to say that they have been defined. Alternatively, you can always define a macro or inline version in the MD headers and not need any ifdefs (if there is a macro or inline MD version, use that; else if there is an extern MD version, use that, else define the MD version to be the generic version). > ==== //depot/projects/sparc64/sys/sys/imgact_aout.h#3 - /home/tmm/p4/sparc64/sys/sys/imgact_aout.h ==== > --- /tmp/tmp.8072.20 Thu Feb 21 19:06:12 2002 > +++ /home/tmm/p4/sparc64/sys/sys/imgact_aout.h Wed Feb 20 00:51:10 2002 > @@ -50,13 +50,13 @@ > ((mag) & 0xffff) ) > > #define N_GETMAGIC_NET(ex) \ > - (__ntohl((ex).a_midmag) & 0xffff) > + (ntohl((ex).a_midmag) & 0xffff) > #define N_GETMID_NET(ex) \ > - ((__ntohl((ex).a_midmag) >> 16) & 0x03ff) > + ((ntohl((ex).a_midmag) >> 16) & 0x03ff) > #define N_GETFLAG_NET(ex) \ > - ((__ntohl((ex).a_midmag) >> 26) & 0x3f) > + ((ntohl((ex).a_midmag) >> 26) & 0x3f) > #define N_SETMAGIC_NET(ex,mag,mid,flag) \ > - ( (ex).a_midmag = __htonl( (((flag)&0x3f)<<26) | (((mid)&0x03ff)<<16) \ > + ( (ex).a_midmag = htonl( (((flag)&0x3f)<<26) | (((mid)&0x03ff)<<16) \ > | (((mag)&0xffff)) ) ) > > #define N_ALIGN(ex,x) \ This seems to be a regression. I think the change should be from __hton* to __bswap*. > +++ /home/tmm/p4/sparc64/sys/libkern/alpha/bswap16.S Tue Feb 19 20:08:13 2002 > @@ -0,0 +1,35 @@ > +/* > + * Copyright (c) 1996 Carnegie-Mellon University. > + * All rights reserved. > + * > + * Author: Chris G. Demetriou > ... > + * $FreeBSD$ > + */ > + > +#define NAME __bswap16 > + > +#include Is this and similar files from NetBSD? It is too trivial to deserve having a long copyright. In libc/i386/string/memcpy.S, similar functionality takes 2 lines: #define MEMCPY #include "bcopy.S" Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 8: 0:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 8506B37B400 for ; Fri, 22 Feb 2002 08:00:09 -0800 (PST) Received: from pool0378.cvx22-bradley.dialup.earthlink.net ([209.179.199.123] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16eI78-0005Ej-00; Fri, 22 Feb 2002 08:00:02 -0800 Message-ID: <3C766AF8.35B9E6B@mindspring.com> Date: Fri, 22 Feb 2002 07:59:52 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: Jeff Roberson , Julian Elischer , arch@FreeBSD.ORG Subject: Re: Prioritized bio patches. (Updated patch) References: <20020219194142.M12686-100000@mail.chesapeake.net> <200202220106.g1M168p12213@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > I was thinking potentially of some sort of priority stealing > mechanism similar to the mechanism that exists for mutexes. > It would not be easy to implement though. We would have to > deal with vnode locks as well as buffer locks. I don't think this can work. There are serious inheritance tracking problems, when it comes to dealing with giving up an inherited priority on a directory entry block, a cylinder group block, or a frag, shared by two processes of different priority. This might be easier without soft updates, or with UFS2, when it happens. > Priority inversion can be a serious problem, especially in more > heavily loaded systems. I don't think it would be safe to commit > a bio priority mechanism without dealing with the problem. I think depending the priority tree from the slowest hardware interface is probably a bad idea, anyway. Really, people who are concerned about this problem will probably need to simply predo their conflicting operations on the directory hierarchy, and then make sure to allocate files in fs_blksiz sized chunks. If you can think of a clean way to inherit the proper priority, and switch back (the only thing I can think of is a lent priority stack per BIO, and that can't be a good idea), then I'd be happy to be proven wrong, since I'd get to learn something from it. 8-). > Another possibility is to guarentee that the I/O request will > eventually go out by slowly bumping up its priority in the queue, > so it doesn't get stuck indefinintely. Soft updates kinda/sorta tries to do this already by selection of the clock slot for insertion. I think that this can't work to "push back" operations of a lower priority, though (maybe it could -- by changing the head of the clock, and then inserting elements, but you'd be limited to 1/2 the clock size worth of inversion handlings before you were screwed). I think if something is *truly* high priority, then it is probably right to starve other processes; an application with near RT-requirements (e.g. video capture) may in fact *need* to hog 90% of system resources, and any "fairness" you inject into the mix to speed up other apps will cause it to break. Ah. I have an idea. Why don't you have a circular list of queued work, and then pick which slot relative to the head based on a deadline? If you can complete all the other I/O before hitting the deadline, fine, but if not, then you would complete it in deadline order. The system is overloaded when you can't meet the deadlines (e.g. there is more work on a queue than can be services in a clock tick), but at that point, you are overloaded, and the best you can do is a "best effort" approach anyway. Yes, this still leaves the possibility of starvation of lower priority processes. Actually, thinking of it this way, this out to be bound up in th RT prio stuff -- processes not running at RT prio should not enter into deadline contention at all: they should use the historical approach. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 9:25:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by hub.freebsd.org (Postfix) with SMTP id 4389F37B400 for ; Fri, 22 Feb 2002 09:25:25 -0800 (PST) Received: (qmail 8454 invoked by uid 0); 22 Feb 2002 17:25:23 -0000 Received: from pd953899c.dip.t-dialin.net (HELO forge.local) (217.83.137.156) by mail.gmx.net (mp014-rz3) with SMTP; 22 Feb 2002 17:25:23 -0000 Received: from tmm by forge.local with local (Exim 3.34 #1) id 16eJRj-0000pg-00; Fri, 22 Feb 2002 18:25:23 +0100 Date: Fri, 22 Feb 2002 18:25:22 +0100 From: Thomas Moestl To: Bruce Evans Cc: freebsd-arch@FreeBSD.ORG Subject: Re: adding more endian conversion and bus space functions Message-ID: <20020222172522.GA302@crow.dom2ip.de> Mail-Followup-To: Bruce Evans , freebsd-arch@FreeBSD.ORG References: <20020222033455.GA289@crow.dom2ip.de> <20020223004643.V25362-100000@gamplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020223004643.V25362-100000@gamplex.bde.org> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 2002/02/23 at 01:22:30 +1100, Bruce Evans wrote: > On Fri, 22 Feb 2002, Thomas Moestl wrote: > > > I've attached an updated version of the patch, which implements some > > helpful suggestions by Mike Barcroft to reduce name space pollution > > when the new functions will be exposed to userland, as well as some > > cleanups... > > > ==== //depot/projects/sparc64/sys/alpha/include/bus.h#1 - /home/tmm/p4/sparc64/sys/alpha/include/bus.h ==== > > --- /tmp/tmp.8072.1 Thu Feb 21 19:06:08 2002 > > +++ /home/tmm/p4/sparc64/sys/alpha/include/bus.h Wed Feb 20 01:37:00 2002 > > @@ -366,6 +366,70 @@ > > (t)->ab_ops->abo_barrier(t, (h)+(o), l, f) > > > > /* > > + * Stream accesses are the same as normal accesses on alpha; there are no > > + * supported bus systems with an endianess different from the host one. > > + */ > > +#define bus_space_read_stream_1(t, h, o) bus_space_read_1((t), (h), (o)) > > ... > > I think these definitions should be in a common header. They are only present on some architectures; powerpc and sparc64 have bus systems with a different endianesses than the CPU, so these defines are different there. I think that an extra header only to be included by those architectures for which the stream and non-stream variants are identical may be a bit of overkill. > > ==== //depot/projects/sparc64/sys/i386/include/endian.h#6 - /home/tmm/p4/sparc64/sys/i386/include/endian.h ==== > > --- /tmp/tmp.8072.10 Thu Feb 21 19:06:10 2002 > > +++ /home/tmm/p4/sparc64/sys/i386/include/endian.h Thu Feb 21 01:53:24 2002 > > @@ -58,12 +58,13 @@ > > #define BYTE_ORDER LITTLE_ENDIAN > > #endif /* ! _POSIX_SOURCE */ > > > > +#ifdef _KERNEL > > #ifdef __GNUC__ > > > > -__BEGIN_DECLS > > - > > +#ifndef _BSWAP32_DEFINED > > +#define _BSWAP32_DEFINED > > static __inline __uint32_t > > -__htonl(__uint32_t __x) > > +__bswap32(__uint32_t __x) > > { > > #if defined(_KERNEL) && (defined(I486_CPU) || defined(I586_CPU) || defined(I686_CPU)) && !defined(I386_CPU) > > __asm ("bswap %0" : "+r" (__x)); > > Can _BSWAP32_DEFINED be already defined here? I think only this file > should define it (for i386's), and the multiple-inclusion protection > prevents it being redefined. Yes, the #ifndef protection is not really needed. I will remove it. > > ==== //depot/projects/sparc64/sys/sys/imgact_aout.h#3 - /home/tmm/p4/sparc64/sys/sys/imgact_aout.h ==== > > --- /tmp/tmp.8072.20 Thu Feb 21 19:06:12 2002 > > +++ /home/tmm/p4/sparc64/sys/sys/imgact_aout.h Wed Feb 20 00:51:10 2002 > > @@ -50,13 +50,13 @@ > > ((mag) & 0xffff) ) > > > > #define N_GETMAGIC_NET(ex) \ > > - (__ntohl((ex).a_midmag) & 0xffff) > > + (ntohl((ex).a_midmag) & 0xffff) > > #define N_GETMID_NET(ex) \ > > - ((__ntohl((ex).a_midmag) >> 16) & 0x03ff) > > + ((ntohl((ex).a_midmag) >> 16) & 0x03ff) > > #define N_GETFLAG_NET(ex) \ > > - ((__ntohl((ex).a_midmag) >> 26) & 0x3f) > > + ((ntohl((ex).a_midmag) >> 26) & 0x3f) > > #define N_SETMAGIC_NET(ex,mag,mid,flag) \ > > - ( (ex).a_midmag = __htonl( (((flag)&0x3f)<<26) | (((mid)&0x03ff)<<16) \ > > + ( (ex).a_midmag = htonl( (((flag)&0x3f)<<26) | (((mid)&0x03ff)<<16) \ > > | (((mag)&0xffff)) ) ) > > > > #define N_ALIGN(ex,x) \ > > This seems to be a regression. I think the change should be from __hton* > to __bswap*. Hmmm, judging from the comments in imgact_aout.c, that code was added to deal with NetBSD executables, which apparently use network byte order for the header fields; however, other executables would presumably be in host byte order on big-endian architectures, too, so executables with little-endian header fields on big-endian architectures should be non-existent. In any case, there would be other modifications required to handle this case in at least imgact_aout.c, so I would rather not do this right now, at least not in that commit. > > +++ /home/tmm/p4/sparc64/sys/libkern/alpha/bswap16.S Tue Feb 19 20:08:13 2002 > > @@ -0,0 +1,35 @@ > > +/* > > + * Copyright (c) 1996 Carnegie-Mellon University. > > + * All rights reserved. > > + * > > + * Author: Chris G. Demetriou > > ... > > + * $FreeBSD$ > > + */ > > + > > +#define NAME __bswap16 > > + > > +#include > > Is this and similar files from NetBSD? It is too trivial to deserve > having a long copyright. In libc/i386/string/memcpy.S, similar > functionality takes 2 lines: > > #define MEMCPY > #include "bcopy.S" They were copied directly from ntoh*.S from libkern, which had this copyright, so I felt uncomfortable to remove it. - thomas To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 9:40:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id B3ACB37B400 for ; Fri, 22 Feb 2002 09:40:09 -0800 (PST) Received: from localhost (eischen@localhost) by mail.pcnet.com (8.12.1/8.12.1) with ESMTP id g1MHe8DZ025071; Fri, 22 Feb 2002 12:40:08 -0500 (EST) Date: Fri, 22 Feb 2002 12:40:08 -0500 (EST) From: Daniel Eischen To: Bruce Evans Cc: Dan Eischen , arch@FreeBSD.ORG Subject: Re: getsetcontext system call In-Reply-To: <20020211210412.X308-100000@gamplex.bde.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Back to this... On Mon, 11 Feb 2002, Bruce Evans wrote: > On Mon, 11 Feb 2002, Daniel Eischen wrote: > > http://people.freebsd.org/~deischen/ucontext/uc-sys.diffs > > http://people.freebsd.org/~deischen/ucontext/uc-libc.diffs > > I noticed a few minor problems with it, but haven't completely reviewed it. > > >From your original mail: > > ! I also changed the alpha a bit. When delivering signals, it > ! dropped the FPU if it was currently owned. This wasn't done > ! on i386, and I didn't see a reason why it would need to be done > ! for alpha. For i386, signal deliver now includes the FPU > ! context. > > Well, signals handlers should get a clean FPU state, and you don't > want spend much time looking at the current state to see if it is > clean. Saving the state into the ucontext and initializing a clean > state without looking is probably best. For i386's without fxsr, > saving the state loads a new, clean state into the FPU whether you > want it to or not, so it would be best to pass that state to the signal > handler and not put it in the pcb, even though signal handlers probably > won't use it (saving it to the pcb would usually be a waste of time, > since the state will usually be restored from the ucontext and not > from the pcb). For i386's with fxsr, saving the state doesn't change > the state in the FPU, so initializing the clean state in the pcb only > is probably better (forget the state in the FPU). For i386's with or > without fxsr, if the signal handler returns it is difficult to tell > if the state being "restored" is already in the FPU, since the signal > handler may have modified the ucontext. Copying the ucontext to the > pcb and forgetting the state in the FPU seems best in both cases. > > get_fpcontext() and set_fpcontext() don't have the right semantics for > signal handling. They try too hard to keep the state in the FPU if it > is already there, but in sendsig() you never want it (the old state) > there. OK, for signal handling the FPU is dropped and a clean FPU state will be loaded if the FPU is used again. > I just noticed a not so minor problem: get_fpcontext() on i386's only > works for the fxsr case, since it assumes that the state is is still > in the FPU after fpusave(). fpusave() and npxsave() probably shouldn't > exist, since they mainly obfuscate the critical differences between > fnsave() and fxsave(). I fixed this by reloading the state in the fxsr case. Someone's going to have to take a look at the alpha parts. How is a clean state handled there? And if you save the FPU state, do you have to reload the state or is it similar to the (x86) fxsr case where the state doesn't get destroyed and init'd after a save. Diffs are updated at: http://people.freebsd.org/~deischen/ucontext/uc-sys.diffs -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 10:36: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 2A9C937B404; Fri, 22 Feb 2002 10:35:54 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1MIZsZ18088; Fri, 22 Feb 2002 10:35:54 -0800 (PST) (envelope-from dillon) Date: Fri, 22 Feb 2002 10:35:54 -0800 (PST) From: Matthew Dillon Message-Id: <200202221835.g1MIZsZ18088@apollo.backplane.com> To: John Baldwin Cc: Julian Elischer , arch@FreeBSD.ORG Subject: Re: RE: that INVARIANT/ucred freeing stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :>:John Baldwin <>< http://www.FreeBSD.org/~jhb/ :> :> Don't try to overengineer the problem. Unless you believe there is :> a serious problem, there is no need to put a check in every single :> conceivable place an error might occur. Just putting a few safety checks :> in a few critical places should be sufficient. : :I don't know where all the places we might look at a ucred wrongly are. That's :why I wanted the much simpler solution of just clearing td_ucred to NULL so we :had an implicit KASSERT for us in all those places. : :-- : :John Baldwin <>< http://www.FreeBSD.org/~jhb/ This doesn't make any sense whatsoever. *NOTHING* was using td_ucred until just a few days ago, so unless *you* are blindly changing all p->p_ucred's into td->td_ucred's, I don't see how it can become an issue. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 11:20:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hub.freebsd.org (Postfix) with ESMTP id F3C1537B400; Fri, 22 Feb 2002 11:20:13 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020222192013.BWEU1147.rwcrmhc52.attbi.com@InterJet.elischer.org>; Fri, 22 Feb 2002 19:20:13 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA74339; Fri, 22 Feb 2002 11:19:05 -0800 (PST) Date: Fri, 22 Feb 2002 11:19:03 -0800 (PST) From: Julian Elischer To: John Baldwin Cc: Matthew Dillon , arch@FreeBSD.org Subject: Re: RE: that INVARIANT/ucred freeing stuff. In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG OK here is my suggestion: We add extra code under DIAGNOSTIC the code does: in proc.h add a field to thread of: #ifdef DIAGNOSTIC td_ucred_cache #endif /* DIAGNOSTIC */ on texiting the kernel: #ifdef DIAGNOSTIC if (td->td_ucred_cache) panic("thread already has cached ucred"); td->td_ucred_cache = td->td_ucred; td->td_ucred = NULL; #endif /* DIAGNOSTIC */ on entering the kernel we do: #ifdef DIAGNOSTIC if (td->td_ucred) panic("thread got a cred form somewhere in userspace"); td->td_cred = td->td_ucred_cache; td->td_ucred_cache = NULL; #endif /* DIAGNOSTIC */ if (td->ucred != p->p_ucred) cred_update_thread(td); we get good performance even when it it is optionned in and still have a NULL ucred pointer when in user space when DIAGNOSTIC is turned on. With no DIAGNOSTICS we get the best performance, and don't even bother to shift the reference. On Fri, 22 Feb 2002, John Baldwin wrote: > > On 22-Feb-02 Matthew Dillon wrote: > >:Fine, stick it under DIAGNOSTIC (which isn't dead.) The problem is that > >:there > >:aren't just 5 places in the kernel that you would need to stick this assert, > >:you would need it all over the place. But I guess no one else has looked at > >:all the places that p_ucred is used and thought about how to ensure we don't > >:use a bogus td_ucred. > >: > >: > >:John Baldwin <>< http://www.FreeBSD.org/~jhb/ > > > > Don't try to overengineer the problem. Unless you believe there is > > a serious problem, there is no need to put a check in every single > > conceivable place an error might occur. Just putting a few safety checks > > in a few critical places should be sufficient. > > I don't know where all the places we might look at a ucred wrongly are. That's > why I wanted the much simpler solution of just clearing td_ucred to NULL so we > had an implicit KASSERT for us in all those places. > > > -Matt > > Matthew Dillon > > > > -- > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 11:29:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 9C2C837B417; Fri, 22 Feb 2002 11:29:47 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1MJTFI20465; Fri, 22 Feb 2002 11:29:15 -0800 (PST) (envelope-from dillon) Date: Fri, 22 Feb 2002 11:29:15 -0800 (PST) From: Matthew Dillon Message-Id: <200202221929.g1MJTFI20465@apollo.backplane.com> To: Julian Elischer Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: RE: that INVARIANT/ucred freeing stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Sigh. Well, if you really want. I don't like the idea of the size of the thread structure changing just due to someone turning on or off DIAGNOSTIC though. -Matt :OK here is my suggestion: : :We add extra code under DIAGNOSTIC :the code does: : :in proc.h : :add a field to thread of: :#ifdef DIAGNOSTIC : td_ucred_cache :#endif /* DIAGNOSTIC */ : : :on texiting the kernel: : :#ifdef DIAGNOSTIC : if (td->td_ucred_cache) : panic("thread already has cached ucred"); : td->td_ucred_cache = td->td_ucred; : td->td_ucred = NULL; :#endif /* DIAGNOSTIC */ : : :on entering the kernel we do: : : :#ifdef DIAGNOSTIC : if (td->td_ucred) : panic("thread got a cred form somewhere in userspace"); : td->td_cred = td->td_ucred_cache; : td->td_ucred_cache = NULL; :#endif /* DIAGNOSTIC */ : if (td->ucred != p->p_ucred) : cred_update_thread(td); : :we get good performance even when it it is optionned in and :still have a NULL ucred pointer when in user space when DIAGNOSTIC :is turned on. With no DIAGNOSTICS we get the best performance, :and don't even bother to shift the reference. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 11:37:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id 3ED5237B404; Fri, 22 Feb 2002 11:37:28 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g1MJbIfH130754; Fri, 22 Feb 2002 14:37:18 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200202221929.g1MJTFI20465@apollo.backplane.com> References: <200202221929.g1MJTFI20465@apollo.backplane.com> Date: Fri, 22 Feb 2002 14:37:18 -0500 To: Matthew Dillon , Julian Elischer From: Garance A Drosihn Subject: Re: RE: that INVARIANT/ucred freeing stuff. Cc: John Baldwin , arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 11:29 AM -0800 2/22/02, Matthew Dillon wrote: > Sigh. Well, if you really want. I don't like the idea of > the size of the thread structure changing just due to someone > turning on or off DIAGNOSTIC though. When Julian said: >:OK here is my suggestion: > >:add a field to thread of: >:#ifdef DIAGNOSTIC >: td_ucred_cache >:#endif /* DIAGNOSTIC */ perhaps leave the field defined when no DIAGNOSTIC, but call it a different name? (just to make sure nothing is referencing that field when most the code for it is #ifdef-ed out) -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 11:41: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id 3108037B429; Fri, 22 Feb 2002 11:40:11 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020222194011.CYAN1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Fri, 22 Feb 2002 19:40:11 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA74419; Fri, 22 Feb 2002 11:34:49 -0800 (PST) Date: Fri, 22 Feb 2002 11:34:48 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: RE: that INVARIANT/ucred freeing stuff. In-Reply-To: <200202221929.g1MJTFI20465@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 22 Feb 2002, Matthew Dillon wrote: > Sigh. Well, if you really want. I don't like the idea of > the size of the thread structure changing just due to someone > turning on or off DIAGNOSTIC though. > I'LL JUST THROW IN A DUMMY (Geez, bloody capslock).. > -Matt > > :OK here is my suggestion: > : > :We add extra code under DIAGNOSTIC > :the code does: > : > :in proc.h > : > :add a field to thread of: > :#ifdef DIAGNOSTIC > : td_ucred_cache > :#endif /* DIAGNOSTIC */ > : > : > :on texiting the kernel: > : > :#ifdef DIAGNOSTIC > : if (td->td_ucred_cache) > : panic("thread already has cached ucred"); > : td->td_ucred_cache = td->td_ucred; > : td->td_ucred = NULL; > :#endif /* DIAGNOSTIC */ > : > : > :on entering the kernel we do: > : > : > :#ifdef DIAGNOSTIC > : if (td->td_ucred) > : panic("thread got a cred form somewhere in userspace"); > : td->td_cred = td->td_ucred_cache; > : td->td_ucred_cache = NULL; > :#endif /* DIAGNOSTIC */ > : if (td->ucred != p->p_ucred) > : cred_update_thread(td); > : > :we get good performance even when it it is optionned in and > :still have a NULL ucred pointer when in user space when DIAGNOSTIC > :is turned on. With no DIAGNOSTICS we get the best performance, > :and don't even bother to shift the reference. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 12:41: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by hub.freebsd.org (Postfix) with ESMTP id 852C137B420; Fri, 22 Feb 2002 12:40:38 -0800 (PST) Received: from vicor-nb.com (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id 5B2061B21D; Fri, 22 Feb 2002 12:40:38 -0800 (PST) Message-ID: <3C76ACC6.2BB699CF@vicor-nb.com> Date: Fri, 22 Feb 2002 12:40:38 -0800 From: Julian Elischer Organization: VICOR X-Mailer: Mozilla 4.76 [en] (X11; U; FreeBSD 4.5-STABLE i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: John Baldwin Cc: Matthew Dillon , arch@FreeBSD.org Subject: Patch for ucred Content-Type: multipart/mixed; boundary="------------A019902EE9583F50659DADDA" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. --------------A019902EE9583F50659DADDA Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Here's my compromise patch: --------------A019902EE9583F50659DADDA Content-Type: text/plain; charset=us-ascii; name="cred.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="cred.diff" Index: kern/subr_trap.c =================================================================== RCS file: /home/ncvs/src/sys/kern/subr_trap.c,v retrieving revision 1.208 diff -u -r1.208 subr_trap.c --- kern/subr_trap.c 17 Feb 2002 01:09:55 -0000 1.208 +++ kern/subr_trap.c 22 Feb 2002 20:36:45 -0000 @@ -131,7 +131,6 @@ #endif KASSERT(TRAPF_USERMODE(framep), ("ast in kernel mode")); - KASSERT(td->td_ucred == NULL, ("leaked ucred")); #ifdef WITNESS if (witness_list(td)) panic("Returning to user mode with mutex(s) held"); @@ -161,6 +160,13 @@ p->p_stats->p_prof.pr_ticks = 0; } mtx_unlock_spin(&sched_lock); + +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("ast:thread got a cred before reaching AST"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); if (flags & KEF_OWEUPC && sflag & PS_PROFIL) @@ -187,12 +193,13 @@ } userret(td, framep, sticks); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("ast:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; td->td_ucred = NULL; -#endif +#endif /* DIAGNOSTIC */ + s = cpu_critical_enter(); } mtx_assert(&Giant, MA_NOTOWNED); Index: kern/kern_fork.c =================================================================== RCS file: /home/ncvs/src/sys/kern/kern_fork.c,v retrieving revision 1.133 diff -u -r1.133 kern_fork.c --- kern/kern_fork.c 22 Feb 2002 13:31:59 -0000 1.133 +++ kern/kern_fork.c 22 Feb 2002 20:36:45 -0000 @@ -477,6 +477,9 @@ PROC_LOCK(p1); p2->p_ucred = crhold(p1->p_ucred); td2->td_ucred = crhold(p2->p_ucred); /* XXXKSE */ +#ifdef DIAGNOSTIC + td2->td_ucred_cache = NULL; +#endif if (p2->p_args) p2->p_args->ar_ref++; @@ -802,12 +805,12 @@ kthread_exit(0); } PROC_UNLOCK(p); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); - td->td_ucred = NULL; -#endif +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("fork_exit:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; + td->td_ucred = NULL; +#endif /* DIAGNOSTIC */ mtx_assert(&Giant, MA_NOTOWNED); } Index: sys/proc.h =================================================================== RCS file: /home/ncvs/src/sys/sys/proc.h,v retrieving revision 1.204 diff -u -r1.204 proc.h --- sys/proc.h 22 Feb 2002 13:32:01 -0000 1.204 +++ sys/proc.h 22 Feb 2002 20:36:45 -0000 @@ -272,6 +272,11 @@ #define td_endcopy td_pcb struct ucred *td_ucred; /* (k) Reference to credentials. */ +#ifdef DIAGNOSTIC + struct ucred *td_ucred_cache; /* (k) hide cred here for DIAGNOSTIC */ +#else + void *td_dontuse; /* keep the size the same if not DIAG */ +#endif struct pcb *td_pcb; /* (k) Kernel VA of pcb and kstack. */ struct callout td_slpcallout; /* (h) Callout for sleep. */ struct trapframe *td_frame; /* (k) */ Index: alpha/alpha/trap.c =================================================================== RCS file: /home/ncvs/src/sys/alpha/alpha/trap.c,v retrieving revision 1.83 diff -u -r1.83 trap.c --- alpha/alpha/trap.c 19 Feb 2002 03:13:39 -0000 1.83 +++ alpha/alpha/trap.c 22 Feb 2002 20:36:46 -0000 @@ -297,7 +297,12 @@ if (user) { sticks = td->td_kse->ke_sticks; td->td_frame = framep; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("trap: thread got a cred while in userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); } else { @@ -625,12 +630,12 @@ framep->tf_regs[FRAME_SP] = alpha_pal_rdusp(); userret(td, framep, sticks); mtx_assert(&Giant, MA_NOTOWNED); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("trap: thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; td->td_ucred = NULL; -#endif +#endif /* DIAGNOSTIC */ } return; @@ -699,7 +704,12 @@ td->td_frame = framep; opc = framep->tf_regs[FRAME_PC] - 4; sticks = td->td_kse->ke_sticks; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("syscall:thread got a cred while in userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); @@ -822,12 +832,12 @@ */ STOPEVENT(p, S_SCX, code); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("syscall:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; td->td_ucred = NULL; -#endif +#endif /* DIAGNOSTIC */ #ifdef WITNESS if (witness_list(td)) { panic("system call %s returning with mutex(s) held\n", Index: i386/i386/trap.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/trap.c,v retrieving revision 1.212 diff -u -r1.212 trap.c --- i386/i386/trap.c 17 Feb 2002 01:09:54 -0000 1.212 +++ i386/i386/trap.c 22 Feb 2002 20:36:46 -0000 @@ -255,7 +255,12 @@ sticks = td->td_kse->ke_sticks; td->td_frame = &frame; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("trap:thread got a cred while userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); @@ -643,12 +648,12 @@ userret(td, &frame, sticks); mtx_assert(&Giant, MA_NOTOWNED); userout: -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); - td->td_ucred = NULL; -#endif +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("trap:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; + td->td_ucred = NULL; +#endif /* DIAGNOSTIC */ out: return; } @@ -954,7 +959,12 @@ sticks = td->td_kse->ke_sticks; td->td_frame = &frame; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("trap:thread got a cred while userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); params = (caddr_t)frame.tf_esp + sizeof(int); @@ -1099,12 +1109,13 @@ */ STOPEVENT(p, S_SCX, code); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); - td->td_ucred = NULL; -#endif +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("syscall:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; + td->td_ucred = NULL; +#endif /* DIAGNOSTIC */ + #ifdef WITNESS if (witness_list(td)) { panic("system call %s returning with mutex(s) held\n", Index: ia64/ia64/trap.c =================================================================== RCS file: /home/ncvs/src/sys/ia64/ia64/trap.c,v retrieving revision 1.43 diff -u -r1.43 trap.c --- ia64/ia64/trap.c 19 Feb 2002 03:16:50 -0000 1.43 +++ ia64/ia64/trap.c 22 Feb 2002 20:36:46 -0000 @@ -323,7 +323,12 @@ if (user) { sticks = td->td_kse->ke_sticks; td->td_frame = framep; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("trap:thread got a cred while userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); } else { @@ -708,12 +713,12 @@ if (user) { userret(td, framep, sticks); mtx_assert(&Giant, MA_NOTOWNED); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); - td->td_ucred = NULL; -#endif +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("trap:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; + td->td_ucred = NULL; +#endif /* DIAGNOSTIC */ } return; @@ -758,7 +763,12 @@ td->td_frame = framep; sticks = td->td_kse->ke_sticks; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("trap:thread got a cred while userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); @@ -865,12 +875,12 @@ */ STOPEVENT(p, S_SCX, code); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); - td->td_ucred = NULL; -#endif +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("trap:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; + td->td_ucred = NULL; +#endif /* DIAGNOSTIC */ #ifdef WITNESS if (witness_list(td)) { panic("system call %s returning with mutex(s) held\n", Index: powerpc/powerpc/trap.c =================================================================== RCS file: /home/ncvs/src/sys/powerpc/powerpc/trap.c,v retrieving revision 1.7 diff -u -r1.7 trap.c --- powerpc/powerpc/trap.c 19 Feb 2002 03:27:08 -0000 1.7 +++ powerpc/powerpc/trap.c 22 Feb 2002 20:36:46 -0000 @@ -226,7 +226,12 @@ if (user) { sticks = td->td_kse->ke_sticks; td->td_frame = frame; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("trap:thread got a cred while userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); @@ -288,6 +293,7 @@ default: trap_fatal(frame); } + /* NOTREACHED */ } if (sig != 0) { if (p->p_sysent->sv_transtrap != NULL) @@ -296,12 +302,12 @@ } userret(td, frame, sticks); mtx_assert(&Giant, MA_NOTOWNED); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); - td->td_ucred = NULL; -#endif +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("trap:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; + td->td_ucred = NULL; +#endif /* DIAGNOSTIC */ } void Index: sparc64/sparc64/trap.c =================================================================== RCS file: /home/ncvs/src/sys/sparc64/sparc64/trap.c,v retrieving revision 1.23 diff -u -r1.23 trap.c --- sparc64/sparc64/trap.c 19 Feb 2002 03:23:28 -0000 1.23 +++ sparc64/sparc64/trap.c 22 Feb 2002 20:36:46 -0000 @@ -175,7 +175,12 @@ if ((type & T_KERNEL) == 0) { sticks = td->td_kse->ke_sticks; td->td_frame = tf; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("trap:thread got a cred while userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); } else { @@ -379,12 +384,12 @@ user: userret(td, tf, sticks); mtx_assert(&Giant, MA_NOTOWNED); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); - td->td_ucred = NULL; -#endif +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("trap:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; + td->td_ucred = NULL; +#endif /* DIAGNOSTIC */ out: CTR1(KTR_TRAP, "trap: td=%p return", td); return; @@ -540,7 +545,12 @@ sticks = td->td_kse->ke_sticks; td->td_frame = tf; - KASSERT(td->td_ucred == NULL, ("already have a ucred")); +#ifdef DIAGNOSTIC + if (td->td_ucred) + panic("syscall:thread got a cred while userspace"); + td->td_cred = td->td_ucred_cache; + td->td_ucred_cache = NULL; +#endif /* DIAGNOSTIC */ if (td->td_ucred != p->p_ucred) cred_update_thread(td); code = tf->tf_global[1]; @@ -677,12 +687,12 @@ */ STOPEVENT(p, S_SCX, code); -#ifdef INVARIANTS - mtx_lock(&Giant); - crfree(td->td_ucred); - mtx_unlock(&Giant); - td->td_ucred = NULL; -#endif +#ifdef DIAGNOSTIC + if (td->td_ucred_cache) + panic("syscall:thread already has cached ucred"); + td->td_ucred_cache = td->td_ucred; + td->td_ucred = NULL; +#endif /* DIAGNOSTIC */ #ifdef WITNESS if (witness_list(td)) { panic("system call %s returning with mutex(s) held\n", --------------A019902EE9583F50659DADDA-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 12:44:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 4253937B478; Fri, 22 Feb 2002 12:44:22 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1MKiHb22726; Fri, 22 Feb 2002 12:44:17 -0800 (PST) (envelope-from dillon) Date: Fri, 22 Feb 2002 12:44:17 -0800 (PST) From: Matthew Dillon Message-Id: <200202222044.g1MKiHb22726@apollo.backplane.com> To: Julian Elischer Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: Patch for ucred References: <3C76ACC6.2BB699CF@vicor-nb.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Looks pretty good to me. Does it work? -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 13: 0:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hub.freebsd.org (Postfix) with ESMTP id 0814537B417; Fri, 22 Feb 2002 13:00:21 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020222210020.ELAC1147.rwcrmhc52.attbi.com@InterJet.elischer.org>; Fri, 22 Feb 2002 21:00:20 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA74736; Fri, 22 Feb 2002 12:48:07 -0800 (PST) Date: Fri, 22 Feb 2002 12:48:07 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: Julian Elischer , John Baldwin , arch@FreeBSD.ORG Subject: Re: Patch for ucred In-Reply-To: <200202222044.g1MKiHb22726@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG just testing as we speak.. On Fri, 22 Feb 2002, Matthew Dillon wrote: > Looks pretty good to me. Does it work? > > -Matt > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 14:25:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id D9BD137B417; Fri, 22 Feb 2002 14:25:10 -0800 (PST) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 91B445343; Fri, 22 Feb 2002 23:25:08 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: OpenPAM From: Dag-Erling Smorgrav Date: 22 Feb 2002 23:25:07 +0100 Message-ID: Lines: 15 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.1 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG OpenPAM Cantaloupe is now available at along with an integration patch for FreeBSD-CURRENT. Since the two previous releases have solicited absolutely no feedback other than to point out a broken link on the project's web page, I assume that everybody is happy with the code, and that nobody will object when I import it into CVS and ditch Linux-PAM later this weekend. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 14:35:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from holly.dyndns.org (adsl-208-191-149-232.dsl.hstntx.swbell.net [208.191.149.232]) by hub.freebsd.org (Postfix) with ESMTP id 7633937B404 for ; Fri, 22 Feb 2002 14:35:31 -0800 (PST) Received: (from chris@localhost) by holly.dyndns.org (8.11.6/8.9.3) id g1MMYxd44289; Fri, 22 Feb 2002 16:34:59 -0600 (CST) (envelope-from chris) Date: Fri, 22 Feb 2002 16:34:58 -0600 From: Chris Costello To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Subject: Re: OpenPAM Message-ID: <20020222163458.K18366@holly.calldei.com> Reply-To: chris@FreeBSD.ORG References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Fri, Feb 22, 2002 at 11:25:07PM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Friday, February 22, 2002, Dag-Erling Smorgrav wrote: > Since the two previous releases have solicited absolutely no feedback > other than to point out a broken link on the project's web page, I > assume that everybody is happy with the code, and that nobody will > object when I import it into CVS and ditch Linux-PAM later this > weekend. I'm all for it. Thanks, DES! -- +-------------------+-------------------------------------------------+ | Chris Costello | This system will self-destruct in five minutes. | | chris@FreeBSD.org | | +-------------------+-------------------------------------------------+ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 14:42:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id C3CE237B402 for ; Fri, 22 Feb 2002 14:42:47 -0800 (PST) Received: from pool0057.cvx40-bradley.dialup.earthlink.net ([216.244.42.57] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16eOOq-000654-00; Fri, 22 Feb 2002 14:42:44 -0800 Message-ID: <3C76C95B.99B6ECEA@mindspring.com> Date: Fri, 22 Feb 2002 14:42:35 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: OpenPAM References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > OpenPAM Cantaloupe is now available at > > > > along with an integration patch for FreeBSD-CURRENT. > > Since the two previous releases have solicited absolutely no feedback > other than to point out a broken link on the project's web page, I > assume that everybody is happy with the code, and that nobody will > object when I import it into CVS and ditch Linux-PAM later this > weekend. Heh. If it doesn't work we can not bother you, and beat on the project admin instead, right? ;^). Personally, I think it's worthwhile, if only to get out from under the license issue, which has caused me pain more than once. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 15: 0:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id AE82137B405; Fri, 22 Feb 2002 15:00:16 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020222230016.IGXF1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Fri, 22 Feb 2002 23:00:16 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA75260; Fri, 22 Feb 2002 14:42:43 -0800 (PST) Date: Fri, 22 Feb 2002 14:42:42 -0800 (PST) From: Julian Elischer To: Chris Costello Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: OpenPAM In-Reply-To: <20020222163458.K18366@holly.calldei.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The advantages to using linux_pam is obviously that we get to piggyback off them for new kinds of pam modules etc. Is this still the case? can a linux_pam module be used (once compiled for FreeBSD) on a FreeBSD system? how much work is it to convert the source for a Linux Pam module to a BSD-PAM module? The deliberatly gave the Linux-poam stuff a BSD copyright originally to allow us to use it.. WHy does it need to be rewritten? On Fri, 22 Feb 2002, Chris Costello wrote: > On Friday, February 22, 2002, Dag-Erling Smorgrav wrote: > > Since the two previous releases have solicited absolutely no feedback > > other than to point out a broken link on the project's web page, I > > assume that everybody is happy with the code, and that nobody will > > object when I import it into CVS and ditch Linux-PAM later this > > weekend. > > I'm all for it. Thanks, DES! > > -- > +-------------------+-------------------------------------------------+ > | Chris Costello | This system will self-destruct in five minutes. | > | chris@FreeBSD.org | | > +-------------------+-------------------------------------------------+ > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 15: 5: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id CACA037B402; Fri, 22 Feb 2002 15:04:56 -0800 (PST) Received: from pool0057.cvx40-bradley.dialup.earthlink.net ([216.244.42.57] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16eOkI-0006fH-00; Fri, 22 Feb 2002 15:04:55 -0800 Message-ID: <3C76CE8D.1660973B@mindspring.com> Date: Fri, 22 Feb 2002 15:04:45 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: Chris Costello , Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: OpenPAM References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer wrote: > The advantages to using linux_pam is obviously that we get to piggyback > off them for new kinds of pam modules etc. Is this still the case? Yes. Pam is just an API. > can a linux_pam module be used (once compiled for FreeBSD) on a FreeBSD > system? Yes. > how much work is it to convert the source for a Linux Pam module to a > BSD-PAM module? Same as now; most of the time, it's just a recompile, unless there are unexpected Linux-isms in the code to hamper it being portable between UNIX systems. > The deliberatly gave the Linux-poam stuff a BSD copyright originally > to allow us to use it.. WHy does it need to be rewritten? I'll let DES answer that one... though have you looked at the Linux-PAM code? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 15:14:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 7482337B402; Fri, 22 Feb 2002 15:14:53 -0800 (PST) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 6C5535341; Sat, 23 Feb 2002 00:14:50 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Julian Elischer Cc: Chris Costello , arch@FreeBSD.ORG Subject: Re: OpenPAM References: From: Dag-Erling Smorgrav Date: 23 Feb 2002 00:14:50 +0100 In-Reply-To: Message-ID: Lines: 18 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.1 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer writes: > The advantages to using linux_pam is obviously that we get to piggyback > off them for new kinds of pam modules etc. Is this still the case? can a > linux_pam module be used (once compiled for FreeBSD) on a FreeBSD system? > how much work is it to convert the source for a Linux Pam module to a > BSD-PAM module? Did you look at the diffs? > The deliberatly gave the Linux-poam stuff a BSD copyright originally > to allow us to use it.. WHy does it need to be rewritten? Because it sucks rocks, it's a nightmare to debug, it has a very slow release cycle, and maintainer response to bug reports is haphazard. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 15:20:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 4B3A837B402; Fri, 22 Feb 2002 15:20:08 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020222232007.EMS2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Fri, 22 Feb 2002 23:20:07 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA75391; Fri, 22 Feb 2002 15:09:00 -0800 (PST) Date: Fri, 22 Feb 2002 15:08:59 -0800 (PST) From: Julian Elischer To: Terry Lambert Cc: Chris Costello , Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: OpenPAM In-Reply-To: <3C76CE8D.1660973B@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 22 Feb 2002, Terry Lambert wrote: > Julian Elischer wrote: > > The advantages to using linux_pam is obviously that we get to piggyback > > off them for new kinds of pam modules etc. Is this still the case? > > Yes. Pam is just an API. > > > can a linux_pam module be used (once compiled for FreeBSD) on a FreeBSD > > system? > > Yes. > > > how much work is it to convert the source for a Linux Pam module to a > > BSD-PAM module? > > Same as now; most of the time, it's just a recompile, unless > there are unexpected Linux-isms in the code to hamper it being > portable between UNIX systems. > > > The deliberatly gave the Linux-poam stuff a BSD copyright originally > > to allow us to use it.. WHy does it need to be rewritten? > > I'll let DES answer that one... though have you looked at the > Linux-PAM code? It was derived from Mr Tso's PAM code (unveiled at USENIX a few years ago.) He was adamant it was Dual Licensed. (at that time at least). > > -- Terry > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 16: 0:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id E1F2737B400; Fri, 22 Feb 2002 16:00:13 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020223000013.DDDK2951.rwcrmhc53.attbi.com@InterJet.elischer.org>; Sat, 23 Feb 2002 00:00:13 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA75518; Fri, 22 Feb 2002 15:44:40 -0800 (PST) Date: Fri, 22 Feb 2002 15:44:38 -0800 (PST) From: Julian Elischer To: Dag-Erling Smorgrav Cc: Chris Costello , arch@FreeBSD.ORG Subject: Re: OpenPAM In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 23 Feb 2002, Dag-Erling Smorgrav wrote: > Julian Elischer writes: > > The advantages to using linux_pam is obviously that we get to piggyback > > off them for new kinds of pam modules etc. Is this still the case? can a > > linux_pam module be used (once compiled for FreeBSD) on a FreeBSD system? > > how much work is it to convert the source for a Linux Pam module to a > > BSD-PAM module? > > Did you look at the diffs? > > > The deliberatly gave the Linux-poam stuff a BSD copyright originally > > to allow us to use it.. WHy does it need to be rewritten? > > Because it sucks rocks, it's a nightmare to debug, it has a very slow > release cycle, and maintainer response to bug reports is haphazard. That's fair enough then My question was "why?" Not a statement that it was a bad idea or anything.. > > DES > -- > Dag-Erling Smorgrav - des@ofug.org > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 16:10:39 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail12.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by hub.freebsd.org (Postfix) with ESMTP id AECD337B400 for ; Fri, 22 Feb 2002 16:10:35 -0800 (PST) Received: (qmail 21666 invoked from network); 23 Feb 2002 00:10:33 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.91.137.227]) (envelope-sender ) by mail12.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 23 Feb 2002 00:10:33 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200202221835.g1MIZsZ18088@apollo.backplane.com> Date: Fri, 22 Feb 2002 19:10:32 -0500 (EST) From: John Baldwin To: Matthew Dillon Subject: Re: RE: that INVARIANT/ucred freeing stuff. Cc: arch@FreeBSD.ORG, Julian Elischer Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 22-Feb-02 Matthew Dillon wrote: > >:>:John Baldwin <>< http://www.FreeBSD.org/~jhb/ >:> >:> Don't try to overengineer the problem. Unless you believe there is >:> a serious problem, there is no need to put a check in every single >:> conceivable place an error might occur. Just putting a few safety >:> checks >:> in a few critical places should be sufficient. >: >:I don't know where all the places we might look at a ucred wrongly are. >:That's >:why I wanted the much simpler solution of just clearing td_ucred to NULL so >:we >:had an implicit KASSERT for us in all those places. >: >:-- >: >:John Baldwin <>< http://www.FreeBSD.org/~jhb/ > > This doesn't make any sense whatsoever. *NOTHING* was using td_ucred > until just a few days ago, so unless *you* are blindly changing all > p->p_ucred's into td->td_ucred's, I don't see how it can become an issue. Yes, almost all the p_ucred's are changing to td_ucred. That is why we have td_ucred. td_ucred doesn't need a lock, but accessing p_ucred does. Did you read the description of td_ucred back when it was first proposed? > -Matt -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 16:25:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 0D1AF37B404; Fri, 22 Feb 2002 16:25:42 -0800 (PST) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 1BC865341; Sat, 23 Feb 2002 01:25:39 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Julian Elischer Cc: Chris Costello , arch@FreeBSD.ORG Subject: Re: OpenPAM References: From: Dag-Erling Smorgrav Date: 23 Feb 2002 01:25:38 +0100 In-Reply-To: Message-ID: Lines: 17 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.1 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer writes: > My question was "why?" > Not a statement that it was a bad idea or anything.. Right, I'm sorry I reacted negatively to your email. To answer your first question, the changes required to make Linux-PAM modules work with OpenPAM are minimal. They're mostly #include fixups (Linux-PAM headers pull in a lot of system headers, so some modules are missing includes). As for FreeBSD's PAM modules, most of the changes are stuff that's actually FreeBSD-specific, because we've added ad-hoc functions for things that OpenPAM does "natively", and my integration patches remove some of those ad-hoc functions. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 16:52: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 873E437B402; Fri, 22 Feb 2002 16:52:00 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1N0q0t32377; Fri, 22 Feb 2002 16:52:00 -0800 (PST) (envelope-from dillon) Date: Fri, 22 Feb 2002 16:52:00 -0800 (PST) From: Matthew Dillon Message-Id: <200202230052.g1N0q0t32377@apollo.backplane.com> To: John Baldwin Cc: arch@FreeBSD.ORG, Julian Elischer Subject: Re: RE: that INVARIANT/ucred freeing stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :>:John Baldwin <>< http://www.FreeBSD.org/~jhb/ :> :> This doesn't make any sense whatsoever. *NOTHING* was using td_ucred :> until just a few days ago, so unless *you* are blindly changing all :> p->p_ucred's into td->td_ucred's, I don't see how it can become an issue. : :Yes, almost all the p_ucred's are changing to td_ucred. That is why we have :td_ucred. td_ucred doesn't need a lock, but accessing p_ucred does. Did you :read the description of td_ucred back when it was first proposed? : :> -Matt : :-- : :John Baldwin <>< http://www.FreeBSD.org/~jhb/ I'm not an idiot, John, I know what td_ucred does. Remember? I'm the guy who has a bunch of piecemeal commits to remove Giant? My prior statement has not changed. It says exactly what it means... if you are so afraid of td_ucreds being used improperly then I'm asking whether you went over your p_ucred to td_ucred changes or whether you just made the change blindly. Well? Me? I much prefer to commit things one at a time. I've been holding on to my commits for over a week now hoping that you would commit something. I am not happy with having to wait. I am not happy with your intention to bring this whole mess into -current all at once. And I am not happy with your stuff apparently not using Giant instrumentation at all. You seem to be having problems debugging your own tree... maybe it's because there is so much fraggin stuff in it you can't figure out which pieces are causing the problem! After this I'm not going to wait or ask any more... not if every time I do I am forced to wait 2 weeks for someone else to finally commit something they've been holding on to in their own private environment. Not when it interferes with development so fragging much! Not when said person wants to taylor the kernel to work around his own problems rather then work his code to the other way. It has been an unbelievable, phenominal waste of my time. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 17:11:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by hub.freebsd.org (Postfix) with ESMTP id D0F3337B400 for ; Fri, 22 Feb 2002 17:11:40 -0800 (PST) Received: (qmail 2997 invoked from network); 23 Feb 2002 01:11:39 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.91.137.227]) (envelope-sender ) by mail6.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 23 Feb 2002 01:11:39 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Fri, 22 Feb 2002 20:11:38 -0500 (EST) From: John Baldwin To: Julian Elischer Subject: Re: RE: that INVARIANT/ucred freeing stuff. Cc: arch@FreeBSD.org, Matthew Dillon Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 22-Feb-02 Julian Elischer wrote: > OK here is my suggestion: > > We add extra code under DIAGNOSTIC > the code does: Hang on. The only reason this stuff is under DIAGNOSTIC is because of the performance. If you put it under DIAGNOSTIC, please don't make it all gross and complicated, just leave it simple. If you want to optimize it put it under INVARIANTS. Geez. Secondly, did anyone try pushing down Giant into crfree() for the case where we actually call free() and see if that helped the performacne? Giant thrashing is probably what the big problem was. In almost every case you would be just decrementing the refcount using the same logic used to justify this caching scheme. Oh well, for my testing I'll just stick this all back under INVARIANTS and push down Giant in my branch. Much cleaner and useful to me personally. If no one else wants to test this (td_ucred) code more power to them. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 17:15:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 1D0D337B404; Fri, 22 Feb 2002 17:15:08 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1N1F8M32679; Fri, 22 Feb 2002 17:15:08 -0800 (PST) (envelope-from dillon) Date: Fri, 22 Feb 2002 17:15:08 -0800 (PST) From: Matthew Dillon Message-Id: <200202230115.g1N1F8M32679@apollo.backplane.com> To: John Baldwin Cc: Julian Elischer , arch@FreeBSD.ORG Subject: Re: RE: that INVARIANT/ucred freeing stuff. References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :On 22-Feb-02 Julian Elischer wrote: :> OK here is my suggestion: :> :> We add extra code under DIAGNOSTIC :> the code does: : :Hang on. The only reason this stuff is under DIAGNOSTIC is because of the :performance. If you put it under DIAGNOSTIC, please don't make it all gross :and complicated, just leave it simple. If you want to optimize it put it under :INVARIANTS. Geez. : :Secondly, did anyone try pushing down Giant into crfree() for the case where we :actually call free() and see if that helped the performacne? Giant thrashing :is probably what the big problem was. In almost every case you would be :just decrementing the refcount using the same logic used to justify this :caching scheme. : :Oh well, for my testing I'll just stick this all back under INVARIANTS and push :down Giant in my branch. Much cleaner and useful to me personally. If no one :else wants to test this (td_ucred) code more power to them. : :-- : :John Baldwin <>< http://www.FreeBSD.org/~jhb/ :"Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ With Julian's caching code crfree() is taken out of the loop. Poof, gone. Becomes a non-issue. For what it is worth, I agree with you in regards to the weird extra field switching Julian is doing... we do not need to optimize the DIAGNOSTIC case. DIAGNOSTIC is just assumed to be universally non-optimal. But, remember, I'm the guy who just wants to scrap the ucred NULLing completely. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 17:38:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id 5FB4037B416 for ; Fri, 22 Feb 2002 17:38:49 -0800 (PST) Received: (qmail 5660 invoked from network); 23 Feb 2002 01:38:47 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([65.91.137.227]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 23 Feb 2002 01:38:47 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200202230052.g1N0q0t32377@apollo.backplane.com> Date: Fri, 22 Feb 2002 20:38:46 -0500 (EST) From: John Baldwin To: Matthew Dillon Subject: Re: RE: that INVARIANT/ucred freeing stuff. Cc: Julian Elischer , arch@FreeBSD.ORG Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 23-Feb-02 Matthew Dillon wrote: >:>:John Baldwin <>< http://www.FreeBSD.org/~jhb/ >:> >:> This doesn't make any sense whatsoever. *NOTHING* was using td_ucred >:> until just a few days ago, so unless *you* are blindly changing all >:> p->p_ucred's into td->td_ucred's, I don't see how it can become an >:> issue. >: >:Yes, almost all the p_ucred's are changing to td_ucred. That is why we have >:td_ucred. td_ucred doesn't need a lock, but accessing p_ucred does. Did you >:read the description of td_ucred back when it was first proposed? >: >:> -Matt >: >:-- >: >:John Baldwin <>< http://www.FreeBSD.org/~jhb/ > > I'm not an idiot, John, I know what td_ucred does. Remember? I'm > the guy who has a bunch of piecemeal commits to remove Giant? > > My prior statement has not changed. It says exactly what it means... > if you are so afraid of td_ucreds being used improperly then I'm asking > whether you went over your p_ucred to td_ucred changes or whether you > just made the change blindly. Well? No, I did look at what I changed, but I'm aware that I am not all knowing, so I like to be uber-paranoid and have extra checks in place in case something does get screwed up. > Me? I much prefer to commit things one at a time. I've been holding > on to my commits for over a week now hoping that you would commit > something. I am not happy with having to wait. I am not happy with > your intention to bring this whole mess into -current all at once. And > I am not happy with your stuff apparently not using Giant instrumentation > at all. You seem to be having problems debugging your own tree... maybe > it's because there is so much fraggin stuff in it you can't figure out > which pieces are causing the problem! Err, no. Mostly the problems are that I can't get to my test machines from work atm since I'm still waiting to get DSL up and running among other things along with having to do things like get driver's license and other non-geek things. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 17:40:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 842ED37B404; Fri, 22 Feb 2002 17:40:10 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020223014010.DSDW2626.rwcrmhc51.attbi.com@InterJet.elischer.org>; Sat, 23 Feb 2002 01:40:10 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA75892; Fri, 22 Feb 2002 17:36:47 -0800 (PST) Date: Fri, 22 Feb 2002 17:36:45 -0800 (PST) From: Julian Elischer To: Matthew Dillon Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: RE: that INVARIANT/ucred freeing stuff. In-Reply-To: <200202230115.g1N1F8M32679@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG will you guys stop bickering.. for the normal case it's now GONE for DIAGNOSTIC it ZEROs it (and isn't so slow) you both got what you wanted. geeze On Fri, 22 Feb 2002, Matthew Dillon wrote: > > :On 22-Feb-02 Julian Elischer wrote: > :> OK here is my suggestion: > :> > :> We add extra code under DIAGNOSTIC > :> the code does: > : > :Hang on. The only reason this stuff is under DIAGNOSTIC is because of the > :performance. If you put it under DIAGNOSTIC, please don't make it all gross > :and complicated, just leave it simple. If you want to optimize it put it under > :INVARIANTS. Geez. > : > :Secondly, did anyone try pushing down Giant into crfree() for the case where we > :actually call free() and see if that helped the performacne? Giant thrashing > :is probably what the big problem was. In almost every case you would be > :just decrementing the refcount using the same logic used to justify this > :caching scheme. > : > :Oh well, for my testing I'll just stick this all back under INVARIANTS and push > :down Giant in my branch. Much cleaner and useful to me personally. If no one > :else wants to test this (td_ucred) code more power to them. > : > :-- > : > :John Baldwin <>< http://www.FreeBSD.org/~jhb/ > :"Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > > With Julian's caching code crfree() is taken out of the loop. Poof, gone. > Becomes a non-issue. > > For what it is worth, I agree with you in regards to the weird extra field > switching Julian is doing... we do not need to optimize the DIAGNOSTIC case. > DIAGNOSTIC is just assumed to be universally non-optimal. But, remember, > I'm the guy who just wants to scrap the ucred NULLing completely. > > -Matt > Matthew Dillon > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 22 22:33:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 0C4A037B402; Fri, 22 Feb 2002 22:33:09 -0800 (PST) Received: from pool0009.cvx40-bradley.dialup.earthlink.net ([216.244.42.9] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16eVk2-0007co-00; Fri, 22 Feb 2002 22:33:07 -0800 Message-ID: <3C773798.85318848@mindspring.com> Date: Fri, 22 Feb 2002 22:32:56 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: Chris Costello , Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: OpenPAM References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer wrote: > > I'll let DES answer that one... though have you looked at the > > Linux-PAM code? > > It was derived from Mr Tso's PAM code (unveiled at USENIX > a few years ago.) He was adamant it was Dual Licensed. > (at that time at least). I meant, have you made a qualitative comparison of the two code bases? That's what DES asked you to do, in the post you were replying to, when he asked for complaints. If you think this is not an improvement, or isn't simply change for the sake of change, then you should speak up, after you have looked at the code that's there, and the proposed replacement. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 23 7:56:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rina.r.dl.itc.u-tokyo.ac.jp (rina.r.dl.itc.u-tokyo.ac.jp [133.11.199.247]) by hub.freebsd.org (Postfix) with ESMTP id CAC6537B404 for ; Sat, 23 Feb 2002 07:56:37 -0800 (PST) Received: from silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp (silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp [IPv6:3ffe:b80:5b0:3:280:c8ff:fe6b:6d73]) by rina.r.dl.itc.u-tokyo.ac.jp (8.12.2/3.7W-rina.r-Nankai-Koya) with ESMTP id g1NFuYkg097416 ; Sun, 24 Feb 2002 00:56:35 +0900 (JST) Received: from silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp (localhost [127.0.0.1]) by silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp (8.12.2/3.7W-carrots-Keikyu-Kurihama) with ESMTP id g1NFu9N9040749 ; Sun, 24 Feb 2002 00:56:33 +0900 (JST) Message-Id: <200202231556.g1NFu9N9040749@silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp> Date: Sun, 24 Feb 2002 00:56:08 +0900 From: Seigo Tanimura To: arch@FreeBSD.org Subject: reclaiming v_data of free vnodes Cc: Seigo Tanimura User-Agent: Wanderlust/2.8.1 (Something) SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-1?Q?Unebigory=F2mae?=) APEL/10.3 MULE XEmacs/21.1 (patch 14) (Cuyahoga Valley) (i386--freebsd) Organization: Digital Library Research Division, Information Techinology Centre, The University of Tokyo MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thanks to vnlru, cvsup.jp.FreeBSD.org is keeping the number of vnodes to quite sane value. (about 330K out of which 190K are in use) The next problem is the overuse of v_data. vmstat(1) at the uptime of about 24 hours says: Type InUse MemUse HighUse Limit Requests Limit Limit Size(s) FFS node323549 80888K 80897K102400K 29347661 0 0 256 which is almost the same as the number of the total vnodes. (in cvsup.jp.FreeBSD.org, almost all in-use vnodes are actually inodes) This seems due to vrele() and vput() not calling VOP_RECLAIM(). One solution is to always reclaim a vnode in vrele()/vput(), while we can also run a kernel thread to scan the free vnodes and reclaim some of them. Which one would be better, or are there any other ways? Any comments are welcome. -- Seigo Tanimura To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 23 8:11: 2 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rina.r.dl.itc.u-tokyo.ac.jp (rina.r.dl.itc.u-tokyo.ac.jp [133.11.199.247]) by hub.freebsd.org (Postfix) with ESMTP id E07F637B405 for ; Sat, 23 Feb 2002 08:11:00 -0800 (PST) Received: from silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp (silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp [IPv6:3ffe:b80:5b0:3:280:c8ff:fe6b:6d73]) by rina.r.dl.itc.u-tokyo.ac.jp (8.12.2/3.7W-rina.r-Nankai-Koya) with ESMTP id g1NGArkg097683 ; Sun, 24 Feb 2002 01:10:57 +0900 (JST) Received: from silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp (localhost [127.0.0.1]) by silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp (8.12.2/3.7W-carrots-Keikyu-Kurihama) with ESMTP id g1NGAHN9044345 ; Sun, 24 Feb 2002 01:10:43 +0900 (JST) Message-Id: <200202231610.g1NGAHN9044345@silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp> Date: Sun, 24 Feb 2002 01:10:17 +0900 From: Seigo Tanimura To: arch@FreeBSD.org Cc: Seigo Tanimura Subject: Re: reclaiming v_data of free vnodes In-Reply-To: <200202231556.g1NFu9N9040749@silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp> References: <200202231556.g1NFu9N9040749@silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp> User-Agent: Wanderlust/2.8.1 (Something) SEMI/1.14.3 (Ushinoya) FLIM/1.14.3 (=?ISO-8859-1?Q?Unebigory=F2mae?=) APEL/10.3 MULE XEmacs/21.1 (patch 14) (Cuyahoga Valley) (i386--freebsd) Organization: Digital Library Research Division, Information Techinology Centre, The University of Tokyo MIME-Version: 1.0 (generated by SEMI 1.14.3 - "Ushinoya") Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 24 Feb 2002 00:56:08 +0900, Seigo Tanimura said: Seigo> Type InUse MemUse HighUse Limit Requests Limit Limit Size(s) Seigo> FFS node323549 80888K 80897K102400K 29347661 0 0 256 Seigo> which is almost the same as the number of the total vnodes. (in Seigo> cvsup.jp.FreeBSD.org, almost all in-use vnodes are actually inodes) Last night the machine paniced because it consumed the whole kmem_map, although I missed the number of inodes. -- Seigo Tanimura To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 23 8:55:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from gull.prod.itd.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by hub.freebsd.org (Postfix) with ESMTP id 774CC37B417 for ; Sat, 23 Feb 2002 08:55:34 -0800 (PST) Received: from pool0212.cvx22-bradley.dialup.earthlink.net ([209.179.198.212] helo=mindspring.com) by gull.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16efSI-0005nN-00; Sat, 23 Feb 2002 08:55:26 -0800 Message-ID: <3C77C972.598B4181@mindspring.com> Date: Sat, 23 Feb 2002 08:55:14 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Seigo Tanimura Cc: arch@FreeBSD.org Subject: Re: reclaiming v_data of free vnodes References: <200202231556.g1NFu9N9040749@silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Seigo Tanimura wrote: > which is almost the same as the number of the total vnodes. (in > cvsup.jp.FreeBSD.org, almost all in-use vnodes are actually inodes) > > This seems due to vrele() and vput() not calling VOP_RECLAIM(). One > solution is to always reclaim a vnode in vrele()/vput(), while we can > also run a kernel thread to scan the free vnodes and reclaim some of > them. Which one would be better, or are there any other ways? > > Any comments are welcome. Taking the first quoted sentence, above, into account, it seems that the SVR4 approach of giving the vnode tot the FS to manage, instead of using a system wide pool, would be the best approach. That way the vnode and in core inode allocation are combined (in fact, probably the same "malloc"). It also inherently parallelizes allocation when two or more FS's are in use, which in turn starts to work around one of the big problems facing the concurrency push-down in SMPng. The ihash cache really needs to go away -- or at least, it needs to keep the page associations, so that clean pages that are in core don't get faulted in again because of a disassociated vnode/inode pair when the inode is reclaimed. This is also at the root of your problem, where there is no explicit reclaim possible because the vnode and inode are not tightly coupled: you can free resources used by one and not the other, but have no way to reclaim the relationship from the vnode dise (the inode side can reclaim the relationship, but loses the cached page associations). Frankly, it makes sense to me, even if the system retains ownership of the vnode, to find a way to pass the vput/vrele notification to the VFS via a VOP_VRELE (or something similar), so that there is explicit notification. THis is pretty much a requirement for more than trivial stacking, at some point, unless you want to greatly complicate each of the stacking layers. Combining would also help in the nfsnode cases, I think. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 23 13:34:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 9574F37B419; Sat, 23 Feb 2002 13:34:54 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id IAA24463; Sun, 24 Feb 2002 08:34:44 +1100 Date: Sun, 24 Feb 2002 08:35:00 +1100 (EST) From: Bruce Evans X-X-Sender: To: Julian Elischer Cc: John Baldwin , Matthew Dillon , Subject: Re: Patch for ucred In-Reply-To: <3C76ACC6.2BB699CF@vicor-nb.com> Message-ID: <20020224082720.S30437-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 22 Feb 2002, Julian Elischer wrote: > Here's my compromise patch: Gak. It's totally over-the-top-engineered. It's not as bad as the committed version though (that version has a 20-line comment which explains everything except _why_ it complicates things to optimize DIAGNOSTIC code, and pointers to the comment from all the diagnostic ifdefs...). Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 23 15:22: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id C750E37B404; Sat, 23 Feb 2002 15:21:52 -0800 (PST) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 19C545341; Sun, 24 Feb 2002 00:21:50 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: bin/25059 (rtld bug) From: Dag-Erling Smorgrav Date: 24 Feb 2002 00:21:49 +0100 Message-ID: Lines: 24 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.1 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --=-=-= Our dlopen(3) man page states: RTLD_GLOBAL Symbols from this shared object and its directed acyclic graph (DAG) of needed objects will be available for resolv ing undefined references from all other shared objects. but this is a lie - only the object itself is searched, not its DAG. The attached patch fixes this by adding a flag to the Obj_Entry structure that marks an object as global, and changing symlook_list() to search each global object's DAG if the object itself doesn't have the requested symbol. With this patch applied, the test program referenced in the PR works. (some might argue that symlook_obj() should be changed instead - but changing symlook_list() was simpler, and the only difference it makes is, in the worst case, a slightly longer search) DES -- Dag-Erling Smorgrav - des@ofug.org --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=rtld.diff Index: rtld.c =================================================================== RCS file: /home/ncvs/src/libexec/rtld-elf/rtld.c,v retrieving revision 1.60 diff -u -r1.60 rtld.c --- rtld.c 17 Feb 2002 07:00:25 -0000 1.60 +++ rtld.c 23 Feb 2002 23:21:28 -0000 @@ -1329,6 +1329,7 @@ elm = NEW(Objlist_Entry); elm->obj = obj; STAILQ_INSERT_TAIL(list, elm, link); + obj->global = true; } static void @@ -1581,7 +1582,7 @@ if (obj) { obj->dl_refcount++; - if (mode & RTLD_GLOBAL && objlist_find(&list_global, obj) == NULL) + if (mode & RTLD_GLOBAL && !obj->global) objlist_push_tail(&list_global, obj); mode &= RTLD_MODEMASK; if (*old_obj_tail != NULL) { /* We loaded something new. */ @@ -1915,7 +1916,7 @@ } } - /* Search all RTLD_GLOBAL objects. */ + /* Search all RTLD_GLOBAL objects and their DAGs. */ if (def == NULL || ELF_ST_BIND(def->st_info) == STB_WEAK) { symp = symlook_list(name, hash, &list_global, &obj, in_plt, &donelist); if (symp != NULL && @@ -1965,6 +1966,12 @@ if (ELF_ST_BIND(def->st_info) != STB_WEAK) break; } + } else if (elm->obj->global) { + /* search the DAGs of global objects */ + symp = symlook_list(name, hash, &elm->obj->dagmembers, + defobj_out, in_plt, dlp); + if (symp != NULL) + return symp; } } if (def != NULL) Index: rtld.h =================================================================== RCS file: /home/ncvs/src/libexec/rtld-elf/rtld.h,v retrieving revision 1.24 diff -u -r1.24 rtld.h --- rtld.h 29 Oct 2001 10:10:02 -0000 1.24 +++ rtld.h 23 Feb 2002 23:02:18 -0000 @@ -155,6 +155,7 @@ bool traced; /* Already printed in ldd trace output */ bool jmpslots_done; /* Already have relocated the jump slots */ bool init_done; /* Already have added object to init list */ + bool global; /* This object is on the global list */ struct link_map linkmap; /* for GDB */ Objlist dldags; /* Object belongs to these dlopened DAGs (%) */ --=-=-=-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 23 19: 0:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id D866937B416; Sat, 23 Feb 2002 19:00:12 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020224030011.MHGI1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Sun, 24 Feb 2002 03:00:11 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id SAA81085; Sat, 23 Feb 2002 18:48:56 -0800 (PST) Date: Sat, 23 Feb 2002 18:48:55 -0800 (PST) From: Julian Elischer To: Bruce Evans Cc: Julian Elischer , John Baldwin , Matthew Dillon , arch@FreeBSD.ORG Subject: Re: Patch for ucred In-Reply-To: <20020224082720.S30437-100000@gamplex.bde.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Do you have anything constructive to add? I want to simply rip out the zeroing stuff.. unconditionally. On Sun, 24 Feb 2002, Bruce Evans wrote: > On Fri, 22 Feb 2002, Julian Elischer wrote: > > > Here's my compromise patch: > > Gak. It's totally over-the-top-engineered. It's not as bad as the > committed version though (that version has a 20-line comment which > explains everything except _why_ it complicates things to optimize > DIAGNOSTIC code, and pointers to the comment from all the diagnostic > ifdefs...). > > Bruce > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 23 19:12:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 88FAA37B400; Sat, 23 Feb 2002 19:12:25 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id OAA07483; Sun, 24 Feb 2002 14:12:14 +1100 Date: Sun, 24 Feb 2002 14:12:31 +1100 (EST) From: Bruce Evans X-X-Sender: To: Julian Elischer Cc: Julian Elischer , John Baldwin , Matthew Dillon , Subject: Re: Patch for ucred In-Reply-To: Message-ID: <20020224140739.Q31494-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 23 Feb 2002, Julian Elischer wrote: > Do you have anything constructive to add? > > I want to simply rip out the zeroing stuff.. > unconditionally. Same here. But I don't mind 10 lines of unobtrusive code for it. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message