From owner-freebsd-arch Sun Jan 13 4:20:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 4035A37B417 for ; Sun, 13 Jan 2002 04:20:31 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id XAA09338; Sun, 13 Jan 2002 23:20:20 +1100 Date: Sun, 13 Jan 2002 23:21:08 +1100 (EST) From: Bruce Evans X-X-Sender: To: Jake Burkholder Cc: Thomas Moestl , Subject: Re: adding more endian conversion and bus space functions In-Reply-To: <20020112115513.L39321@locore.ca> Message-ID: <20020113230455.K709-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 12 Jan 2002, Jake Burkholder wrote: > Apparently, On Fri, Jan 11, 2002 at 10:00:45PM +1100, > Bruce Evans said words to the effect of; > > Mike Barcroft is doing this. IIRC, it doesn't have , since > > the ntohl() family is declared in according to POSIX, so > > would be neither standard no useful. > > It is useful to if we don't have to duplicate C implementations > of byte swapping functions 5 times. The current ones are in libc. Mike moved most of the common parts to and/or . > > I think the bus > > space headers should not depend on any endianness support in other > > headers except defining _[_]BYTE_ORDER. > > Why? I disagree. Because they are specialized for bus accesses and need to support many more types of accesses than . They can easily duplicate the small part of ntohl(), etc., that they need (if they need it), like the i386 one already does for most of the i/o instructions in the i386 cpufunc.h. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 8:56:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from aslan.scsiguy.com (aslan.scsiguy.com [63.229.232.106]) by hub.freebsd.org (Postfix) with ESMTP id 8305437B417 for ; Sun, 13 Jan 2002 08:56:22 -0800 (PST) Received: from scsiguy.com (localhost [127.0.0.1]) by aslan.scsiguy.com (8.11.5/8.11.5) with ESMTP id g0DGuJg21488; Sun, 13 Jan 2002 09:56:19 -0700 (MST) (envelope-from gibbs@scsiguy.com) Message-Id: <200201131656.g0DGuJg21488@aslan.scsiguy.com> To: Thomas Moestl Cc: freebsd-arch@FreeBSD.ORG Subject: Re: adding more endian conversion and bus space functions In-Reply-To: Your message of "Fri, 11 Jan 2002 01:52:07 +0100." <20020111005207.GA7246@crow.dom2ip.de> Date: Sun, 13 Jan 2002 09:56:19 -0700 From: "Justin T. Gibbs" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >3. stream variants of the bus space access functions; these become > needed in the case where a bus system uses another endianess than > the process, as is the case with PCI on sparc64. The normal bus > space functions will perform byte-swapping as appropriate to convert > the data to the host byte order (supported by hardware on sparc64). > However if 16- or 32-bit accesses are used for stream data, > i.e. data that should not be byte-swapped, these functions need be > used instead. > These will simply be defined to their non-stream counterparts on > architectures where bus and processor byte order do not differ, > i.e. all other currently supported architectures. Are you implying that the non-stream variants of certain bus functions will auto endian convert? This would be really bad for drivers that already perform the endian conversion manually if it is necessary. This would be akin to having a network board auto-convert all TCP or IP header information to host order while dma'ing the packet to host memory. Confusion would abound. PCI is LE. Hiding that from drivers is a bad thing. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 9:58: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from musique.teaser.net (musique.teaser.net [213.91.2.11]) by hub.freebsd.org (Postfix) with ESMTP id E2DB637B416 for ; Sun, 13 Jan 2002 09:57:58 -0800 (PST) Received: from notbsdems.nantes.kisoft-services.com (nantes.kisoft-services.com [193.56.60.243]) by musique.teaser.net (Postfix) with ESMTP id 9F5977250B for ; Sun, 13 Jan 2002 18:57:57 +0100 (CET) Received: by notbsdems.nantes.kisoft-services.com (Postfix, from userid 1001) id 5C927E6DB0; Sun, 13 Jan 2002 11:25:04 +0100 (CET) To: Mailing List FreeBSD Arch Subject: Cryptographic hardware support From: Eric Masson X-Operating-System: FreeBSD 4.5-PRERELEASE i386 Date: Sun, 13 Jan 2002 11:25:03 +0100 Message-ID: <867kqm1rb4.fsf@notbsdems.nantes.kisoft-services.com> Lines: 24 User-Agent: Gnus/5.090005 (Oort Gnus v0.05) XEmacs/21.1 (Cuyahoga Valley, i386--freebsd) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, I've googled about FreeBSD crypto hardware support and didn't find any whitepaper or draft about selected implementation. Is there any paper describing the implementation framework ? Has anyone considered the way OpenBSD did this integration (flaws, strenghts...) ? Wouldn't it be interesting to share this type of support across the bsd's (no duplicate of work for hardware support for example) ? Is there any roadmap for inclusion in the tree ? Thanks for your answers. Eric Masson -- > How do I run Crack under DOS/Win95? Reformat your hard-drive and install Linux, then try again. CAUTION: this process may lose data. Crack5.0a in Guide du linuxien pervers : "Il y a toujours une solution !" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 10:11:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by hub.freebsd.org (Postfix) with SMTP id 63C8737B41D for ; Sun, 13 Jan 2002 10:11:35 -0800 (PST) Received: (qmail 20777 invoked by uid 0); 13 Jan 2002 18:11:33 -0000 Received: from pd9e16c58.dip.t-dialin.net (HELO forge.local) (217.225.108.88) by mail.gmx.net (mp003-rz3) with SMTP; 13 Jan 2002 18:11:33 -0000 Received: from tmm by forge.local with local (Exim 3.30 #1) id 16Pp6r-00018N-00; Sun, 13 Jan 2002 19:11:57 +0100 Date: Sun, 13 Jan 2002 19:11:57 +0100 From: Thomas Moestl To: "Justin T. Gibbs" Cc: freebsd-arch@FreeBSD.ORG Subject: Re: adding more endian conversion and bus space functions Message-ID: <20020113181157.GA451@crow.dom2ip.de> Mail-Followup-To: "Justin T. Gibbs" , freebsd-arch@FreeBSD.ORG References: <20020111005207.GA7246@crow.dom2ip.de> <200201131656.g0DGuJg21488@aslan.scsiguy.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200201131656.g0DGuJg21488@aslan.scsiguy.com> User-Agent: Mutt/1.3.25i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 2002/01/13 at 09:56:19 -0700, Justin T. Gibbs wrote: > >3. stream variants of the bus space access functions; these become > > needed in the case where a bus system uses another endianess than > > the process, as is the case with PCI on sparc64. The normal bus > > space functions will perform byte-swapping as appropriate to convert > > the data to the host byte order (supported by hardware on sparc64). > > However if 16- or 32-bit accesses are used for stream data, > > i.e. data that should not be byte-swapped, these functions need be > > used instead. > > These will simply be defined to their non-stream counterparts on > > architectures where bus and processor byte order do not differ, > > i.e. all other currently supported architectures. > > Are you implying that the non-stream variants of certain bus functions > will auto endian convert? Yes. > This would be really bad for drivers that > already perform the endian conversion manually if it is necessary. This > would be akin to having a network board auto-convert all TCP or IP > header information to host order while dma'ing the packet to host memory. > Confusion would abound. PCI is LE. Hiding that from drivers is a > bad thing. The rationale for that is that some architectures (of which UltraSPARC is one) have hardware support for this conversion. Also, the cases where the stream variants are needed are relatively rare compared to those where the existing functions would be used (i.e. the accessed data is interpreted is an integer of the given size), so it usually minimizes the changes that are needed to make drivers work on big-endian architectures. Additionally, it makes it easier to write drivers for devices that are available for bus systems of different endianesses (e.g. hme). These points add up to quite a win IMHO, so I really would like to do it this way. Are there many drivers around that already do their own conversions? To me, it seems that most that are prepared to be used on big-endian systems already rely on the bus space functions to do the appropriate conversions (since on NetBSD, they do so), but I may be wrong. - thomas To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 10:49: 8 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 1C64F37B419 for ; Sun, 13 Jan 2002 10:49:02 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.3/8.11.3) with ESMTP id g0DIn0l04533; Sun, 13 Jan 2002 11:49:01 -0700 (MST) (envelope-from imp@village.org) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.11.6/8.11.6) with ESMTP id g0DIn0x68134; Sun, 13 Jan 2002 11:49:00 -0700 (MST) (envelope-from imp@village.org) Date: Sun, 13 Jan 2002 11:48:28 -0700 (MST) Message-Id: <20020113.114828.53910112.imp@village.org> To: tmoestl@gmx.net Cc: gibbs@scsiguy.com, freebsd-arch@FreeBSD.ORG Subject: Re: adding more endian conversion and bus space functions From: "M. Warner Losh" In-Reply-To: <20020113181157.GA451@crow.dom2ip.de> References: <20020111005207.GA7246@crow.dom2ip.de> <200201131656.g0DGuJg21488@aslan.scsiguy.com> <20020113181157.GA451@crow.dom2ip.de> X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <20020113181157.GA451@crow.dom2ip.de> Thomas Moestl writes: : Are there many drivers around that already do their own conversions? : To me, it seems that most that are prepared to be used on big-endian : systems already rely on the bus space functions to do the appropriate : conversions (since on NetBSD, they do so), but I may be wrong. The extent to which these things are supported on FreeBSD (that is running Xendian host with Yendian bus) is in the bus layer. Nothing does the conversion now. I disagree with Justin because of one thing he said: "PCI is LE". None of the drivers in the tree right now assure that they data they write to the bus is LE. They just call bus_space_write (or in extremely lame cases like the cardbus bridge code, write directly to the memory mapped area with *(foo->offset) = 0x12345678;). NetBSD does this fairly well right now, so I don't think that the confusion that Justin is worried about exists there. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 12:12:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id 8075A37B417 for ; Sun, 13 Jan 2002 12:12:47 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id HAA27621; Mon, 14 Jan 2002 07:12:30 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37641) with ESMTP id <01KD1WTL4UJ4VFMSOV@cim.alcatel.com.au>; Mon, 14 Jan 2002 07:12:48 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.6/8.11.6) id g0DKCRR24555; Mon, 14 Jan 2002 07:12:27 +1100 Content-return: prohibited Date: Mon, 14 Jan 2002 07:12:26 +1100 From: Peter Jeremy Subject: Re: Request for review: getcontext, setcontext, etc In-reply-to: <20020111145159.N7984@elvis.mu.org>; from bright@mu.org on Fri, Jan 11, 2002 at 02:51:59PM -0800 To: Alfred Perlstein Cc: Nate Williams , Bakul Shah , Dan Eischen , arch@FreeBSD.ORG Mail-Followup-To: Alfred Perlstein , Nate Williams , Bakul Shah , Dan Eischen , arch@FreeBSD.ORG Message-id: <20020114071226.R561@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <3C37E559.B011DF29@vigrid.com> <200201112141.QAA25529@devonshire.cnchost.com> <15423.27120.926839.725176@caddis.yogotech.com> <20020111145159.N7984@elvis.mu.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-Jan-11 14:51:59 -0800, Alfred Perlstein wrote: >* Nate Williams [020111 14:46] wrote: >> >> The point is that this may not be a valid assumption w/regard to the FPU >> state. The necessity of saving/restoring the FPU state *IS* the primary >> subject of the the entire discussion, with the secondary part being that >> x86 hardware is broken, so it may not be possible to guarantee delivery >> of FPU exceptions to the same context that caused it. > >Couldn't this just be simply done by calling the "wait for fop to >complete" instruction before switching out an FP using thread? Given the current implementation (lazy FPU context load), if the thread hadn't previously use FP, then issuing an fwait will cause a "device not available" trap to the kernel. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 12:25:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 96C9F37B405 for ; Sun, 13 Jan 2002 12:25:35 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0DKPYle063133 for ; Sun, 13 Jan 2002 21:25:34 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0DKPYHD063130 for ; Sun, 13 Jan 2002 21:25:34 +0100 (CET) Date: Sun, 13 Jan 2002 21:25:34 +0100 (CET) From: Michal Mertl To: arch@freebsd.org Subject: 64 bit counters again Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG If I'm writing to this forum about inappropriate issue, you could tell me, rather than just ignore me. But anyway I continued on some work on STABLE (but believe lot of my work could be used on CURRENT after some modification) and get kernel and world building with 64 bit counters on network interfaces (struct if_data) and in protocols (struct ipstat, tcpstat, udpstat, icmpstat, igmpstat). I implemented new machine dependant include file (but it could be quite easily made MI with some cpp magic I believe) and atomic_??_quad functions (conditionally included). I have new API for counters which resembles lot atomic?? functions but has one more parameter. That parametr is used to tell in which part of kernel source is the variable and the functions then does either atomic or "normal" operation on the counter. This variable needn't to be defined this way, it could also be used to directly tell what kind of operation is required but I believe this way it's more flexible. example: ------------------------- static __inline void syscntr_add(const int loc, volatile u_int64_t *p, u_int32_t v) { switch (loc) { case CNTR_NET_PROTO: case CNTR_NET_DEV: #if defined(SYSCNTR_ASM) __asm __volatile ("#syscntr_add_quad addl %1,(%0) adcl $0,4(%0)" : : "S" (p), "q" (v)); #else *p+=v; #endif break; case CNTR_SOMETHING: atomic_add_quad(p,v); break; default: atomic_add_quad(p,v); } } ------------------------ It should always inline to either call to atomic function (in kernel modules and external programs) or the simple operation. I did small benchmark (on UP but will soon test hopefully on SMP and let you know). I moved 256MB trhough 100FDX and measured the throughput and load (using vmstat). I dind't notice any performance difference between 32/64 bit, atomic/non-atomic functions (but atomic was without lock istructions). Before you start telling it's the wrong way to go (on CURRENT at least) I can see there may be other modes of operation of the functions, PER_CPU data of whatever - using something like the API provided would hide it nicely. Once something is in a subsystem which isn't under giant or some other change with locking or such you would change just one place and the variables would be kept accessed the right way. In case someone wants to see the whole thing it's at http://home.eunet.cz/mime/syscntr.patch.0113.gz I know there are some style problems - at least too-long lines. I have them noted down and I'm ready to split them. Only I don't know what is the right maximum length and how much spaces I should count for tabs. I also noticed, that some drivers do define similar counters like if_data in their sources - if_ar.c and others - which is for netgraph I believe. I could quite easily change these variables to the newly defined type too but I don't know, if one can easily change variable type in netgraph - I didn't want to violate some standard. Also there are several drivers (bge) which use dot3stats variables from if_mib for error counting. These are also 32 bits. -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 12:43: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id 116A037B400 for ; Sun, 13 Jan 2002 12:43:00 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id HAA02107; Mon, 14 Jan 2002 07:42:42 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37640) with ESMTP id <01KD1XUNKHY8VLQITH@cim.alcatel.com.au>; Mon, 14 Jan 2002 07:42:41 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.6/8.11.6) id g0DKgdp24650; Mon, 14 Jan 2002 07:42:39 +1100 Content-return: prohibited Date: Mon, 14 Jan 2002 07:42:39 +1100 From: Peter Jeremy Subject: Re: Request for review: getcontext, setcontext, etc In-reply-to: <20020112205919.E5372-100000@gamplex.bde.org>; from bde@zeta.org.au on Sat, Jan 12, 2002 at 09:40:20PM +1100 To: Bruce Evans Cc: Terry Lambert , Peter Wemm , Alfred Perlstein , Kelly Yancey , Nate Williams , Daniel Eischen , Dan Eischen , Archie Cobbs , arch@FreeBSD.ORG Mail-Followup-To: Bruce Evans , Terry Lambert , Peter Wemm , Alfred Perlstein , Kelly Yancey , Nate Williams , Daniel Eischen , Dan Eischen , Archie Cobbs , arch@FreeBSD.ORG Message-id: <20020114074238.S561@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <3C4001A3.5ECCAEB9@mindspring.com> <20020112205919.E5372-100000@gamplex.bde.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-Jan-12 21:40:20 +1100, Bruce Evans wrote: >On Sat, 12 Jan 2002, Terry Lambert wrote: ... >> Assuming a 64 bit data path, then we are talking a minimum of >> 3 * 512/(64/8) * (16:1) or 3k (3076) clocks to save the damn FPU >> state off to main memory (a store in a loop is 3 clocks ignoring >> the setup and crap, right?). Add another 3k clocks to bring it >> back. >> >> Best case, God loves us, and we spill and restore from L1 >> without an IPI or an invalidation, and without starting the >> thread on a CPU other than the one where it was suspended, and >> all spills are to cacheable write-through pages. That's a 16 >> times speed increase because we get to ignore the bus speed >> differential, or 3 * 512/(65/8) * 2 = (6k/16) = 384 clocks. > >This seems to be off by a bit. Actual timing on an Athlon1600 >overclocked a little gives the following times for some crtical >parts of context switching for each iteration of instructions in >a loop (not counting 2 cycles of loop overhead): > >pushal; popal: 9 cycles >pushl %ds; popl %ds: 21 cycles >fxsave; fxrstor: 105 cycles >fnsave; frstor: 264 cycles I can think of a possible reason: The FPU knows when it has been used vs just having executed fninit. In the latter case, all it needs to save is "I've been initialised". Also the FPU architecture includes "used" flags associated with each register - possibly the f*save instructions don't flush unused registers. Do the above numbers change when you push real data into the FP registers? Also, how expensive is a DNA trap? Would it be cheaper overall to always load FPU context on a switch - this is more expensive for processes that don't use FP, but saves a DNA trap per context switch (assuming they use FP in that slice) for those that do. To add some further numbers, in December 1999, I did some measurements on FP switching by patching npx.c. This was on a PII-266 running then -current. (The original e-mail was sent to -arch on Mon, 20 Dec 1999 07:34:06 +1100 in a thread titled "Concrete plans for ucontext/ mcontext changes around 4.0" - I don't have the message-id available). ctxt DNA FP swtch traps swtch 1754982 281557 59753 build world and a few CVS operations [1] 79044 18811 10341 gnuplot and xv in parallel [2] 800 138 130 parallel FP-intensive progs [3]. In the above, `ctxt swtch' is the number of context switches counted via vm.stats.sys.v_swtch. `DNA traps' is the number of device not available traps registered and `FP swtch' is the number of DNA traps where the FP context loaded is different to that saved on the preceeding context switch. Notes: [1] Boot to single user, run 'make buildworld' inside script(1). The buildworld had a few hiccups along the way which I patched around and then re-ran 'make everything'. [2] I ran the gnuplot demos and the xv visual schnauzer updating a large directory of pictures in parallel. (Multi-user X11). [3] This was four parallel copies of a circuit analysis program I wrote. It spent most of its time solving a complex 26x26 matrix using Gaussian elimination. (Multi-user console). Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 12:54:32 2002 Delivered-To: freebsd-arch@freebsd.org Received: from hawk.prod.itd.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by hub.freebsd.org (Postfix) with ESMTP id 92C9637B41A for ; Sun, 13 Jan 2002 12:54:29 -0800 (PST) Received: from pool0417.cvx40-bradley.dialup.earthlink.net ([216.244.43.162] helo=mindspring.com) by hawk.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Pre2-0007AV-00; Sun, 13 Jan 2002 12:54:22 -0800 Message-ID: <3C41F3FD.4ECC8CD@mindspring.com> Date: Sun, 13 Jan 2002 12:54:21 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Michal Mertl Cc: arch@freebsd.org Subject: Re: 64 bit counters again References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Michal Mertl wrote: > But anyway I continued on some work on STABLE (but believe lot of my work > could be used on CURRENT after some modification) and get kernel and world > building with 64 bit counters on network interfaces (struct if_data) > and in protocols (struct ipstat, tcpstat, udpstat, icmpstat, igmpstat). Not to discount the value of this work, but... 1) It makes counting slower on 32 bit processors. 2) The values are not accessible from SNMP, which is limited to 32 bit counters. 3) While you could export these values as strings and not numbers over SNMP, doing this would mean you would need to use a MIB which was not a superset of an RFC'ed MIB. So it seems to me that the utility of this on 32 bit machines is incredibly limited (e.g. shell programs). PS: FWIW, I agree that these things should be 64 bits on 64 bit architectures, even if they can only export the low 32 bits for SNMP. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 12:57:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from hawk.prod.itd.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by hub.freebsd.org (Postfix) with ESMTP id 44D8137B402 for ; Sun, 13 Jan 2002 12:57:28 -0800 (PST) Received: from pool0417.cvx40-bradley.dialup.earthlink.net ([216.244.43.162] helo=mindspring.com) by hawk.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16PrgV-0001yM-00; Sun, 13 Jan 2002 12:56:56 -0800 Message-ID: <3C41F495.3660F8F6@mindspring.com> Date: Sun, 13 Jan 2002 12:56:53 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Peter Jeremy Cc: Bruce Evans , Peter Wemm , Alfred Perlstein , Kelly Yancey , Nate Williams , Daniel Eischen , Dan Eischen , Archie Cobbs , arch@FreeBSD.ORG Subject: Re: Request for review: getcontext, setcontext, etc References: <3C4001A3.5ECCAEB9@mindspring.com> <20020112205919.E5372-100000@gamplex.bde.org> <20020114074238.S561@gsmx07.alcatel.com.au> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Jeremy wrote: > Also, how expensive is a DNA trap? Would it be cheaper overall to > always load FPU context on a switch - this is more expensive for > processes that don't use FP, but saves a DNA trap per context switch > (assuming they use FP in that slice) for those that do. I'd like to see a kernel that saves FPU state always benched against a kernel that does lazy binding, using the "make world weighted average of three runs" benchmark. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 13:33:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from freesbee.wheel.dk (freesbee.wheel.dk [193.162.159.97]) by hub.freebsd.org (Postfix) with ESMTP id D971D37B417 for ; Sun, 13 Jan 2002 13:33:52 -0800 (PST) Received: by freesbee.wheel.dk (Postfix, from userid 1001) id 8B4435DCD; Sun, 13 Jan 2002 22:33:51 +0100 (CET) Date: Sun, 13 Jan 2002 22:33:51 +0100 From: Jesper Skriver To: Terry Lambert Cc: Michal Mertl , arch@freebsd.org Subject: Re: 64 bit counters again Message-ID: <20020113223351.A42689@FreeBSD.org> Mail-Followup-To: Jesper Skriver , Terry Lambert , Michal Mertl , arch@freebsd.org References: <3C41F3FD.4ECC8CD@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3C41F3FD.4ECC8CD@mindspring.com>; from tlambert2@mindspring.com on Sun, Jan 13, 2002 at 12:54:21PM -0800 X-PGP-Fingerprint: 6B88 9CE8 66E9 E631 C9C5 5EB4 22AB F0EC F956 1C31 X-PGP-Public-Key: http://freesbee.wheel.dk/~jesper/gpgkey.pub Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Jan 13, 2002 at 12:54:21PM -0800, Terry Lambert wrote: > Michal Mertl wrote: > > But anyway I continued on some work on STABLE (but believe lot of my work > > could be used on CURRENT after some modification) and get kernel and world > > building with 64 bit counters on network interfaces (struct if_data) > > and in protocols (struct ipstat, tcpstat, udpstat, icmpstat, igmpstat). > > Not to discount the value of this work, but... > > 1) It makes counting slower on 32 bit processors. > > 2) The values are not accessible from SNMP, which is > limited to 32 bit counters. SNMPv2 can do 64 bit counters ... > 3) While you could export these values as strings and > not numbers over SNMP, doing this would mean you > would need to use a MIB which was not a superset > of an RFC'ed MIB. One could also export them as 2 32 bit counters, one for the high 32 bits one for the low 32 bits, this is a common trick ... > So it seems to me that the utility of this on 32 bit machines > is incredibly limited (e.g. shell programs). > > PS: FWIW, I agree that these things should be 64 bits on > 64 bit architectures, even if they can only export > the low 32 bits for SNMP. /Jesper -- Jesper Skriver, jesper(at)skriver(dot)dk - CCIE #5456 Work: Network manager @ AS3292 (Tele Danmark DataNetworks) Private: FreeBSD committer @ AS2109 (A much smaller network ;-) One Unix to rule them all, One Resolver to find them, One IP to bring them all and in the zone to bind them. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 14:16: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id ADD0C37B41C for ; Sun, 13 Jan 2002 14:16:02 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id JAA12162; Mon, 14 Jan 2002 09:15:57 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37640) with ESMTP id <01KD2148DCLSVM3HF1@cim.alcatel.com.au>; Mon, 14 Jan 2002 09:15:55 +1100 Received: from cirb503493.alcatel.com.au (pj1592 [192.168.123.159]) by gsmx07.alcatel.com.au (8.11.6/8.11.6) with ESMTP id g0DMFqY25466; Mon, 14 Jan 2002 09:15:52 +1100 Received: (from jeremyp@localhost) by cirb503493.alcatel.com.au (8.11.6/8.11.6) id g0DLrUm38634; Mon, 14 Jan 2002 08:53:30 +1100 (EST envelope-from jeremyp) Date: Mon, 14 Jan 2002 08:53:30 +1100 From: Peter Jeremy Subject: Re: Request for review: getcontext, setcontext, etc In-reply-to: <20020110135217.M7984@elvis.mu.org>; from bright@mu.org on Thu, Jan 10, 2002 at 01:52:17PM -0800 To: Alfred Perlstein Cc: Kelly Yancey , Nate Williams , Terry Lambert , Daniel Eischen , Dan Eischen , Peter Wemm , Archie Cobbs , arch@FreeBSD.ORG Message-id: <20020114085330.B38605@cirb503493.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <15421.64170.308581.606485@caddis.yogotech.com> <20020110135217.M7984@elvis.mu.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Jan 10, 2002 at 01:52:17PM -0800, Alfred Perlstein wrote: >1) Is atomicity required? (looks like a "no") >2) Are states like FP usage trackable from userspace? > (looks like a "yes" with some kernel help) I gather you are thinking of a userland approach like: if (thread_has_used_FP) { save_FP_context(curthread); mark_FP_unused(); } save_nonFP_context(curthread); restore_nonFP_context(newthread); if (FP_is_used(newthread)) pass_FP_context_to_kernel(newthread); curthread = newthread; Since FP status is per-CPU, this needs atomicity to an extent: You must ensure that the active thread does not migrate between CPUs between the "thread_has_used_FP" test and following save. I agree this is unlikely, but it needs to be catered for. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 15:15: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from voi.aagh.net (pc1-hart4-0-cust168.mid.cable.ntl.com [62.254.84.168]) by hub.freebsd.org (Postfix) with ESMTP id CB8E137B402 for ; Sun, 13 Jan 2002 15:15:04 -0800 (PST) Received: from freaky by voi.aagh.net with local (Exim 3.34 #1) id 16Ptq7-0007x8-00 for arch@freebsd.org; Sun, 13 Jan 2002 23:14:59 +0000 Date: Sun, 13 Jan 2002 23:14:59 +0000 From: Thomas Hurst To: arch@freebsd.org Subject: Re: 64 bit counters again Message-ID: <20020113231459.GA30349@voi.aagh.net> Mail-Followup-To: arch@freebsd.org References: <3C41F3FD.4ECC8CD@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3C41F3FD.4ECC8CD@mindspring.com> User-Agent: Mutt/1.3.25i Organization: Not much. X-Operating-System: FreeBSD/4.5-PRERELEASE (i386) X-Uptime: 11:08PM up 24 days, 7:53, 3 users, load averages: 2.01, 2.04, 2.00 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Terry Lambert (tlambert2@mindspring.com) wrote: > So it seems to me that the utility of this on 32 bit machines is > incredibly limited (e.g. shell programs). The utility of the counters on anything but, perhaps, dialup, is practically zero, since they reset every 4.2GB - even my single user cable gateway exceeds that in a few days. Even if some things will break (how badly?), I'd rather have potentially useful counters over known useless ones. How about keeping them at 32 bit and also counting the number of times they've overflowed? User level stuff can then do potentially slow 64 bit ops if needed. -- Thomas 'Freaky' Hurst - freaky@aagh.net - http://www.aagh.net/ - It's better to burn out than to fade away. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 15:29:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 3D5A737B417 for ; Sun, 13 Jan 2002 15:29:41 -0800 (PST) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.12.1/8.12.1) id g0DNSTvN018698; Sun, 13 Jan 2002 18:28:29 -0500 (EST) Date: Sun, 13 Jan 2002 18:28:29 -0500 (EST) From: Daniel Eischen To: Peter Jeremy Cc: Bruce Evans , Terry Lambert , Peter Wemm , Alfred Perlstein , Kelly Yancey , Nate Williams , Archie Cobbs , arch@FreeBSD.ORG Subject: Re: Request for review: getcontext, setcontext, etc In-Reply-To: <20020114074238.S561@gsmx07.alcatel.com.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 14 Jan 2002, Peter Jeremy wrote: > On 2002-Jan-12 21:40:20 +1100, Bruce Evans wrote: > >On Sat, 12 Jan 2002, Terry Lambert wrote: > ... > >> Assuming a 64 bit data path, then we are talking a minimum of > >> 3 * 512/(64/8) * (16:1) or 3k (3076) clocks to save the damn FPU > >> state off to main memory (a store in a loop is 3 clocks ignoring > >> the setup and crap, right?). Add another 3k clocks to bring it > >> back. > >> > >> Best case, God loves us, and we spill and restore from L1 > >> without an IPI or an invalidation, and without starting the > >> thread on a CPU other than the one where it was suspended, and > >> all spills are to cacheable write-through pages. That's a 16 > >> times speed increase because we get to ignore the bus speed > >> differential, or 3 * 512/(65/8) * 2 = (6k/16) = 384 clocks. > > > >This seems to be off by a bit. Actual timing on an Athlon1600 > >overclocked a little gives the following times for some crtical > >parts of context switching for each iteration of instructions in > >a loop (not counting 2 cycles of loop overhead): > > > >pushal; popal: 9 cycles > >pushl %ds; popl %ds: 21 cycles > >fxsave; fxrstor: 105 cycles > >fnsave; frstor: 264 cycles > > I can think of a possible reason: The FPU knows when it has been used > vs just having executed fninit. In the latter case, all it needs to > save is "I've been initialised". Also the FPU architecture includes > "used" flags associated with each register - possibly the f*save > instructions don't flush unused registers. Do the above numbers > change when you push real data into the FP registers? > > Also, how expensive is a DNA trap? Would it be cheaper overall to > always load FPU context on a switch - this is more expensive for > processes that don't use FP, but saves a DNA trap per context switch > (assuming they use FP in that slice) for those that do. Or perhaps keep some sort of heuristic (per-process) on whether or not it needs to FPU? If you've got N traps per M quanta, perhaps you could always load the FPU context for that process. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 17:19: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from swan.prod.itd.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by hub.freebsd.org (Postfix) with ESMTP id 46EDE37B420; Sun, 13 Jan 2002 17:18:33 -0800 (PST) Received: from pool0611.cvx21-bradley.dialup.earthlink.net ([209.179.194.101] helo=mindspring.com) by swan.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Pvlg-0000f0-00; Sun, 13 Jan 2002 17:18:32 -0800 Message-ID: <3C4231E5.1B7FA359@mindspring.com> Date: Sun, 13 Jan 2002 17:18:29 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jesper Skriver Cc: Michal Mertl , arch@freebsd.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113223351.A42689@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jesper Skriver wrote: > > 2) The values are not accessible from SNMP, which is > > limited to 32 bit counters. > > SNMPv2 can do 64 bit counters ... Oops. Looks like you are right; it was a console program limitation. Never mind about that argument... Though I do think doing the calculation is unnecessarily high overhead for 32 bit processors... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 17:42:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by hub.freebsd.org (Postfix) with ESMTP id 61EBB37B404; Sun, 13 Jan 2002 17:42:10 -0800 (PST) Received: from isi.edu (n5lir8c92gyfhy11@b.postel.org [128.9.112.66]) by boreas.isi.edu (8.11.6/8.11.2) with ESMTP id g0E1g8N27150; Sun, 13 Jan 2002 17:42:08 -0800 (PST) Message-ID: <3C423770.5000308@isi.edu> Date: Sun, 13 Jan 2002 17:42:08 -0800 From: Lars Eggert User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.6) Gecko/20011203 X-Accept-Language: en, de MIME-Version: 1.0 To: Terry Lambert Cc: Jesper Skriver , Michal Mertl , arch@freebsd.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113223351.A42689@FreeBSD.org> <3C4231E5.1B7FA359@mindspring.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > Though I do think doing the calculation is unnecessarily > high overhead for 32 bit processors... Well, if machines are fast enough to generate enough data to overflow the counters quickly, the 64bit operations can't be that much of an overhead... ;-) But I agree that it should probably be made optional for embedded systems and such. Lars -- Lars Eggert Information Sciences Institute http://www.isi.edu/larse/ University of Southern California To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 17:49: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from swan.prod.itd.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by hub.freebsd.org (Postfix) with ESMTP id 1D06337B400 for ; Sun, 13 Jan 2002 17:49:02 -0800 (PST) Received: from pool0611.cvx21-bradley.dialup.earthlink.net ([209.179.194.101] helo=mindspring.com) by swan.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16PwFA-0007Bs-00; Sun, 13 Jan 2002 17:49:00 -0800 Message-ID: <3C42390A.F9E9F533@mindspring.com> Date: Sun, 13 Jan 2002 17:48:58 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Thomas Hurst Cc: arch@freebsd.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thomas Hurst wrote: > How about keeping them at 32 bit and also counting the number of times > they've overflowed? User level stuff can then do potentially slow 64 > bit ops if needed. And make it adjacent anyway? Maybe using a union to avoid byte/word order issues? This is much less expensive, I think, since the math doesn't go 64 bit, though it does cost another compare, per over just doing 32 bits. My personal opinion is that the utility of the counters is mostly in the examination of small deltas, or delta measurement over intervals (c.v. the recent "sar" discussion); is there a particular reason that making them more expensive to get 64 bits of accuracy is needed to fulfill? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 18:32:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 2F7C037B419 for ; Sun, 13 Jan 2002 18:32:44 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id NAA02626; Mon, 14 Jan 2002 13:32:02 +1100 Date: Mon, 14 Jan 2002 13:31:20 +1100 (EST) From: Bruce Evans X-X-Sender: To: Peter Jeremy Cc: Terry Lambert , Peter Wemm , Alfred Perlstein , Kelly Yancey , Nate Williams , Daniel Eischen , Dan Eischen , Archie Cobbs , Subject: Re: Request for review: getcontext, setcontext, etc In-Reply-To: <20020114074238.S561@gsmx07.alcatel.com.au> Message-ID: <20020114120026.S3794-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 14 Jan 2002, Peter Jeremy wrote: > On 2002-Jan-12 21:40:20 +1100, Bruce Evans wrote: > >This seems to be off by a bit. Actual timing on an Athlon1600 > >overclocked a little gives the following times for some crtical > >parts of context switching for each iteration of instructions in > >a loop (not counting 2 cycles of loop overhead): > > > >pushal; popal: 9 cycles > >pushl %ds; popl %ds: 21 cycles > >fxsave; fxrstor: 105 cycles > >fnsave; frstor: 264 cycles > > I can think of a possible reason: The FPU knows when it has been used > vs just having executed fninit. In the latter case, all it needs to > save is "I've been initialised". There are a lot of reasons. Peter Wemm mentioned some alignment issues and not doing an (in)convenient implicit fninit. The 512-byte SSE context is more than half "reserved", and this most of half is not written to by fxsave and presumably not read from by frstor (I didn't check the manual for this; I just fxsave'ed over 0xff bytes and checked what changed). > Also the FPU architecture includes > "used" flags associated with each register - possibly the f*save > instructions don't flush unused registers. Do the above numbers > change when you push real data into the FP registers? The manual seems to say that fxsave is clever about this, but it doesn't seem to be. Loading registers didn't make any difference to the times with the following register contents: all 1.0's 6 1.0's, one 0.0, one +Inf, and a masked DIVZ exception for creating the +Inf all xmm registers explicitly loaded with 0 OTOH, with the DIVZ unmasked (but never trapped for), fxsave/frstor takes 187 cycles. I think fnsave/frstor is normally slowed down by (unmasked only?) exceptions too. > Also, how expensive is a DNA trap? Would it be cheaper overall to I counted 200 kernel instructions for non-SMP in -current. Half of these are for recent pessimizations in -current: crhold() and crfree() and locking for these take about 100 instructions. I haven't measured the expense in real time. The basic (mostly unnecessary) overhead is the same as for pagefault traps. Mmap latency on this machine is 153nsec on the same machine according to lmbench2. > always load FPU context on a switch - this is more expensive for > processes that don't use FP, but saves a DNA trap per context switch > (assuming they use FP in that slice) for those that do. Not overall, since most timeslices don't use the FPU (at least for processes that I run :-). > To add some further numbers, in December 1999, I did some measurements > on FP switching by patching npx.c. This was on a PII-266 running then > -current. (The original e-mail was sent to -arch on Mon, 20 Dec 1999 > 07:34:06 +1100 in a thread titled "Concrete plans for ucontext/ > mcontext changes around 4.0" - I don't have the message-id available). > > ctxt DNA FP > swtch traps swtch > 1754982 281557 59753 build world and a few CVS operations [1] > 79044 18811 10341 gnuplot and xv in parallel [2] > 800 138 130 parallel FP-intensive progs [3]. > > In the above, `ctxt swtch' is the number of context switches counted > via vm.stats.sys.v_swtch. `DNA traps' is the number of device not > available traps registered and `FP swtch' is the number of DNA traps > where the FP context loaded is different to that saved on the > preceeding context switch. That's a lot more DNA traps than I would have expected for buildworld and a bit less than I would have expected for the others. I guess many of the ones for buildworld are for the FP in setjmp() for jumps that are never taken. 220000 extra FP context switches at 264 cycles each would increase my buildworld time by a whole 0.34 seconds or 0.025%. There may be more important things to optimize :-). Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 20: 1:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from ebb.errno.com (ebb.errno.com [66.127.85.87]) by hub.freebsd.org (Postfix) with ESMTP id 1E4AD37B405 for ; Sun, 13 Jan 2002 20:01:53 -0800 (PST) Received: from melange (melange.errno.com [66.127.85.82]) (authenticated bits=0) by ebb.errno.com (8.12.1/8.12.1) with ESMTP id g0E41ouX038517 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO); Sun, 13 Jan 2002 20:01:50 -0800 (PST)?g (envelope-from sam@errno.com)œ Message-ID: <1e6601c19cb0$41606ca0$5a557f42@errno.com> From: "Sam Leffler" To: "Mailing List FreeBSD Arch" , "Eric Masson" References: <867kqm1rb4.fsf@notbsdems.nantes.kisoft-services.com> Subject: Re: Cryptographic hardware support Date: Sun, 13 Jan 2002 20:02:15 -0800 Organization: Errno Consulting MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 X-DCC-errno-Metrics: ebb.errno.com 1006; IP=19 env_From=38 From=38 Subject=2 Message-ID=2 Body=2 Fuz1=2 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I'm working on this to support Hifn 7951 parts in Soekris net4501 machines. Initially I'm just porting the OpenBSD driver and crypto framework so I can accelerate IPsec and get access to the RNG. With luck I'll have something to test drive in a couple of weeks. Sam ----- Original Message ----- From: "Eric Masson" To: "Mailing List FreeBSD Arch" Sent: Sunday, January 13, 2002 2:25 AM Subject: Cryptographic hardware support > Hello, > > I've googled about FreeBSD crypto hardware support and didn't find any > whitepaper or draft about selected implementation. > > Is there any paper describing the implementation framework ? > > Has anyone considered the way OpenBSD did this integration (flaws, > strenghts...) ? > > Wouldn't it be interesting to share this type of support across the > bsd's (no duplicate of work for hardware support for example) ? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 20:32:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id 8CA4337B400 for ; Sun, 13 Jan 2002 20:32:04 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id PAA28403; Mon, 14 Jan 2002 15:31:54 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37641) with ESMTP id <01KD2E9PO4I8VFMH76@cim.alcatel.com.au>; Mon, 14 Jan 2002 15:32:11 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.6/8.11.6) id g0E4VnG26895; Mon, 14 Jan 2002 15:31:49 +1100 Content-return: prohibited Date: Mon, 14 Jan 2002 15:31:49 +1100 From: Peter Jeremy Subject: Re: Request for review: getcontext, setcontext, etc In-reply-to: <20020114120026.S3794-100000@gamplex.bde.org>; from bde@zeta.org.au on Mon, Jan 14, 2002 at 01:31:20PM +1100 To: Bruce Evans Cc: Terry Lambert , Peter Wemm , Alfred Perlstein , Kelly Yancey , Nate Williams , Daniel Eischen , Dan Eischen , Archie Cobbs , arch@FreeBSD.ORG Mail-Followup-To: Bruce Evans , Terry Lambert , Peter Wemm , Alfred Perlstein , Kelly Yancey , Nate Williams , Daniel Eischen , Dan Eischen , Archie Cobbs , arch@FreeBSD.ORG Message-id: <20020114153148.W561@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <20020114074238.S561@gsmx07.alcatel.com.au> <20020114120026.S3794-100000@gamplex.bde.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-Jan-14 13:31:20 +1100, Bruce Evans wrote: >> always load FPU context on a switch - this is more expensive for >> processes that don't use FP, but saves a DNA trap per context switch >> (assuming they use FP in that slice) for those that do. > >Not overall, since most timeslices don't use the FPU (at least for >processes that I run :-). My 2 year old figures suggest that even with FP-intensive progs, the majority of timeslices don't use the FPU. >> To add some further numbers, in December 1999, I did some measurements >> on FP switching by patching npx.c. This was on a PII-266 running then >> -current. (The original e-mail was sent to -arch on Mon, 20 Dec 1999 >> 07:34:06 +1100 in a thread titled "Concrete plans for ucontext/ >> mcontext changes around 4.0" - I don't have the message-id available). >> >> ctxt DNA FP >> swtch traps swtch >> 1754982 281557 59753 build world and a few CVS operations [1] >> 79044 18811 10341 gnuplot and xv in parallel [2] >> 800 138 130 parallel FP-intensive progs [3]. >> >> In the above, `ctxt swtch' is the number of context switches counted >> via vm.stats.sys.v_swtch. `DNA traps' is the number of device not >> available traps registered and `FP swtch' is the number of DNA traps >> where the FP context loaded is different to that saved on the >> preceeding context switch. > >That's a lot more DNA traps than I would have expected for buildworld >and a bit less than I would have expected for the others. I guess many >of the ones for buildworld are for the FP in setjmp() for jumps that >are never taken. It's also possible that gcc does some FP operations even when compiling integer-only code. >220000 extra FP context switches at 264 cycles each would increase my >buildworld time by a whole 0.34 seconds or 0.025%. There may be more >important things to optimize :-). Except that removing the lazy FPU switching would translate 280,000 DNA traps into 1,755,000 f*rstor's. Though you could probably cut this number down by changing the FPU switching code to always do an f*rstor if the process ever uses FP. (As someone else suggested). If anyone is interested, my original patches are below. Based on a quick look, they seem to be still valid (though there will be some fuzz on -CURRENT due to the added cpu_critical_enter() call). Index: npx.c =================================================================== RCS file: /home/peter/cvs/src/sys/i386/isa/npx.c,v retrieving revision 1.78 diff -u -r1.78 npx.c --- npx.c 1999/09/21 10:51:47 1.78 +++ npx.c 1999/12/17 09:53:02 @@ -779,6 +779,15 @@ } } +static int fp_dna; /* number of DNA traps */ +static int fp_swtch; /* Number of real FP context switches */ +static struct proc *fpuproc; /* Last proc to use FPU */ + +SYSCTL_INT(_hw, OID_AUTO, fp_dna, CTLFLAG_RW, &fp_dna, 0, + "Number of NPX DNA traps"); +SYSCTL_INT(_hw, OID_AUTO, fp_swtch, CTLFLAG_RW, &fp_swtch, 0, + "Number of NPX context switches"); + /* * Implement device not available (DNA) exception * @@ -797,6 +806,11 @@ panic("npxdna"); } stop_emulating(); + fp_dna++; + if (curproc != fpuproc) { + fpuproc = curproc; + fp_swtch++; + } /* * Record new context early in case frstor causes an IRQ13. */ Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 21: 7:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id B1A5B37B402 for ; Sun, 13 Jan 2002 21:07:14 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id QAA22610; Mon, 14 Jan 2002 16:06:34 +1100 Date: Mon, 14 Jan 2002 16:07:37 +1100 (EST) From: Bruce Evans X-X-Sender: To: Peter Wemm Cc: Terry Lambert , Alfred Perlstein , Kelly Yancey , Nate Williams , Daniel Eischen , Dan Eischen , Archie Cobbs , Subject: Re: Request for review: getcontext, setcontext, etc In-Reply-To: <20020112203429.EE98738CC@overcee.netplex.com.au> Message-ID: <20020114151640.F4268-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 12 Jan 2002, Peter Wemm wrote: > Bruce Evans wrote: > > I agree that fnsave/frstor are still incredibly expensive if the > > above times are correct. fxsave/fxrstor is only credibly expensive. > > However, the overheads for fnsave/frstor are small compared with > > the overheads for the !*#*$% segment registers. We switch 3 segment > > registers explicitly and 2 implicitly on every switch to the kernel. > > According to the above, this has the same overhead as 1 fxsave/frstor. > > It gets done much more often than context switches. I hoped to get > > rid of the 2 expicit segment register switches, but couldn't keep > > up with the forces of bloat that added a 3rd. Now I don't notice > > this bloat unless I count cycles and forget that a billion of them > > is a lot :-). > > Heh. That reminds me, I need to talk over some IPI vector tweaks > with you. I had forgotten that segment register operations were so bad. I barely understand IPI. > Hmm. What are they again? I see %ds, %es and %fs. I assume the two > implicit ones were %cs and %ss. Which had you hoped to remove? What *is* > %es used for anyway? I hoped to remove switches of %ds and %es. My idea for removing the switches is: use the same %ds and %es in userland as in the kernel, unless the process has an extended LDT with problematic descriptors in it. This works for the usual descriptors because they can all cover all of memory, and the kernel part is protected from user access at the page level. Userland can still load the unusual value 0, which will cause traps in the kernel; these traps can be caught and the segment registers fixed up. Segment registers with an unusual limit (which need only occur in an extended LDT) can be switched lazily too, but segment registers with an unusual base would cause invalid references without a trap if they were used in the kernel. %es is used mainly for string operations. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 21:40:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id B78C937B402 for ; Sun, 13 Jan 2002 21:40:46 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id QAA26377; Mon, 14 Jan 2002 16:40:14 +1100 Date: Mon, 14 Jan 2002 16:41:17 +1100 (EST) From: Bruce Evans X-X-Sender: To: Peter Jeremy Cc: Terry Lambert , Peter Wemm , Alfred Perlstein , Kelly Yancey , Nate Williams , Daniel Eischen , Dan Eischen , Archie Cobbs , Subject: Re: Request for review: getcontext, setcontext, etc In-Reply-To: <20020114153148.W561@gsmx07.alcatel.com.au> Message-ID: <20020114162150.S4436-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 14 Jan 2002, Peter Jeremy wrote: > On 2002-Jan-14 13:31:20 +1100, Bruce Evans wrote: > >>... > >> ctxt DNA FP > >> swtch traps swtch > >> 1754982 281557 59753 build world and a few CVS operations [1] > >> 79044 18811 10341 gnuplot and xv in parallel [2] > >> 800 138 130 parallel FP-intensive progs [3]. > >>... > >... > >220000 extra FP context switches at 264 cycles each would increase my > >buildworld time by a whole 0.34 seconds or 0.025%. There may be more > >important things to optimize :-). > > Except that removing the lazy FPU switching would translate 280,000 > DNA traps into 1,755,000 f*rstor's. Though you could probably cut > this number down by changing the FPU switching code to always do an > f*rstor if the process ever uses FP. (As someone else suggested). Increases in CPU speeds have cut the numbers down to not many more than 280000. The following times are for buildworld with many local optimizations and no kernels or modules: Celeron366 @ 5.5 * 95 MHz -------------------------------------------------------------- >>> elf make world completed on Mon Nov 19 15:10:42 EST 2001 (started Mon Nov 19 14:00:22 EST 2001) -------------------------------------------------------------- 4219.83 real 3253.04 user 667.64 sys ... 365764 voluntary context switches 1074456 involuntary context switches Athlon1600 & 10.5 * 146 MHz -------------------------------------------------------------- >>> elf make world completed on Fri Jan 4 00:00:15 EST 2002 (started on Thu Jan 3 23:37:04 EST 2002) -------------------------------------------------------------- 1390.18 real 913.56 user 232.63 sys ... 346041 voluntary context switches 423794 involuntary context switches This is under my version -current where there are a lot more context switches for interrupt tasks, especially in my version where there are switches for clock interrupts. 128+100 context switches for clock interrupts is actually quite a lot from one viewpoint. 4219 seconds worth of these accounts for 916332 of the 1074456 involuntary context switches on the old machine. However, it is almost zero from a practical viewpoint. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jan 13 23: 5:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 4A80937B402 for ; Sun, 13 Jan 2002 23:05:51 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0E75nle083548; Mon, 14 Jan 2002 08:05:50 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0E75noR083545; Mon, 14 Jan 2002 08:05:49 +0100 (CET) Date: Mon, 14 Jan 2002 08:05:49 +0100 (CET) From: Michal Mertl To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Re: 64 bit counters again In-Reply-To: <3C41F3FD.4ECC8CD@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 13 Jan 2002, Terry Lambert wrote: > Michal Mertl wrote: > > But anyway I continued on some work on STABLE (but believe lot of my work > > could be used on CURRENT after some modification) and get kernel and world > > building with 64 bit counters on network interfaces (struct if_data) > > and in protocols (struct ipstat, tcpstat, udpstat, icmpstat, igmpstat). > > Not to discount the value of this work, but... > > 1) It makes counting slower on 32 bit processors. How much slower? 64 bit addition on i386 is only 2 simple instructions instead of 1 for 32 bit. > > 2) The values are not accessible from SNMP, which is > limited to 32 bit counters. > Not true. You do better check things before explaining them to anyone. > 3) While you could export these values as strings and > not numbers over SNMP, doing this would mean you > would need to use a MIB which was not a superset > of an RFC'ed MIB. As stated before there is 64bit counter type in SNMP. On cisco I use it every other day. > So it seems to me that the utility of this on 32 bit machines > is incredibly limited (e.g. shell programs). > I don't see that much utility either, only to have right values. But I got at least one email asking for the patch because the guy uses the counters. He said they overflow in several minutes for him so are really useless. > PS: FWIW, I agree that these things should be 64 bits on > 64 bit architectures, even if they can only export > the low 32 bits for SNMP. > I think some counters are already 64 bit on 64 bit architectures, because they are defined as u_long which is (I believe) 64 bit. The most importatnt thing is that I don't believe the operations are expensive (when not using locked atomic ops at least). -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 2:43:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 7DD3437B41A for ; Mon, 14 Jan 2002 02:43:11 -0800 (PST) Received: from pool0023.cvx22-bradley.dialup.earthlink.net ([209.179.198.23] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Q4a4-0007Td-00; Mon, 14 Jan 2002 02:43:08 -0800 Message-ID: <3C42B63A.24CDBC33@mindspring.com> Date: Mon, 14 Jan 2002 02:43:06 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Michal Mertl Cc: arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Michal Mertl wrote: > > 1) It makes counting slower on 32 bit processors. > > How much slower? 64 bit addition on i386 is only 2 simple instructions > instead of 1 for 32 bit. Plus the carry, plus the atomicity guarantees that you have to be able to make so that the value is not examined if a context switch is forced between those instructions, or an interrupt (or higher priority interrupt) occurs. > > 2) The values are not accessible from SNMP, which is > > limited to 32 bit counters. > > Not true. You do better check things before explaining them to anyone. I already commented that it was a limitation of my SNMP console, more than a day before you wrote this. > > So it seems to me that the utility of this on 32 bit machines > > is incredibly limited (e.g. shell programs). > > I don't see that much utility either, only to have right values. But I got > at least one email asking for the patch because the guy uses the counters. > He said they overflow in several minutes for him so are really useless. I would argue that he should be counting packets instead of bytes, or packets per second, or something other than what he is counting, if he is overflowing that quickly. A 64 bit counter gives us 8000 years, if it overflows in a minute, and that coupled with Moore's law applied to network speeds gives us about 2 years before the 64 bit counters are useless for the same reasons. I will tell you that at ClickArray, the logging overhead, which is in the same general category as these counters ("administrative eye candy for humans that has no bearing on the functionality of the technology itself, when it comes to whether or not you can push packets) grew into over 30% of the overall overhead for the product, even after a zero system call boundary transition clock function and other nifty optimizations. I guess if you are a telephone company, hell bent on charging per packet, you're willing to take 30% of your actual real capability, and sink it into accounting instead of getting work done. > > PS: FWIW, I agree that these things should be 64 bits on > > 64 bit architectures, even if they can only export > > the low 32 bits for SNMP. > > I think some counters are already 64 bit on 64 bit architectures, > because they are defined as u_long which is (I believe) 64 bit. The blythe thing to say at this point is "use 64 bit hardware"... 8-) 8-). Not a good answer, I know. > The most importatnt thing is that I don't believe the operations > are expensive (when not using locked atomic ops at least). I think the operations would have to be locked. A received bad packet checksum counter (for example) would have to be accessed at interrupt in the driver. Locking and atomicity are problematic, and are guaranteed to add overhead (better to keep it per interface [==per device instance], and total on demand, actually). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 5: 7:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from a96180.upc-a.chello.nl (a96180.upc-a.chello.nl [62.163.96.180]) by hub.freebsd.org (Postfix) with ESMTP id B2F7437B419; Mon, 14 Jan 2002 05:07:15 -0800 (PST) Received: by a96180.upc-a.chello.nl (Postfix, from userid 1001) id A5A48216F; Mon, 14 Jan 2002 14:07:13 +0100 (CET) Date: Mon, 14 Jan 2002 14:07:13 +0100 From: Jeroen Ruigrok/asmodai To: Eric Melville Cc: Thierry Herbelot , arch@FreeBSD.ORG Subject: Re: xfree4 by default? Message-ID: <20020114130713.GG61638@daemon.ninth-circle.org> References: <20011231183928.F37696@bsd.havk.org> <78699.1009878927@winston.freebsd.org> <200201021425.g02EPks11288@dungeon.home> <3C334C5A.A5E56055@mindspring.com> <20020102125455.A44310@FreeBSD.org> <3C3375DD.36947896@herbelot.com> <20020102134820.B44310@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020102134820.B44310@FreeBSD.org> User-Agent: Mutt/1.3.24i Organisation: Ninth Circle Enterprises Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG -On [20020102 23:00], Eric Melville (eric@FreeBSD.org) wrote: >I seem to talk about it on the binup mailing list, mostly because shipping >everything in packages is the first logical step to really slick binary >updates. Soon I'll have a summary and such for the wider audience. Did you look at Mark Murray's emails in 2001 about this issue? He already presented ideas and polled for more ideas/feedback. -- Jeroen Ruigrok van der Werven / asmodai / Kita no Mono / xMach coreteam asmodai@[wxs.nl|xmach.org], finger asmodai@ninth-circle.org http://www.softweyr.com/asmodai/ We must all hang together, else we shall all hang separately... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 6:18:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from thehousleys.net (frenchknot.ne.mediaone.net [66.31.234.148]) by hub.freebsd.org (Postfix) with ESMTP id 45EE637B404; Mon, 14 Jan 2002 06:18:11 -0800 (PST) Received: (from root@localhost) by thehousleys.net (8.11.6/8.11.2) id g0EEI3C29056; Mon, 14 Jan 2002 09:18:03 -0500 (EST) (envelope-from jeh@FreeBSD.org) Received: from FreeBSD.org (baby.int.thehousleys.net [192.168.0.125]) (authenticated) by thehousleys.net (8.11.6/8.11.6) with ESMTP id g0EEI1N29048; Mon, 14 Jan 2002 09:18:01 -0500 (EST) (envelope-from jeh@FreeBSD.org) Message-ID: <3C42E899.CB21BD0A@FreeBSD.org> Date: Mon, 14 Jan 2002 09:18:01 -0500 From: "James E. Housley" Organization: FreeBSD X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > My personal opinion is that the utility of the counters is > mostly in the examination of small deltas, or delta measurement > over intervals (c.v. the recent "sar" discussion); is there a > particular reason that making them more expensive to get 64 > bits of accuracy is needed to fulfill? > Right now I am trying to do network usage on the overflows the 32 bit counters in 60 120 seconds. That means the netstat values are useless, mrtg and other programs that poll for results every 5 minutes or so are useless. If I want to get good data I have to get the counts every 10 seconds or so to get reasonable results. Jim -- /"\ ASCII Ribbon Campaign . \ / - NO HTML/RTF in e-mail . X - NO Word docs in e-mail . / \ ----------------------------------------------------------------- jeh@FreeBSD.org http://www.FreeBSD.org The Power to Serve jim@TheHousleys.Net http://www.TheHousleys.net jhousley@SimTel.Net http://www.SimTel.Net --------------------------------------------------------------------- "It's a damn poor mind that can only think of one way to spell a word." -- Andrew Jackson To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 8: 1:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from technokratis.com (modemcable099.144-201-24.mtl.mc.videotron.ca [24.201.144.99]) by hub.freebsd.org (Postfix) with ESMTP id B23D837B400; Mon, 14 Jan 2002 07:57:13 -0800 (PST) Received: (from bmilekic@localhost) by technokratis.com (8.11.4/8.11.3) id g0EFx3K24691; Mon, 14 Jan 2002 10:59:03 -0500 (EST) (envelope-from bmilekic) Date: Mon, 14 Jan 2002 10:58:59 -0500 From: Bosko Milekic To: "James E. Housley" Cc: Terry Lambert , Thomas Hurst , arch@FreeBSD.ORG Subject: Re: 64 bit counters again Message-ID: <20020114105859.A24635@technokratis.com> References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3C42E899.CB21BD0A@FreeBSD.org>; from jeh@FreeBSD.ORG on Mon, Jan 14, 2002 at 09:18:01AM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Jan 14, 2002 at 09:18:01AM -0500, James E. Housley wrote: > Terry Lambert wrote: > > > > My personal opinion is that the utility of the counters is > > mostly in the examination of small deltas, or delta measurement > > over intervals (c.v. the recent "sar" discussion); is there a > > particular reason that making them more expensive to get 64 > > bits of accuracy is needed to fulfill? > > > > Right now I am trying to do network usage on the overflows the 32 bit > counters in 60 120 seconds. That means the netstat values are useless, > mrtg and other programs that poll for results every 5 minutes or so are > useless. If I want to get good data I have to get the counts every 10 > seconds or so to get reasonable results. You know, if you have a counter overflowing in 60 to 120 seconds, then perhaps what it's counting should not be counted to begin with. I stumbled upon several statistics issues while writing mb_alloc and I tried to the best of my abilities to group the manipulation of the counters under some common already existing lock. In some odd cases, this was impossible. But I refused to introduce a bus-locked instruction or, worse, a whole new lock just to deal with the statistics. It's too much overhead for mere statistics and, in the latter case, it's even very bug-prone. To be honest, after actually examining more closely what the counters collected statistics on, I realized that they were pretty much totally redundant and am considering removing those few problematic ones some time in the future. > Jim > -- > /"\ ASCII Ribbon Campaign . > \ / - NO HTML/RTF in e-mail . > X - NO Word docs in e-mail . > / \ ----------------------------------------------------------------- > jeh@FreeBSD.org http://www.FreeBSD.org The Power to Serve > jim@TheHousleys.Net http://www.TheHousleys.net > jhousley@SimTel.Net http://www.SimTel.Net > --------------------------------------------------------------------- > "It's a damn poor mind that can only think of one way to > spell a word." > -- Andrew Jackson Regards, -- Bosko Milekic bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 8:24:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from thehousleys.net (frenchknot.ne.mediaone.net [66.31.234.148]) by hub.freebsd.org (Postfix) with ESMTP id D0DA637B432; Mon, 14 Jan 2002 08:23:10 -0800 (PST) Received: (from root@localhost) by thehousleys.net (8.11.6/8.11.2) id g0EGN4p00717; Mon, 14 Jan 2002 11:23:04 -0500 (EST) (envelope-from jeh@FreeBSD.org) Received: from FreeBSD.org (baby.int.thehousleys.net [192.168.0.125]) (authenticated) by thehousleys.net (8.11.6/8.11.6) with ESMTP id g0EGN1g00708; Mon, 14 Jan 2002 11:23:01 -0500 (EST) (envelope-from jeh@FreeBSD.org) Message-ID: <3C4305E5.65BB32A6@FreeBSD.org> Date: Mon, 14 Jan 2002 11:23:01 -0500 From: "James E. Housley" Organization: FreeBSD X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Bosko Milekic Cc: Terry Lambert , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bosko Milekic wrote: > > On Mon, Jan 14, 2002 at 09:18:01AM -0500, James E. Housley wrote: > > Terry Lambert wrote: > > > > > > My personal opinion is that the utility of the counters is > > > mostly in the examination of small deltas, or delta measurement > > > over intervals (c.v. the recent "sar" discussion); is there a > > > particular reason that making them more expensive to get 64 > > > bits of accuracy is needed to fulfill? > > > > > > > Right now I am trying to do network usage on the overflows the 32 bit > > counters in 60 120 seconds. That means the netstat values are useless, > > mrtg and other programs that poll for results every 5 minutes or so are > > useless. If I want to get good data I have to get the counts every 10 > > seconds or so to get reasonable results. > > You know, if you have a counter overflowing in 60 to 120 seconds, > then perhaps what it's counting should not be counted to begin with. I am just trying to count bytes in and out, too keep track of usage and head off a large overage and a larger bill then necessary. Counting packets is worthless. But just do the math. With a GigE NIC, at what data rate do you start overflowing the counters too quickly. I suppose there is another possibility, that the ti GigE driver is counting the data multiple times. But I don't think so, because at 200Mbits/sec the counter should overflow in 172 seconds. And this machine is easily doing this most of the day. > > I stumbled upon several statistics issues while writing mb_alloc and > I tried to the best of my abilities to group the manipulation of the > counters under some common already existing lock. In some odd cases, > this was impossible. But I refused to introduce a bus-locked instruction > or, worse, a whole new lock just to deal with the statistics. It's > too much overhead for mere statistics and, in the latter case, it's > even very bug-prone. To be honest, after actually examining more > closely what the counters collected statistics on, I realized that they > were pretty much totally redundant and am considering removing those > few problematic ones some time in the future. > That all sounds reasonable. And it make sense to move the counters under existing locks. But, 32-bit machines are going to be around for awhile longer and fast network connections are going to get faster and more common. Maybe the counters should be completely removed from the 32-bit arch.s since they give such misleading results and only have them on the 64-bit machines. That way no one will be confused by the data. Of course I am not completely serious about removing the counters, but it is not hard to make them very wrong. Jim -- /"\ ASCII Ribbon Campaign . \ / - NO HTML/RTF in e-mail . X - NO Word docs in e-mail . / \ ----------------------------------------------------------------- jeh@FreeBSD.org http://www.FreeBSD.org The Power to Serve jim@TheHousleys.Net http://www.TheHousleys.net jhousley@SimTel.Net http://www.SimTel.Net --------------------------------------------------------------------- PC hardware is the ductape of the computer inustry... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 8:47:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from technokratis.com (modemcable099.144-201-24.mtl.mc.videotron.ca [24.201.144.99]) by hub.freebsd.org (Postfix) with ESMTP id AA5EC37B405; Mon, 14 Jan 2002 08:47:20 -0800 (PST) Received: (from bmilekic@localhost) by technokratis.com (8.11.4/8.11.3) id g0EGnBj25064; Mon, 14 Jan 2002 11:49:11 -0500 (EST) (envelope-from bmilekic) Date: Mon, 14 Jan 2002 11:49:11 -0500 From: Bosko Milekic To: "James E. Housley" Cc: Terry Lambert , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again Message-ID: <20020114114911.A24990@technokratis.com> References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3C4305E5.65BB32A6@FreeBSD.org>; from jeh@FreeBSD.org on Mon, Jan 14, 2002 at 11:23:01AM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Jan 14, 2002 at 11:23:01AM -0500, James E. Housley wrote: > Bosko Milekic wrote: > > > Right now I am trying to do network usage on the overflows the 32 bit > > > counters in 60 120 seconds. That means the netstat values are useless, > > > mrtg and other programs that poll for results every 5 minutes or so are > > > useless. If I want to get good data I have to get the counts every 10 > > > seconds or so to get reasonable results. > > > > You know, if you have a counter overflowing in 60 to 120 seconds, > > then perhaps what it's counting should not be counted to begin with. > > I am just trying to count bytes in and out, too keep track of usage and > head off a large overage and a larger bill then necessary. Counting > packets is worthless. But just do the math. With a GigE NIC, at what > data rate do you start overflowing the counters too quickly. I suppose > there is another possibility, that the ti GigE driver is counting the > data multiple times. But I don't think so, because at 200Mbits/sec the > counter should overflow in 172 seconds. And this machine is easily > doing this most of the day. You're going to run into the same problem with a 64-bit counter, as Terry pointed out. You're just going to end up moving the time it takes from 172 seconds to some other, slightly longer, although equally unreasonable, time. Instead of counting bytes, perhaps you should count K or M Bytes. Just take the delta you're about to add to your counter and divide it by something reasonable before adding it. If you are really bent on getting accurate results, maintain a separate counter and sum remainders in it. I'm sure you'll get slightly more accurate data while not changing all counters to be 64-bits wide on non-64 bit platforms (i.e., you'll keep the added execution time local to that specific code you're working on). [...] > That all sounds reasonable. And it make sense to move the counters > under existing locks. But, 32-bit machines are going to be around for > awhile longer and fast network connections are going to get faster and > more common. Maybe the counters should be completely removed from the > 32-bit arch.s since they give such misleading results and only have them > on the 64-bit machines. That way no one will be confused by the data. > > Of course I am not completely serious about removing the counters, but > it is not hard to make them very wrong. I still think that this sort of data should not be counted, but if it should, having a 64-bit counter won't fix your problem, ultimately; besides for allowing you to run mrtg at longer intervals (and that's no solution; in fact, you can do that even with 32-bit counters as long as you had a second daemon cook your mrtg data files for you by having the daemon execute much more often and just perform sysctls and produce deltas. It could then flush the deltas to the data file every 5 minutes or so, and you can have mrtg run at every 5 minutes, 30 seconds so regardless, you are just stuck with the same problem, ultimately, even with 64-bit counters in this case). > Jim > -- > /"\ ASCII Ribbon Campaign . > \ / - NO HTML/RTF in e-mail . > X - NO Word docs in e-mail . > / \ ----------------------------------------------------------------- > jeh@FreeBSD.org http://www.FreeBSD.org The Power to Serve > jim@TheHousleys.Net http://www.TheHousleys.net > jhousley@SimTel.Net http://www.SimTel.Net > --------------------------------------------------------------------- > PC hardware is the ductape of the computer inustry... -- Bosko Milekic bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 9:42:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from math.teaser.net (math.teaser.net [213.91.2.4]) by hub.freebsd.org (Postfix) with ESMTP id 6D46A37B404 for ; Mon, 14 Jan 2002 09:42:35 -0800 (PST) Received: from notbsdems.nantes.kisoft-services.com (nantes.kisoft-services.com [193.56.60.243]) by math.teaser.net (Postfix) with ESMTP id 043AC6C813; Mon, 14 Jan 2002 18:42:34 +0100 (CET) Received: by notbsdems.nantes.kisoft-services.com (Postfix, from userid 1001) id CB6C7E6C8F; Mon, 14 Jan 2002 18:42:39 +0100 (CET) To: "Sam Leffler" Cc: "Mailing List FreeBSD Arch" Subject: Re: Cryptographic hardware support References: <867kqm1rb4.fsf@notbsdems.nantes.kisoft-services.com> <1e6601c19cb0$41606ca0$5a557f42@errno.com> From: Eric Masson In-Reply-To: <1e6601c19cb0$41606ca0$5a557f42@errno.com> ("Sam Leffler"'s message of "Sun, 13 Jan 2002 20:02:15 -0800") X-Operating-System: FreeBSD 4.5-PRERELEASE i386 Date: Mon, 14 Jan 2002 18:42:38 +0100 Message-ID: <86zo3gq169.fsf@notbsdems.nantes.kisoft-services.com> Lines: 26 User-Agent: Gnus/5.090005 (Oort Gnus v0.05) XEmacs/21.1 (Cuyahoga Valley, i386--freebsd) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "Sam" == Sam Leffler writes: Sam> I'm working on this to support Hifn 7951 parts in Soekris net4501 Sam> machines. Great, Soekris pci module is the main reason of this inquiry, it's damned cheap compared to the average price of cryto devices. Sam> Initially I'm just porting the OpenBSD driver and crypto framework Sam> so I can accelerate IPsec and get access to the RNG. So you need to patch the ip stack to defer all crypto computations to the crypto subsystem, don't you ? Sam> With luck I'll have something to test drive in a couple of weeks. Do you work on current or stable ? Thanks for your answer. Eric Masson -- Je sais pas. Mais un très vieux proverbe macintoshien dit : Quand BowelsOfTheMemoryMgr dans Macsbug apparoit, redémarer tu dois. -+- AG in Guide du Macounet Pervers : ...à la mode, à la modeuu... -+- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 9:59:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from root.com (root.com [209.102.106.178]) by hub.freebsd.org (Postfix) with ESMTP id 8B15B37B404; Mon, 14 Jan 2002 09:59:54 -0800 (PST) Received: (from dg@localhost) by root.com (8.11.2/8.11.2) id g0EHlcp09025; Mon, 14 Jan 2002 09:47:38 -0800 (PST) (envelope-from dg) Date: Mon, 14 Jan 2002 09:47:38 -0800 From: David Greenman To: Bosko Milekic Cc: "James E. Housley" , Terry Lambert , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again Message-ID: <20020114094738.A8955@nexus.root.com> References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020114114911.A24990@technokratis.com>; from bmilekic@technokratis.com on Mon, Jan 14, 2002 at 11:49:11AM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >> I am just trying to count bytes in and out, too keep track of usage and >> head off a large overage and a larger bill then necessary. Counting >> packets is worthless. But just do the math. With a GigE NIC, at what >> data rate do you start overflowing the counters too quickly. I suppose >> there is another possibility, that the ti GigE driver is counting the >> data multiple times. But I don't think so, because at 200Mbits/sec the >> counter should overflow in 172 seconds. And this machine is easily >> doing this most of the day. > > You're going to run into the same problem with a 64-bit counter, as >Terry pointed out. You're just going to end up moving the time it >takes from 172 seconds to some other, slightly longer, although equally >unreasonable, time. Instead of counting bytes, perhaps you should count >K or M Bytes. Just take the delta you're about to add to your counter >and divide it by something reasonable before adding it. If you are >really bent on getting accurate results, maintain a separate counter >and sum remainders in it. I'm sure you'll get slightly more accurate >data while not changing all counters to be 64-bits wide on non-64 bit >platforms (i.e., you'll keep the added execution time local to that >specific code you're working on). I think counting bytes with a 64bit counter is a perfectly reasonable thing to do. I don't think this means that we need to update ALL of the counters in the network stack to be 64bits, just the ones that are useful to network admins (which is essentially just bytes in/out and packets in/out). All the rest can stay 32bit. This isn't that much overhead to add and the results are very useful, despite what you and Terry would have some people believe. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 10:10:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from swan.prod.itd.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by hub.freebsd.org (Postfix) with ESMTP id A974C37B445; Mon, 14 Jan 2002 10:09:48 -0800 (PST) Received: from pool0303.cvx21-bradley.dialup.earthlink.net ([209.179.193.48] helo=mindspring.com) by swan.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QBYF-0003Nm-00; Mon, 14 Jan 2002 10:09:44 -0800 Message-ID: <3C431EE5.1CFF557B@mindspring.com> Date: Mon, 14 Jan 2002 10:09:41 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "James E. Housley" Cc: Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "James E. Housley" wrote: > Right now I am trying to do network usage on the overflows the 32 bit > counters in 60 120 seconds. That means the netstat values are useless, > mrtg and other programs that poll for results every 5 minutes or so are > useless. If I want to get good data I have to get the counts every 10 > seconds or so to get reasonable results. Actually, I was thinking that if what you are counting overflowed in such a short period of time, then you are counting the wrong thing. Instead of bytes, try counting K or frames, instead of frames, count packets, etc.. You still haven't told us exactly what counter is overflowing... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 10:18: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from root.com (root.com [209.102.106.178]) by hub.freebsd.org (Postfix) with ESMTP id 741E537B402; Mon, 14 Jan 2002 10:18:02 -0800 (PST) Received: (from dg@localhost) by root.com (8.11.2/8.11.2) id g0EI6Xf09120; Mon, 14 Jan 2002 10:06:33 -0800 (PST) (envelope-from dg) Date: Mon, 14 Jan 2002 10:06:33 -0800 From: David Greenman To: Terry Lambert Cc: "James E. Housley" , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again Message-ID: <20020114100633.B8955@nexus.root.com> References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3C431EE5.1CFF557B@mindspring.com>; from tlambert2@mindspring.com on Mon, Jan 14, 2002 at 10:09:41AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >"James E. Housley" wrote: >> Right now I am trying to do network usage on the overflows the 32 bit >> counters in 60 120 seconds. That means the netstat values are useless, >> mrtg and other programs that poll for results every 5 minutes or so are >> useless. If I want to get good data I have to get the counts every 10 >> seconds or so to get reasonable results. > >Actually, I was thinking that if what you are counting overflowed >in such a short period of time, then you are counting the wrong >thing. > >Instead of bytes, try counting K or frames, instead of frames, >count packets, etc.. Kbytes doesn't have enough resolution to provide accurate results since the packets themselves are not integer multiples of Kbytes. >You still haven't told us exactly what counter is overflowing... He's just trying to do what every network admin is trying to do: measure link utilization (mostly bits per second) so that he can make intelligent decisions about capacity, use, and associated costs. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 10:30:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from swan.prod.itd.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by hub.freebsd.org (Postfix) with ESMTP id EC49F37B416; Mon, 14 Jan 2002 10:30:49 -0800 (PST) Received: from pool0303.cvx21-bradley.dialup.earthlink.net ([209.179.193.48] helo=mindspring.com) by swan.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QBsT-0003Nl-00; Mon, 14 Jan 2002 10:30:37 -0800 Message-ID: <3C4323CA.13461AA1@mindspring.com> Date: Mon, 14 Jan 2002 10:30:34 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bosko Milekic Cc: "James E. Housley" , Thomas Hurst , arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bosko Milekic wrote: > You know, if you have a counter overflowing in 60 to 120 seconds, > then perhaps what it's counting should not be counted to begin with. > > I stumbled upon several statistics issues while writing mb_alloc and > I tried to the best of my abilities to group the manipulation of the > counters under some common already existing lock. In some odd cases, > this was impossible. But I refused to introduce a bus-locked instruction > or, worse, a whole new lock just to deal with the statistics. It's > too much overhead for mere statistics and, in the latter case, it's > even very bug-prone. To be honest, after actually examining more > closely what the counters collected statistics on, I realized that they > were pretty much totally redundant and am considering removing those > few problematic ones some time in the future. Exactly. I realize that some people's revenue model's depend on this level of statistic keeping, but I have no problem making *only* them pay for that level of detail, instead of everyone. "B-1!" ... "Hey! You sank my business model!" -- Alfred Perlstein -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 10:34:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id 4F0FB37B402 for ; Mon, 14 Jan 2002 10:34:22 -0800 (PST) Received: from peter3.wemm.org ([12.232.27.13]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020114183421.LIND10951.rwcrmhc51.attbi.com@peter3.wemm.org> for ; Mon, 14 Jan 2002 18:34:21 +0000 Received: from overcee.wemm.org (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id g0EIYLs80472 for ; Mon, 14 Jan 2002 10:34:21 -0800 (PST) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.wemm.org (Postfix) with ESMTP id F37E0380A; Mon, 14 Jan 2002 10:34:20 -0800 (PST) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Bruce Evans Cc: arch@FreeBSD.ORG Subject: Re: Request for review: getcontext, setcontext, etc In-Reply-To: <20020114120026.S3794-100000@gamplex.bde.org> Date: Mon, 14 Jan 2002 10:34:20 -0800 From: Peter Wemm Message-Id: <20020114183420.F37E0380A@overcee.wemm.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bruce Evans wrote: > 220000 extra FP context switches at 264 cycles each would increase my > buildworld time by a whole 0.34 seconds or 0.025%. There may be more > important things to optimize :-). Hmm. Sounds like I should do some work more on the IPI TLB shootdown stuff to finish it off. That speeds up a test kernel compile (compiling entirely from memory) by 2% to 2.5%. on one of my SMP boxes. I expect it to be a bit better when the kernel has to do page stealing. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 11:42:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from gull.prod.itd.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by hub.freebsd.org (Postfix) with ESMTP id C39A437B404; Mon, 14 Jan 2002 11:42:37 -0800 (PST) Received: from pool0325.cvx40-bradley.dialup.earthlink.net ([216.244.43.70] helo=mindspring.com) by gull.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QD00-0001zO-00; Mon, 14 Jan 2002 11:42:28 -0800 Message-ID: <3C4334A1.6601C5ED@mindspring.com> Date: Mon, 14 Jan 2002 11:42:25 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Greenman Cc: Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114094738.A8955@nexus.root.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG David Greenman wrote: > I think counting bytes with a 64bit counter is a perfectly reasonable > thing to do. I don't think this means that we need to update ALL of the > counters in the network stack to be 64bits, just the ones that are useful > to network admins (which is essentially just bytes in/out and packets > in/out). All the rest can stay 32bit. This isn't that much overhead to > add and the results are very useful, despite what you and Terry would have > some people believe. How many straws does it take to break a performance camel's back? Of course, let us not forget that you are invested in 64 bit, which won't care about this, anyway. I have a hard time paying for something I'm not going to use, that most people are not going to use, even if it's "only a few percent here and there". It's obvious that he's running a Gigabit ethernet at about 1/4 wire speed. He claims a 32 bit byte counter overflow in a couple of minutes, and the theoretical max load limit puts overflow of such a counter at 32 seconds, so that implies a byte count at 250Mbit/S. It's just as obvious that, since the counting occurs at interrupt, making atomicity guarantees in the face of hardware interrupts potentially occurring between component instructions is going to be a computationally expensive proposition. Let me tell you that, in the field, he is unlikely to find any colocation facility with the ability to overflow in under 5 minutes (two OC3's, full saturated by his one box, are only 100Mbit/S). You *know* this from running the most massive FTP server on the net for years, right? It's perfectly reasonable to recommend that he keep 1024 byte precision, while maintaining the same accuracy (e.g. using a modular byte counter, and conting overflows), which would amplify his "couple minutes" into 34 hours (he's got a bad lab setup: he should be able to get full wire speed, even out of a Tigon II, if he tunes correctly). Worst case, it's 8 1/2 hours in his lab; in reality, it's going to be 3 1/2 days in the field, assuming he fully saturates two OC3's (and assuming he's even permitted to do that). Given the volumes of data he's transitting, I'd up that to 1Mbyte accuracy, byt keeping the 1 byte precision -- that gives 1 year in the lab and 10 years in the field until an overflow. Burst throughput, sustained throughput, and the knee between them will be very close together, and stress-testing should not be combined with the other two, in any case. I think that what he's trying to measure is probably a bogus "figure of merit". -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 11:45:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from gull.prod.itd.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by hub.freebsd.org (Postfix) with ESMTP id 6F76F37B476; Mon, 14 Jan 2002 11:45:38 -0800 (PST) Received: from pool0325.cvx40-bradley.dialup.earthlink.net ([216.244.43.70] helo=mindspring.com) by gull.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QD32-0006p8-00; Mon, 14 Jan 2002 11:45:37 -0800 Message-ID: <3C43355D.8C9A35A0@mindspring.com> Date: Mon, 14 Jan 2002 11:45:33 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Greenman Cc: "James E. Housley" , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> <20020114100633.B8955@nexus.root.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG David Greenman wrote: > >Actually, I was thinking that if what you are counting overflowed > >in such a short period of time, then you are counting the wrong > >thing. > > > >Instead of bytes, try counting K or frames, instead of frames, > >count packets, etc.. > > Kbytes doesn't have enough resolution to provide accurate results since > the packets themselves are not integer multiples of Kbytes. Use a modular counter; count bytes, and reflect them out at 1K boundaries. Thus the accuracy is 1K, but the precision is 1 byte. I have a hard time beliving he needs 1 byte accuracy for the amount of data he is pushing. Accounting records could not be written to disk fast enough. > >You still haven't told us exactly what counter is overflowing... > > He's just trying to do what every network admin is trying to do: measure > link utilization (mostly bits per second) so that he can make intelligent > decisions about capacity, use, and associated costs. OK, he can do that with 1 byte precision, and 1K (or 1M) accuracy, so it's no problem... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 11:47:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from thehousleys.net (frenchknot.ne.mediaone.net [66.31.234.148]) by hub.freebsd.org (Postfix) with ESMTP id 01E7637B404; Mon, 14 Jan 2002 11:47:19 -0800 (PST) Received: (from root@localhost) by thehousleys.net (8.11.6/8.11.2) id g0EJl8d03887; Mon, 14 Jan 2002 14:47:08 -0500 (EST) (envelope-from jeh@FreeBSD.org) Received: from FreeBSD.org (baby.int.thehousleys.net [192.168.0.125]) (authenticated) by thehousleys.net (8.11.6/8.11.6) with ESMTP id g0EJl6g03878; Mon, 14 Jan 2002 14:47:06 -0500 (EST) (envelope-from jeh@FreeBSD.org) Message-ID: <3C4335BA.63828FD0@FreeBSD.org> Date: Mon, 14 Jan 2002 14:47:06 -0500 From: "James E. Housley" Organization: FreeBSD X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > "James E. Housley" wrote: > > Right now I am trying to do network usage on the overflows the 32 bit > > counters in 60 120 seconds. That means the netstat values are useless, > > mrtg and other programs that poll for results every 5 minutes or so are > > useless. If I want to get good data I have to get the counts every 10 > > seconds or so to get reasonable results. > > Actually, I was thinking that if what you are counting overflowed > in such a short period of time, then you are counting the wrong > thing. > > Instead of bytes, try counting K or frames, instead of frames, > count packets, etc.. > > You still haven't told us exactly what counter is overflowing... > Bytes out is overflowing on a nice new DG box. Jim -- /"\ ASCII Ribbon Campaign . \ / - NO HTML/RTF in e-mail . X - NO Word docs in e-mail . / \ ----------------------------------------------------------------- jeh@FreeBSD.org http://www.FreeBSD.org The Power to Serve jim@TheHousleys.Net http://www.TheHousleys.net jhousley@SimTel.Net http://www.SimTel.Net --------------------------------------------------------------------- Nothing is fool proof, because fools are too ingenious. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 11:49:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from gull.prod.itd.earthlink.net (gull.mail.pas.earthlink.net [207.217.120.84]) by hub.freebsd.org (Postfix) with ESMTP id 61FF137B400; Mon, 14 Jan 2002 11:49:54 -0800 (PST) Received: from pool0325.cvx40-bradley.dialup.earthlink.net ([216.244.43.70] helo=mindspring.com) by gull.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QD7B-0005cB-00; Mon, 14 Jan 2002 11:49:54 -0800 Message-ID: <3C43365F.6A75B37B@mindspring.com> Date: Mon, 14 Jan 2002 11:49:51 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "James E. Housley" Cc: Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> <3C4335BA.63828FD0@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "James E. Housley" wrote: > > Actually, I was thinking that if what you are counting overflowed > > in such a short period of time, then you are counting the wrong > > thing. > > > > Instead of bytes, try counting K or frames, instead of frames, > > count packets, etc.. > > > > You still haven't told us exactly what counter is overflowing... > > Bytes out is overflowing on a nice new DG box. DG = Data General? David Greenman? 8-) I think you should use a modular counter. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12: 1:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from thehousleys.net (frenchknot.ne.mediaone.net [66.31.234.148]) by hub.freebsd.org (Postfix) with ESMTP id 6A7BB37B438; Mon, 14 Jan 2002 12:00:45 -0800 (PST) Received: (from root@localhost) by thehousleys.net (8.11.6/8.11.2) id g0EK0in04158; Mon, 14 Jan 2002 15:00:44 -0500 (EST) (envelope-from jeh@FreeBSD.org) Received: from FreeBSD.org (baby.int.thehousleys.net [192.168.0.125]) (authenticated) by thehousleys.net (8.11.6/8.11.6) with ESMTP id g0EK0gg04150; Mon, 14 Jan 2002 15:00:42 -0500 (EST) (envelope-from jeh@FreeBSD.org) Message-ID: <3C4338EA.1D698EF1@FreeBSD.org> Date: Mon, 14 Jan 2002 15:00:42 -0500 From: "James E. Housley" Organization: FreeBSD X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: David Greenman , Bosko Milekic , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114094738.A8955@nexus.root.com> <3C4334A1.6601C5ED@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > David Greenman wrote: > > I think counting bytes with a 64bit counter is a perfectly reasonable > > thing to do. I don't think this means that we need to update ALL of the > > counters in the network stack to be 64bits, just the ones that are useful > > to network admins (which is essentially just bytes in/out and packets > > in/out). All the rest can stay 32bit. This isn't that much overhead to > > add and the results are very useful, despite what you and Terry would have > > some people believe. > > How many straws does it take to break a performance camel's back? > > Of course, let us not forget that you are invested in 64 bit, > which won't care about this, anyway. > > I have a hard time paying for something I'm not going to use, > that most people are not going to use, even if it's "only a few > percent here and there". > > It's obvious that he's running a Gigabit ethernet at about 1/4 > wire speed. He claims a 32 bit byte counter overflow in a couple > of minutes, and the theoretical max load limit puts overflow of > such a counter at 32 seconds, so that implies a byte count at > 250Mbit/S. > > It's just as obvious that, since the counting occurs at > interrupt, making atomicity guarantees in the face of hardware > interrupts potentially occurring between component instructions > is going to be a computationally expensive proposition. What a minute. If we are in the interrupt routine for this piece of hardware and interrupts. Why can't we do the addition during this initial poriton while the interrupts are blocked. No one else should be incrementing this counter since we are presently servicing it. I do understand that in a SMP system there could be a problem with reading the data correctly, so this might not work. > > Let me tell you that, in the field, he is unlikely to find any > colocation facility with the ability to overflow in under 5 > minutes (two OC3's, full saturated by his one box, are only > 100Mbit/S). You *know* this from running the most massive > FTP server on the net for years, right? This is a very nice, big box and it is running at that capacity or more. If you want more details ask DG. He built and installed the box. > > It's perfectly reasonable to recommend that he keep 1024 byte > precision, while maintaining the same accuracy (e.g. using a > modular byte counter, and conting overflows), which would > amplify his "couple minutes" into 34 hours (he's got a bad > lab setup: he should be able to get full wire speed, even out > of a Tigon II, if he tunes correctly). > > Worst case, it's 8 1/2 hours in his lab; in reality, it's > going to be 3 1/2 days in the field, assuming he fully > saturates two OC3's (and assuming he's even permitted to do > that). > > Given the volumes of data he's transitting, I'd up that to > 1Mbyte accuracy, byt keeping the 1 byte precision -- that > gives 1 year in the lab and 10 years in the field until an > overflow. > > Burst throughput, sustained throughput, and the knee between > them will be very close together, and stress-testing should > not be combined with the other two, in any case. I think > that what he's trying to measure is probably a bogus "figure > of merit". > That sounds reasonable. 1024 might be a problem becuase of smaller ACK or setup packets, but the concept is reasonable. I would like to know the amount of data transmitted and received because that is what we are billed on. And at that datarate, to the bit accuracy is crazy. But the results should be close enough to reality to be able to trust it with a small fudge fact for saftey. Jim -- /"\ ASCII Ribbon Campaign . \ / - NO HTML/RTF in e-mail . X - NO Word docs in e-mail . / \ ----------------------------------------------------------------- jeh@FreeBSD.org http://www.FreeBSD.org The Power to Serve jim@TheHousleys.Net http://www.TheHousleys.net jhousley@SimTel.Net http://www.SimTel.Net --------------------------------------------------------------------- Nothing is fool proof, because fools are too ingenious. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12: 1:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prism.flugsvamp.com (66-188-92-95.mad.wi.charter.com [66.188.92.95]) by hub.freebsd.org (Postfix) with ESMTP id 1727237B41F for ; Mon, 14 Jan 2002 12:01:49 -0800 (PST) Received: (from jlemon@localhost) by prism.flugsvamp.com (8.11.6/8.11.6) id g0EJwh113608; Mon, 14 Jan 2002 13:58:43 -0600 (CST) (envelope-from jlemon) Date: Mon, 14 Jan 2002 13:58:43 -0600 From: Jonathan Lemon To: Eric Masson Cc: Mailing List FreeBSD Arch Subject: Re: Cryptographic hardware support Message-ID: <20020114135843.J59128@prism.flugsvamp.com> References: <867kqm1rb4.fsf@notbsdems.nantes.kisoft-services.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: <867kqm1rb4.fsf@notbsdems.nantes.kisoft-services.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Jan 13, 2002 at 11:25:03AM +0100, Eric Masson wrote: > > Wouldn't it be interesting to share this type of support across the > bsd's (no duplicate of work for hardware support for example) ? This would be a worthy goal, but has some practical obstacles. The OpenBSD driver doesn't fit completely into the FreeBSD framework, and the existing driver needs substantial cleanup as well. The OpenBSD guys are currently revamping their framework, so the target is in a state of flux. > Is there any roadmap for inclusion in the tree ? Yes, I have funding to do crytographic hardware offloading. I have some sample boards from PowerCrypt and Soekris, and hope to have the code ready for distribution by the end of the month. This would include the drivers for the board, as well as the network changes to support offloading. -- Jonathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12: 2:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from thehousleys.net (frenchknot.ne.mediaone.net [66.31.234.148]) by hub.freebsd.org (Postfix) with ESMTP id 5D71737B43A; Mon, 14 Jan 2002 12:02:13 -0800 (PST) Received: (from root@localhost) by thehousleys.net (8.11.6/8.11.2) id g0EK29I04234; Mon, 14 Jan 2002 15:02:09 -0500 (EST) (envelope-from jeh@FreeBSD.org) Received: from FreeBSD.org (baby.int.thehousleys.net [192.168.0.125]) (authenticated) by thehousleys.net (8.11.6/8.11.6) with ESMTP id g0EK27g04226; Mon, 14 Jan 2002 15:02:07 -0500 (EST) (envelope-from jeh@FreeBSD.org) Message-ID: <3C43393F.6B07427E@FreeBSD.org> Date: Mon, 14 Jan 2002 15:02:07 -0500 From: "James E. Housley" Organization: FreeBSD X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> <3C4335BA.63828FD0@FreeBSD.org> <3C43365F.6A75B37B@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > "James E. Housley" wrote: > > > Actually, I was thinking that if what you are counting overflowed > > > in such a short period of time, then you are counting the wrong > > > thing. > > > > > > Instead of bytes, try counting K or frames, instead of frames, > > > count packets, etc.. > > > > > > You still haven't told us exactly what counter is overflowing... > > > > Bytes out is overflowing on a nice new DG box. > > DG = Data General? David Greenman? 8-) > David Greenman -- /"\ ASCII Ribbon Campaign . \ / - NO HTML/RTF in e-mail . X - NO Word docs in e-mail . / \ ----------------------------------------------------------------- jeh@FreeBSD.org http://www.FreeBSD.org The Power to Serve jim@TheHousleys.Net http://www.TheHousleys.net jhousley@SimTel.Net http://www.SimTel.Net --------------------------------------------------------------------- Nothing is fool proof, because fools are too ingenious. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12: 3:22 2002 Delivered-To: freebsd-arch@freebsd.org Received: from root.com (root.com [209.102.106.178]) by hub.freebsd.org (Postfix) with ESMTP id CB85D37B486; Mon, 14 Jan 2002 12:03:08 -0800 (PST) Received: (from dg@localhost) by root.com (8.11.2/8.11.2) id g0EJp5k09573; Mon, 14 Jan 2002 11:51:05 -0800 (PST) (envelope-from dg) Date: Mon, 14 Jan 2002 11:51:04 -0800 From: David Greenman To: Terry Lambert Cc: Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again Message-ID: <20020114115104.C8955@nexus.root.com> References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114094738.A8955@nexus.root.com> <3C4334A1.6601C5ED@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3C4334A1.6601C5ED@mindspring.com>; from tlambert2@mindspring.com on Mon, Jan 14, 2002 at 11:42:25AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >Let me tell you that, in the field, he is unlikely to find any >colocation facility with the ability to overflow in under 5 >minutes (two OC3's, full saturated by his one box, are only >100Mbit/S). You *know* this from running the most massive >FTP server on the net for years, right? You are so incredibly naive. Today's colo's use gigabit ethernet. And yes, I am speaking as someone who runs a company that overflows these counters on our servers in less than five minutes and has machines in said colo's connected with gigabit ethernet. Actually, the overflow in 5 minutes problem happens even with a single 155Mbps OC3 circuit...but that's really irrelevant to this discussion. >It's perfectly reasonable to recommend that he keep 1024 byte >precision, while maintaining the same accuracy (e.g. using a >modular byte counter, and conting overflows), which would >amplify his "couple minutes" into 34 hours (he's got a bad >lab setup: he should be able to get full wire speed, even out >of a Tigon II, if he tunes correctly). Um, that still results in a 64bit counter since that's how most 32bit machines do 64bit adds. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12: 6: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from root.com (root.com [209.102.106.178]) by hub.freebsd.org (Postfix) with ESMTP id AEA6A37B417; Mon, 14 Jan 2002 12:06:01 -0800 (PST) Received: (from dg@localhost) by root.com (8.11.2/8.11.2) id g0EJsXl09613; Mon, 14 Jan 2002 11:54:33 -0800 (PST) (envelope-from dg) Date: Mon, 14 Jan 2002 11:54:33 -0800 From: David Greenman To: "James E. Housley" Cc: Terry Lambert , Bosko Milekic , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again Message-ID: <20020114115433.D8955@nexus.root.com> References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114094738.A8955@nexus.root.com> <3C4334A1.6601C5ED@mindspring.com> <3C4338EA.1D698EF1@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3C4338EA.1D698EF1@FreeBSD.org>; from jeh@FreeBSD.org on Mon, Jan 14, 2002 at 03:00:42PM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >That sounds reasonable. 1024 might be a problem becuase of smaller ACK >or setup packets, but the concept is reasonable. I would like to know >the amount of data transmitted and received because that is what we are >billed on. And at that datarate, to the bit accuracy is crazy. But the >results should be close enough to reality to be able to trust it with a >small fudge fact for saftey. It's not reasonable because most packets are either small (40 bytes) or MTU sized. The distribution of sizes is not random and keeping only 1K internal precision would not result in anything near accurate in terms of bps. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12: 7:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id D256137B417 for ; Mon, 14 Jan 2002 12:07:18 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.3/8.11.3) with ESMTP id g0EK7Hl10261; Mon, 14 Jan 2002 13:07:17 -0700 (MST) (envelope-from imp@village.org) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.11.6/8.11.6) with ESMTP id g0EK7Gx74793; Mon, 14 Jan 2002 13:07:17 -0700 (MST) (envelope-from imp@village.org) Date: Mon, 14 Jan 2002 13:07:05 -0700 (MST) Message-Id: <20020114.130705.84407599.imp@village.org> To: bmilekic@technokratis.com Cc: arch@FreeBSD.ORG Subject: Re: 64 bit counters again From: "M. Warner Losh" In-Reply-To: <20020114114911.A24990@technokratis.com> References: <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> X-Mailer: Mew version 2.1 on Emacs 21.1 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message: <20020114114911.A24990@technokratis.com> Bosko Milekic writes: : You're going to run into the same problem with a 64-bit counter, as : Terry pointed out. You're just going to end up moving the time it : takes from 172 seconds to some other, slightly longer, although equally : unreasonable, time. A 64-bit qauntity would take 738734374912 seconds to overflow if the 32-bit one overflowed in 172 seconds. That strikes me as a reasonable enough period of time. If I did the math right, that's on the order of 23 millinia (23425 years). : Instead of counting bytes, perhaps you should count : K or M Bytes. Just take the delta you're about to add to your counter : and divide it by something reasonable before adding it. If you are : really bent on getting accurate results, maintain a separate counter : and sum remainders in it. I'm sure you'll get slightly more accurate : data while not changing all counters to be 64-bits wide on non-64 bit : platforms (i.e., you'll keep the added execution time local to that : specific code you're working on). That's a whole lot more hair than a 64-bit counter :-). Maybe you could do it in k bytes, but the typical packet size is on the order of 1.5k, so you have an error in the neighborhood of 50%. This would be 0M always. It seems to be that the atomicacy of the updates for any accumlated remainder scheme would be the same as for a 64-bit counter, and a whole lot more hair. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12: 9:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from root.com (root.com [209.102.106.178]) by hub.freebsd.org (Postfix) with ESMTP id E88F637B417; Mon, 14 Jan 2002 12:09:08 -0800 (PST) Received: (from dg@localhost) by root.com (8.11.2/8.11.2) id g0EJvdx09665; Mon, 14 Jan 2002 11:57:39 -0800 (PST) (envelope-from dg) Date: Mon, 14 Jan 2002 11:57:39 -0800 From: David Greenman To: Terry Lambert Cc: "James E. Housley" , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again Message-ID: <20020114115739.E8955@nexus.root.com> References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> <20020114100633.B8955@nexus.root.com> <3C43355D.8C9A35A0@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3C43355D.8C9A35A0@mindspring.com>; from tlambert2@mindspring.com on Mon, Jan 14, 2002 at 11:45:33AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >David Greenman wrote: >> >Actually, I was thinking that if what you are counting overflowed >> >in such a short period of time, then you are counting the wrong >> >thing. >> > >> >Instead of bytes, try counting K or frames, instead of frames, >> >count packets, etc.. >> >> Kbytes doesn't have enough resolution to provide accurate results since >> the packets themselves are not integer multiples of Kbytes. > >Use a modular counter; count bytes, and reflect them out at 1K >boundaries. Thus the accuracy is 1K, but the precision is 1 >byte. It's not how we "reflect them out" that's the problem. It's the overflow on the internal 32bit precision that's the problem. You could export the data in Kbps for all I care, but that has nothing to do with the problem. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12:10:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from voi.aagh.net (pc1-hart4-0-cust168.mid.cable.ntl.com [62.254.84.168]) by hub.freebsd.org (Postfix) with ESMTP id 0263437B417 for ; Mon, 14 Jan 2002 12:10:20 -0800 (PST) Received: from freaky by voi.aagh.net with local (Exim 3.34 #1) id 16QDQw-000Glu-00 for arch@FreeBSD.org; Mon, 14 Jan 2002 20:10:18 +0000 Date: Mon, 14 Jan 2002 20:10:18 +0000 From: Thomas Hurst To: arch@FreeBSD.org Subject: Re: 64 bit counters again Message-ID: <20020114201018.GA64041@voi.aagh.net> Mail-Followup-To: arch@FreeBSD.org References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3C431EE5.1CFF557B@mindspring.com> User-Agent: Mutt/1.3.25i Organization: Not much. X-Operating-System: FreeBSD/4.5-PRERELEASE (i386) X-Uptime: 7:51PM up 25 days, 4:36, 4 users, load averages: 2.17, 2.08, 2.03 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Terry Lambert (tlambert2@mindspring.com) wrote: > "James E. Housley" wrote: > > Right now I am trying to do network usage on the overflows the 32 bit > > counters in 60 120 seconds. 4.2GB/minute? Tasty. > Actually, I was thinking that if what you are counting overflowed > in such a short period of time, then you are counting the wrong > thing. James is? I thought FreeBSD was doing the counting? :) > You still haven't told us exactly what counter is overflowing... This machine is the box sitting in front of my single user LAN connected to a cable modem: tcp: 11962790 packets sent 7754863 data packets (3000140145 bytes) 12199847 packets received 5346910 acks (for 2999095921 bytes) 8383470 packets (4099500609 bytes) received in-sequence 320301 out-of-order packets (423292462 bytes) It's been up for 25 days, and every single one of those byte counters is totally bogus (well, *maybe* sent is accurate, received cetainly isn't, or won't be very soon). I'm just interested in how much bandwidth I'm using, but if the counters are going to overflow this easily, they cease being terribly useful, unless I'm going to actively count them and watch for overflows in userspace. I'm not really bothered if they're totally accurate, since they're just a guide - how about a float that counts kb? ;) -- Thomas 'Freaky' Hurst - freaky@aagh.net - http://www.aagh.net/ - Authors are easy to get on with -- if you're fond of children. -- Michael Joseph, "Observer" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 12:26:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from musique.teaser.net (musique.teaser.net [213.91.2.11]) by hub.freebsd.org (Postfix) with ESMTP id 5C11B37B41E for ; Mon, 14 Jan 2002 12:26:48 -0800 (PST) Received: from notbsdems.nantes.kisoft-services.com (nantes.kisoft-services.com [193.56.60.243]) by musique.teaser.net (Postfix) with ESMTP id F38AD7250E; Mon, 14 Jan 2002 21:26:46 +0100 (CET) Received: by notbsdems.nantes.kisoft-services.com (Postfix, from userid 1001) id 3F89FE6C99; Mon, 14 Jan 2002 21:27:01 +0100 (CET) To: Jonathan Lemon Cc: Mailing List FreeBSD Arch Subject: Re: Cryptographic hardware support References: <867kqm1rb4.fsf@notbsdems.nantes.kisoft-services.com> <20020114135843.J59128@prism.flugsvamp.com> From: Eric Masson In-Reply-To: <20020114135843.J59128@prism.flugsvamp.com> (Jonathan Lemon's message of "Mon, 14 Jan 2002 13:58:43 -0600") X-Operating-System: FreeBSD 4.5-PRERELEASE i386 Date: Mon, 14 Jan 2002 21:27:00 +0100 Message-ID: <86n0zgoezv.fsf@notbsdems.nantes.kisoft-services.com> Lines: 34 User-Agent: Gnus/5.090005 (Oort Gnus v0.05) XEmacs/21.1 (Cuyahoga Valley, i386--freebsd) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "Jonathan" == Jonathan Lemon writes: Jonathan> The OpenBSD driver doesn't fit completely into the FreeBSD Jonathan> framework, and the existing driver needs substantial cleanup Jonathan> as well. From a quick glance at the code, It didn't look familiar (I'm trying to understand driver programming atm) Jonathan> The OpenBSD guys are currently revamping their framework, so Jonathan> the target is in a state of flux. Too bad, a common api for offloading (ipsec, rng, ssl computations) would be nice. Jonathan> Yes, I have funding to do crytographic hardware offloading. I Jonathan> have some sample boards from PowerCrypt and Soekris, and hope Jonathan> to have the code ready for distribution by the end of the Jonathan> month. Jonathan> This would include the drivers for the board, as well as the Jonathan> network changes to support offloading. Great news, my little experimentation box (podp5v83) lacks power for ipsec and a Soekris vpn1201 would really help. Thanks for your answer. Eric -- >Une RedHat (je ne connais pas les autres distributions) ce configure >aussi simplement que windows pour un poste client. Hélas, elle génère un maximum de traffic sur Usenet -+- TP in guide du linuxien pervers - "Je veux revoir ma SLS ! -+- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 13:19:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from iclub.nsu.ru (iclub.nsu.ru [193.124.222.66]) by hub.freebsd.org (Postfix) with ESMTP id 5E1E137B402 for ; Mon, 14 Jan 2002 13:19:18 -0800 (PST) Received: (from fjoe@localhost) by iclub.nsu.ru (8.11.6/8.11.6) id g0ELJ2899346 for arch@freebsd.org; Tue, 15 Jan 2002 03:19:02 +0600 (NS) (envelope-from fjoe) Date: Tue, 15 Jan 2002 03:19:01 +0600 From: Max Khon To: arch@freebsd.org Subject: request for review Message-ID: <20020115031901.A99067@iclub.nsu.ru> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="qDbXVdCdHGoSgWSk" Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --qDbXVdCdHGoSgWSk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline hi, there! please review attached patch. it should fix `diff' when one of the compared files is a tty (e.g. 'diff -u - /etc/motd'). The problem is that fstat(0, &s) returns s.st_blocksize == 0 in this case and this breaks `diff'. I do not know if this is correct fix (should `diff' be fixed instead?) but under OpenBSD 2.9 and Linux 2.2.x non-zero values are returned. /fjoe --qDbXVdCdHGoSgWSk Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="vfs_vnops.c.diff" ? current-diffs Index: vfs_vnops.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_vnops.c,v retrieving revision 1.125 diff -u -p -r1.125 vfs_vnops.c --- vfs_vnops.c 18 Dec 2001 20:48:54 -0000 1.125 +++ vfs_vnops.c 14 Jan 2002 18:04:45 -0000 @@ -579,6 +579,8 @@ vn_stat(vp, sb, td) sb->st_blksize = vp->v_rdev->si_bsize_phys; if (sb->st_blksize < BLKDEV_IOSIZE) sb->st_blksize = BLKDEV_IOSIZE; + } else if (vap->va_type == VCHR) { + sb->st_blksize = vap->va_blocksize; } else { sb->st_blksize = 0; } --qDbXVdCdHGoSgWSk-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 14:35:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from technokratis.com (modemcable099.144-201-24.mtl.mc.videotron.ca [24.201.144.99]) by hub.freebsd.org (Postfix) with ESMTP id 4121237B41D for ; Mon, 14 Jan 2002 14:35:22 -0800 (PST) Received: (from bmilekic@localhost) by technokratis.com (8.11.4/8.11.3) id g0EMbAm26960; Mon, 14 Jan 2002 17:37:10 -0500 (EST) (envelope-from bmilekic) Date: Mon, 14 Jan 2002 17:37:10 -0500 From: Bosko Milekic To: Jonathan Lemon Cc: Eric Masson , Mailing List FreeBSD Arch Subject: Re: Cryptographic hardware support Message-ID: <20020114173710.A26934@technokratis.com> References: <867kqm1rb4.fsf@notbsdems.nantes.kisoft-services.com> <20020114135843.J59128@prism.flugsvamp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020114135843.J59128@prism.flugsvamp.com>; from jlemon@flugsvamp.com on Mon, Jan 14, 2002 at 01:58:43PM -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Jan 14, 2002 at 01:58:43PM -0600, Jonathan Lemon wrote: > On Sun, Jan 13, 2002 at 11:25:03AM +0100, Eric Masson wrote: > > > > Wouldn't it be interesting to share this type of support across the > > bsd's (no duplicate of work for hardware support for example) ? > > This would be a worthy goal, but has some practical obstacles. The > OpenBSD driver doesn't fit completely into the FreeBSD framework, and > the existing driver needs substantial cleanup as well. The OpenBSD > guys are currently revamping their framework, so the target is in a > state of flux. > > > > Is there any roadmap for inclusion in the tree ? > > Yes, I have funding to do crytographic hardware offloading. I have > some sample boards from PowerCrypt and Soekris, and hope to have the > code ready for distribution by the end of the month. > > This would include the drivers for the board, as well as the network > changes to support offloading. This is really cool. One question: Isn't Mark Murray also doing something along these lines? > -- > Jonathan > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > -- Bosko Milekic bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 18:28: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id DB9A537B404; Mon, 14 Jan 2002 18:27:59 -0800 (PST) Received: from pool0039.cvx22-bradley.dialup.earthlink.net ([209.179.198.39] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QJK8-0002HQ-00; Mon, 14 Jan 2002 18:27:40 -0800 Message-ID: <3C439398.D29AB1C2@mindspring.com> Date: Mon, 14 Jan 2002 18:27:36 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Greenman Cc: Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114094738.A8955@nexus.root.com> <3C4334A1.6601C5ED@mindspring.com> <20020114115104.C8955@nexus.root.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG David Greenman wrote: > Um, that still results in a 64bit counter since that's how most 32bit > machines do 64bit adds. No, it requires him to have an unlocked device instance remainder field, with a locked 32 bit global K-counter field. You could even update the global at NETISR or some other less time critical juncture. Doing a 64 bit add would need locked add with carry for the whole thing. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 18:36:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id 7562837B41E; Mon, 14 Jan 2002 18:36:35 -0800 (PST) Received: from pool0039.cvx22-bradley.dialup.earthlink.net ([209.179.198.39] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QJSd-0006M9-00; Mon, 14 Jan 2002 18:36:27 -0800 Message-ID: <3C4395A7.EC3F89A7@mindspring.com> Date: Mon, 14 Jan 2002 18:36:23 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Greenman Cc: "James E. Housley" , Bosko Milekic , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114094738.A8955@nexus.root.com> <3C4334A1.6601C5ED@mindspring.com> <3C4338EA.1D698EF1@FreeBSD.org> <20020114115433.D8955@nexus.root.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG David Greenman wrote: > >That sounds reasonable. 1024 might be a problem becuase of smaller ACK > >or setup packets, but the concept is reasonable. I would like to know > >the amount of data transmitted and received because that is what we are > >billed on. And at that datarate, to the bit accuracy is crazy. But the > >results should be close enough to reality to be able to trust it with a > >small fudge fact for saftey. > > It's not reasonable because most packets are either small (40 bytes) > or MTU sized. The distribution of sizes is not random and keeping only > 1K internal precision would not result in anything near accurate in terms > of bps. There is a difference between *precision* and *accuracy*. Here is how he could get a 2^10 reduce precision, with 0 reduction in accuracy: He would update his 32 bit byte counter, then he would div it by 1024 (load register from register, shift 10), then add this value to his global counter. Note that he *never* throws away the 10 bits of byte count, he just doesn't externalize it. It gets added to the next set of update values, so at worst, he will be low by a movign average of 512b in any given interval. There would be no loss of overall information, as you seem to be implying with your reference to 40 byte packets. Since the BPS is an *average*, and since the second resolution is *vastly* larger than the modification interval, there should be no problem with this approach. Obviously, the same approach works for any modularl counter modulo, so 20 bits (M instead of K) works, as well, if he needs that resolution because the data rate per second is so high relative to the sampling interval that overflow is ever a risk. Make sense? The point is that the modular counter never needs to be globally locked against contention, so we don't add all the locking that would be necessary for a full 64 bits of precision on a 32 bit architecture. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 18:41:33 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id B59D737B400; Mon, 14 Jan 2002 18:41:30 -0800 (PST) Received: from pool0039.cvx22-bradley.dialup.earthlink.net ([209.179.198.39] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QJXU-0005JK-00; Mon, 14 Jan 2002 18:41:28 -0800 Message-ID: <3C4396D5.93A84596@mindspring.com> Date: Mon, 14 Jan 2002 18:41:25 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Greenman Cc: "James E. Housley" , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> <20020114100633.B8955@nexus.root.com> <3C43355D.8C9A35A0@mindspring.com> <20020114115739.E8955@nexus.root.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG David Greenman wrote: > >Use a modular counter; count bytes, and reflect them out at 1K > >boundaries. Thus the accuracy is 1K, but the precision is 1 > >byte. > > It's not how we "reflect them out" that's the problem. It's the overflow > on the internal 32bit precision that's the problem. You could export the > data in Kbps for all I care, but that has nothing to do with the problem. The overflow you fear never happens, because you subtract out the div'ed value from the internal counter each time you add it to the externalized one, which is why it's a "modular counter", e.g.: driver_byte_count_32 += bytes_this_ISR; driver_k_count_32 = driver_byte_count_32 >> 10; lock_global_stat(); global_k_count_32 += driver_k_count_32; unlock_global_stat(); /* avoid feared overflow: this is a modular counter */ driver_byte_count_32 -= driver_k_count_32 << 10; -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 18:47:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id BFC3337B402; Mon, 14 Jan 2002 18:47:18 -0800 (PST) Received: from pool0039.cvx22-bradley.dialup.earthlink.net ([209.179.198.39] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QJd7-00054P-00; Mon, 14 Jan 2002 18:47:17 -0800 Message-ID: <3C439832.C52B4379@mindspring.com> Date: Mon, 14 Jan 2002 18:47:14 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "James E. Housley" Cc: David Greenman , Bosko Milekic , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114094738.A8955@nexus.root.com> <3C4334A1.6601C5ED@mindspring.com> <3C4338EA.1D698EF1@FreeBSD.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "James E. Housley" wrote: > > It's just as obvious that, since the counting occurs at > > interrupt, making atomicity guarantees in the face of hardware > > interrupts potentially occurring between component instructions > > is going to be a computationally expensive proposition. > > What a minute. If we are in the interrupt routine for this piece of > hardware and interrupts. Why can't we do the addition during this > initial poriton while the interrupts are blocked. No one else should be > incrementing this counter since we are presently servicing it. I do > understand that in a SMP system there could be a problem with reading > the data correctly, so this might not work. It's a good idea, but it won't work with the direction FreeBSD is headed with regards to processing interrupts in "ithreads". The problem is that the actual increment will be in the "top end" interrupt thread, not in the "bottom end" ISR handler. > That sounds reasonable. 1024 might be a problem becuase of smaller ACK > or setup packets, but the concept is reasonable. I would like to know > the amount of data transmitted and received because that is what we are > billed on. And at that datarate, to the bit accuracy is crazy. But the > results should be close enough to reality to be able to trust it with a > small fudge fact for saftey. See the pseudo-code in my previous posting. Having a lower precision on the reported value is not the same thing as losing accuracy. The smaller ACK and setup packets would be counted at the byte level by the modular counter, without damaging the accuracy of the count. In fact, if you were willing to take it in two pieces at splimp, you could report accuract to the byte in a 64 bit value, if you were hell-bent on doing that, with the only real impact being at the time of the report, rather than at the time of the count (which is what I'm trying to avoid here). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 19: 0:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id 31F8E37B402 for ; Mon, 14 Jan 2002 19:00:00 -0800 (PST) Received: from pool0039.cvx22-bradley.dialup.earthlink.net ([209.179.198.39] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QJpH-00067F-00; Mon, 14 Jan 2002 18:59:52 -0800 Message-ID: <3C439B25.27DCCB2D@mindspring.com> Date: Mon, 14 Jan 2002 18:59:49 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "M. Warner Losh" Cc: bmilekic@technokratis.com, arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114.130705.84407599.imp@village.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "M. Warner Losh" wrote: > : You're going to run into the same problem with a 64-bit counter, as > : Terry pointed out. You're just going to end up moving the time it > : takes from 172 seconds to some other, slightly longer, although equally > : unreasonable, time. > > A 64-bit qauntity would take 738734374912 seconds to overflow if the > 32-bit one overflowed in 172 seconds. That strikes me as a reasonable > enough period of time. If I did the math right, that's on the order > of 23 millinia (23425 years). Actually, it's closer to 4000 years, if the counter is for a Gigabit Ethernet running a theoretical max wire speed, where a 32 bit byte counter overflows in 32 seconds (1Gbit/S divided by 8 to get bytes/S). Enough interfaces to saturate PCI-X drops this to 500 years (8Gbit/S). The Apple 802.11e FireWire wireless is already at 5Gbit. The idea that some counter size will never be exceeded by technology by the time it is obsolete is what has people complaining about 32 bit counters in this thread already; are you prepared to make the same base assumption in the choice of 64 bit counters? A terabit network could push a 64 bit counter over in under 6 months. > : Instead of counting bytes, perhaps you should count > : K or M Bytes. Just take the delta you're about to add to your counter > : and divide it by something reasonable before adding it. If you are > : really bent on getting accurate results, maintain a separate counter > : and sum remainders in it. I'm sure you'll get slightly more accurate > : data while not changing all counters to be 64-bits wide on non-64 bit > : platforms (i.e., you'll keep the added execution time local to that > : specific code you're working on). > > That's a whole lot more hair than a 64-bit counter :-). Maybe you > could do it in k bytes, but the typical packet size is on the order of > 1.5k, so you have an error in the neighborhood of 50%. This would be > 0M always. It seems to be that the atomicacy of the updates for any > accumlated remainder scheme would be the same as for a 64-bit counter, > and a whole lot more hair. Please see other posting on how to do modular math to get a differential between bits of precision without losing accuracy. The main win here is that we don't end up with a global counter that requires us to do the "hokey-pokey" to update it because we can't use a single locked instruction to do the add because our processor is only capable of doing that with 32 bit values. Making the same atomicity guarantees for a 64 bit value is not the same as simply doing two instructions; it's a heck of a lot more expensive. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 14 19:41: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from thehousleys.net (frenchknot.ne.mediaone.net [66.31.234.148]) by hub.freebsd.org (Postfix) with ESMTP id 8655837B41C; Mon, 14 Jan 2002 19:41:01 -0800 (PST) Received: (from root@localhost) by thehousleys.net (8.11.6/8.11.2) id g0F3ewH10081; Mon, 14 Jan 2002 22:40:58 -0500 (EST) (envelope-from jeh@FreeBSD.org) Received: from FreeBSD.org (baby.int.thehousleys.net [192.168.0.125]) (authenticated) by thehousleys.net (8.11.6/8.11.6) with ESMTP id g0F3eug10070; Mon, 14 Jan 2002 22:40:56 -0500 (EST) (envelope-from jeh@FreeBSD.org) Message-ID: <3C43A4C8.F3DCDAB8@FreeBSD.org> Date: Mon, 14 Jan 2002 22:40:56 -0500 From: "James E. Housley" Organization: FreeBSD X-Mailer: Mozilla 4.79 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: Terry Lambert Cc: David Greenman , Bosko Milekic , Thomas Hurst , arch@FreeBSD.org Subject: Re: 64 bit counters again References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114094738.A8955@nexus.root.com> <3C4334A1.6601C5ED@mindspring.com> <3C4338EA.1D698EF1@FreeBSD.org> <3C439832.C52B4379@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by AMaViS perl-10 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > > "James E. Housley" wrote: > > > It's just as obvious that, since the counting occurs at > > > interrupt, making atomicity guarantees in the face of hardware > > > interrupts potentially occurring between component instructions > > > is going to be a computationally expensive proposition. > > > > What a minute. If we are in the interrupt routine for this piece of > > hardware and interrupts. Why can't we do the addition during this > > initial poriton while the interrupts are blocked. No one else should be > > incrementing this counter since we are presently servicing it. I do > > understand that in a SMP system there could be a problem with reading > > the data correctly, so this might not work. > > It's a good idea, but it won't work with the direction FreeBSD > is headed with regards to processing interrupts in "ithreads". > The problem is that the actual increment will be in the "top end" > interrupt thread, not in the "bottom end" ISR handler. I have been following the SMPng and was almost sure my suggestion would fail for the reason you stated, but in didn't hurt to much to ask and be sure. > > > That sounds reasonable. 1024 might be a problem becuase of smaller ACK > > or setup packets, but the concept is reasonable. I would like to know > > the amount of data transmitted and received because that is what we are > > billed on. And at that datarate, to the bit accuracy is crazy. But the > > results should be close enough to reality to be able to trust it with a > > small fudge fact for saftey. > > See the pseudo-code in my previous posting. Having a lower > precision on the reported value is not the same thing as losing I did read that. > accuracy. The smaller ACK and setup packets would be counted > at the byte level by the modular counter, without damaging the > accuracy of the count. In fact, if you were willing to take it > in two pieces at splimp, you could report accuract to the byte > in a 64 bit value, if you were hell-bent on doing that, with the > only real impact being at the time of the report, rather than at > the time of the count (which is what I'm trying to avoid here). > That seems like a resonable solution. All the data would be counted. In the simplest form the number of whole K would be returned with a slight amount of work to get the "full" data when it is needed. I would have no objections to this concept. Jim -- /"\ ASCII Ribbon Campaign . \ / - NO HTML/RTF in e-mail . X - NO Word docs in e-mail . / \ ----------------------------------------------------------------- jeh@FreeBSD.org http://www.FreeBSD.org The Power to Serve jim@TheHousleys.Net http://www.TheHousleys.net jhousley@SimTel.Net http://www.SimTel.Net --------------------------------------------------------------------- If it happens once, it's a bug. If it happens twice, it's a feature. If it happens more than twice, it's windows. -- Luiz de Barros To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 1:10:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id AD12F37B402; Tue, 15 Jan 2002 01:09:54 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0F99qle001391; Tue, 15 Jan 2002 10:09:52 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0F99qJe001384; Tue, 15 Jan 2002 10:09:52 +0100 (CET) Date: Tue, 15 Jan 2002 10:09:52 +0100 (CET) From: Michal Mertl To: Bosko Milekic Cc: "James E. Housley" , Terry Lambert , Thomas Hurst , Subject: Re: 64 bit counters again In-Reply-To: <20020114114911.A24990@technokratis.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The more important part of the code (I should probably have used different Subject) is to have all (some) kernel counters use an API which implements different operations for counters being modified in different situations. If the SYSCNTR_T is defined 32 and default branch in switch, which decides what operation to do, would do regular addition, the kernel operates the same as before with no performance hit and slightly less prone to problems because operation isn't read-modify-write. I wrote it so it's just matter of one #define change to have all counters switch to 32 or 64 bit. It also would be possible to have something like atomic funtctions - different counter functions for various counter sizes, maybe even something like "use counter at least 32 bits wide - If we need atomicity on <586 we can't simply implement these ops" - I agree that some counters would have sufficient precision and scale in 32 bit. You could have a counter, which can be modified by multiple CPUs, ISRs at the same time - you would need atomic ops. You could also have a counter, which is guaranteed to be modified only from single point in code where there's no possibility other thing would try to do that at the same time. If I understand correctly (and I won't be surprised if I am wrong, that's why I asked some questions about that here a week or so ago), on STABLE almost everything in kernel is very protected (Giant Mutex) - so we can get away with simple implementation. Remember - our current STABLE (and CURRENT) implementation uses just compiler generated operations (i+=10;). That in some cases (SMP in some nonlocked part of kernel) probably isn't right even in 32 bit. If compiler generates read, modify, write instructions, it can probably go wrong even on UP. I give you a tool to have these "bugs" cleanly fixed. There can be performance penalty for this fix - if we need atomic ops and don't use them and then start to use them, it's inevitable. But as I wrote in my original post, I tried to roughly measure 32 bit implementation (equivalent to what we have now), 32 bit atomic and 64 bit (atomic/non-atomic) implementation in real world - pushing traffic through an interface and saw no difference. Things can probably go quite a bit worse, if atomic implementations included lock instructions (SMP) and multiple simultaneous modifications were atempted, but I don't believe it will occur often in real world. I'll measure SMP atomic lock;cmpxchg8b anyway soon. I kind of measured the speed of just different addition implementation (DUAL pIII cumine, ServerWorks and 440BX dual boards with SMP kernel) - 64 bit simple (addl change,(mem),adcl 0,4(mem)): 8.22 clocks 64 bit atomic UP (cmpxchg8b): 34.92 64 bit atomic SMP (lock; cmpxchg8b): 49.68 32 bit simple (compiler generates read-modify-write - wrong but used throughout kernel!!!, only when counter variable is declared volatile compiler generates addl change,(mem)): 3.00 32 bit atomic UP (addl change,(mem)): 3.00 32 bit atomic SMP (lock; addl change,(mem)): 20.94 It seems you and others were right. SMP atomic implementations are a bit expensive. But even the worst case 50 clocks for locked cmpxchg8b isn't that bad but IMMV. Anyway - again, what's there now is '32 bit simple' - it's either bad (could be wrong esp. on SMP) or good enough. If it's bad it should be changed. Easy way is using something like I proposed. If it's good than '64 bit simple' is at least as good (it isn't read-modify-write) and only bit more expensive (8 clocks against 3 - both very small numbers). --- I totally disagree with Terry that 64 bit counter is only slightly more difficult to overflow then 32 bit. We will never (in near future - several years) have terabit interfaces nor wirespeed 10Gbit (and 10Gbit fullduplex would overflow in 500 years). If you don't like 64 bit, your fix (to count in KB) only adds 10 bits of range, switch to 64 bit adds 32 bit. I understand that global counter could be made 64 bit a the counter you're actually counting on 32 bit so you would add 42 bits. That won't overflow for sure :-). --- I deeply apologise that the patch I posted was badly broken. I revamped the code quite a lot before doing the diff and it resulted in lots of cut-n-paste and similar errors :-(. Kernel&world build went without problems but the code was wrong (in atomic.h and syscntr.h). New patch will be available soon. -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 1:12:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from smtp.tninet.se (sheridan.tninet.se [195.100.94.102]) by hub.freebsd.org (Postfix) with ESMTP id B386537B402 for ; Tue, 15 Jan 2002 01:12:40 -0800 (PST) Received: from kairos.algonet.se (kairos.algonet.se [194.213.75.171]) by sheridan.tninet.se (BMR ErlangTM/OTP 3.0) with ESMTP id 865528.85957.1011.1s3624182sheridan for ; Tue, 15 Jan 2002 10:12:37 +0100 Received: (qmail 23323 invoked by uid 2493); 15 Jan 2002 09:12:35 -0000 Date: 15 Jan 2002 09:12:35 -0000 Message-ID: <20020115091235.23322.qmail@kairos.algonet.se> From: Mats Lofkvist To: tlambert2@mindspring.com Cc: imp@village.org, bmilekic@technokratis.com, arch@FreeBSD.ORG In-reply-to: <3C439B25.27DCCB2D@mindspring.com> (message from Terry Lambert on Mon, 14 Jan 2002 18:59:49 -0800) Subject: Re: 64 bit counters again References: <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114.130705.84407599.imp@village.org> <3C439B25.27DCCB2D@mindspring.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG From: Terry Lambert > > A 64-bit qauntity would take 738734374912 seconds to overflow if the > > 32-bit one overflowed in 172 seconds. That strikes me as a reasonable > > enough period of time. If I did the math right, that's on the order > > of 23 millinia (23425 years). > > Actually, it's closer to 4000 years, if the counter is for a > Gigabit Ethernet running a theoretical max wire speed, where a > 32 bit byte counter overflows in 32 seconds (1Gbit/S divided by 8 > to get bytes/S). > > Enough interfaces to saturate PCI-X drops this to 500 years (8Gbit/S). > > The Apple 802.11e FireWire wireless is already at 5Gbit. > > The idea that some counter size will never be exceeded by > technology by the time it is obsolete is what has people > complaining about 32 bit counters in this thread already; > are you prepared to make the same base assumption in the > choice of 64 bit counters? A terabit network could push > a 64 bit counter over in under 6 months. And it (a terabit network) would overflow the 1kB modular counter solution in 35 seconds or so. Yes, I know, larger blocks than 1kB should be used in the global counter, and I guess multiple levels of (32-bit) modular counters can be used to achieve higher limits, but will it still be cheaper than just using a 64 bit counter? _ Mats Lofkvist mal@algonet.se To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 2:56:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 82C7737B402 for ; Tue, 15 Jan 2002 02:56:47 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0FArkU19070; Tue, 15 Jan 2002 11:53:46 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Max Khon Cc: arch@FreeBSD.ORG Subject: Re: request for review In-Reply-To: Your message of "Tue, 15 Jan 2002 03:19:01 +0600." <20020115031901.A99067@iclub.nsu.ru> Date: Tue, 15 Jan 2002 11:53:46 +0100 Message-ID: <19068.1011092026@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I think it is bogus for devices other than disks to flout a va_blocksize, but I am on the other hand not sure what the relevant (if any) standards say. If this patch makes things better, I'm all for it. Poul-Henning In message <20020115031901.A99067@iclub.nsu.ru>, Max Khon writes: > >--qDbXVdCdHGoSgWSk >Content-Type: text/plain; charset=us-ascii >Content-Disposition: inline > >hi, there! > >please review attached patch. it should fix `diff' when one of the >compared files is a tty (e.g. 'diff -u - /etc/motd'). >The problem is that fstat(0, &s) returns s.st_blocksize == 0 in this case >and this breaks `diff'. > >I do not know if this is correct fix (should `diff' be fixed instead?) >but under OpenBSD 2.9 and Linux 2.2.x non-zero values are returned. > >/fjoe > >--qDbXVdCdHGoSgWSk >Content-Type: text/plain; charset=us-ascii >Content-Disposition: attachment; filename="vfs_vnops.c.diff" > >? current-diffs >Index: vfs_vnops.c >=================================================================== >RCS file: /home/ncvs/src/sys/kern/vfs_vnops.c,v >retrieving revision 1.125 >diff -u -p -r1.125 vfs_vnops.c >--- vfs_vnops.c 18 Dec 2001 20:48:54 -0000 1.125 >+++ vfs_vnops.c 14 Jan 2002 18:04:45 -0000 >@@ -579,6 +579,8 @@ vn_stat(vp, sb, td) > sb->st_blksize = vp->v_rdev->si_bsize_phys; > if (sb->st_blksize < BLKDEV_IOSIZE) > sb->st_blksize = BLKDEV_IOSIZE; >+ } else if (vap->va_type == VCHR) { >+ sb->st_blksize = vap->va_blocksize; > } else { > sb->st_blksize = 0; > } > >--qDbXVdCdHGoSgWSk-- > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-arch" in the body of the message > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 6:39:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 3B1C337B41F for ; Tue, 15 Jan 2002 06:39:33 -0800 (PST) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id GAA17347; Tue, 15 Jan 2002 06:35:13 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda17337; Tue Jan 15 06:34:55 2002 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.6/8.9.1) id g0FEYoK30450; Tue, 15 Jan 2002 06:34:50 -0800 (PST) Received: from UNKNOWN(10.1.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdK30440; Tue Jan 15 06:34:48 2002 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.6/8.9.1) id g0FEYlK72569; Tue, 15 Jan 2002 06:34:47 -0800 (PST) Message-Id: <200201151434.g0FEYlK72569@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdL72556; Tue Jan 15 06:34:44 2002 X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-Sender: schubert To: Poul-Henning Kamp Cc: Max Khon , arch@FreeBSD.ORG Subject: Re: request for review In-reply-to: Your message of "Tue, 15 Jan 2002 11:53:46 +0100." <19068.1011092026@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 15 Jan 2002 06:34:44 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In addition to making FreeBSD consistent with OpenBSD and Linux, as stated by the originator of this thread, the patch also makes FreeBSD consistent with Solaris and Tru64-UNIX. I'm for it. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Email: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD Ministry of Management Services Province of BC FreeBSD UNIX: cy@FreeBSD.org In message <19068.1011092026@critter.freebsd.dk>, Poul-Henning Kamp writes: > > I think it is bogus for devices other than disks to flout a > va_blocksize, but I am on the other hand not sure what the > relevant (if any) standards say. > > If this patch makes things better, I'm all for it. > > Poul-Henning > > In message <20020115031901.A99067@iclub.nsu.ru>, Max Khon writes: > > > >--qDbXVdCdHGoSgWSk > >Content-Type: text/plain; charset=us-ascii > >Content-Disposition: inline > > > >hi, there! > > > >please review attached patch. it should fix `diff' when one of the > >compared files is a tty (e.g. 'diff -u - /etc/motd'). > >The problem is that fstat(0, &s) returns s.st_blocksize == 0 in this case > >and this breaks `diff'. > > > >I do not know if this is correct fix (should `diff' be fixed instead?) > >but under OpenBSD 2.9 and Linux 2.2.x non-zero values are returned. > > > >/fjoe > > > >--qDbXVdCdHGoSgWSk > >Content-Type: text/plain; charset=us-ascii > >Content-Disposition: attachment; filename="vfs_vnops.c.diff" > > > >? current-diffs > >Index: vfs_vnops.c > >=================================================================== > >RCS file: /home/ncvs/src/sys/kern/vfs_vnops.c,v > >retrieving revision 1.125 > >diff -u -p -r1.125 vfs_vnops.c > >--- vfs_vnops.c 18 Dec 2001 20:48:54 -0000 1.125 > >+++ vfs_vnops.c 14 Jan 2002 18:04:45 -0000 > >@@ -579,6 +579,8 @@ vn_stat(vp, sb, td) > > sb->st_blksize = vp->v_rdev->si_bsize_phys; > > if (sb->st_blksize < BLKDEV_IOSIZE) > > sb->st_blksize = BLKDEV_IOSIZE; > >+ } else if (vap->va_type == VCHR) { > >+ sb->st_blksize = vap->va_blocksize; > > } else { > > sb->st_blksize = 0; > > } > > > >--qDbXVdCdHGoSgWSk-- > > > >To Unsubscribe: send mail to majordomo@FreeBSD.org > >with "unsubscribe freebsd-arch" in the body of the message > > > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 7:26:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from grant.org (grant.org [206.190.164.98]) by hub.freebsd.org (Postfix) with ESMTP id AADB537B41C; Tue, 15 Jan 2002 07:26:16 -0800 (PST) Received: (from devon@localhost) by grant.org (8.11.6/8.11.6) id g0FFQFX02180; Tue, 15 Jan 2002 10:26:15 -0500 (EST) (envelope-from devon) Date: Tue, 15 Jan 2002 10:26:15 -0500 (EST) Message-Id: <200201151526.g0FFQFX02180@grant.org> From: FreeBSD@jovi.net To: cjc@FreeBSD.ORG Cc: freebsd@jovi.net, freebsd-gnats-submit@FreeBSD.ORG Cc: freebsd-arch@FreeBSD.ORG In-reply-to: <20020115011230.D28767@blossom.cjclark.org> Subject: Re: kern/33904: secure mode bug References: <200201142344.g0ENimK91227@freefall.freebsd.org> <20020115011230.D28767@blossom.cjclark.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thanks for your reply. I suggest escalating the trouble. Correct time is mission-critical on many systems and this is an issue of unreliable service under FreeBSD. A settimeofday other than that programmed is worse than doing nothing. Three orders of magnitude is a complete failure by every reasonable standard. Breaking date, ntpdate, ntpd, ... is a reliable indication of severe failure. These programs now need rewriting to operate reliably. Cheers --Devon /"\ \ / ASCII Ribbon Campaign X Help Cure HTML Mail / \ Date: Tue, 15 Jan 2002 01:12:30 -0800 From: "Crist J . Clark" Cc: freebsd-gnats-submit@FreeBSD.ORG On Mon, Jan 14, 2002 at 03:44:48PM -0800, D. McCullough wrote: [snip] > >Description: > settimeofday silently fails in secure mode. > > I wasted a lot of time baffled -- > this makes all the various date > apps and clients silently fail. The problem is that it doesn't _completely_ fail. > >Fix: > When an operation prohibited by secure mode fails, > this failure must be accurately reported by the API. If you run your test program, # ./testtime gettimeofday = 0; t = 1011071149.590024 settimeofday = 0; t = 1011067549.590024 gettimeofday = 0; t = 1011071148.591402 At securelevel == 2, we see that the time _has_ been changed by the call, but like the log message says, Jan 14 21:05:44 bubbles kernel: Time adjustment clamped to -1 second So the call of settimeofday(2) didn't actually fail due to "permission denied," but the result is not exactly what the caller expected either. Now, whether this whole approach is wrong is more of a discussion for freebsd-arch@freebsd.org. -- "It's always funny until someone gets hurt. Then it's hilarious." Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 8:30:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from green.bikeshed.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 469BE37B416; Tue, 15 Jan 2002 08:30:10 -0800 (PST) Received: from localhost (green@localhost) by green.bikeshed.org (8.11.6/8.11.6) with ESMTP id g0FGU5k77068; Tue, 15 Jan 2002 11:30:05 -0500 (EST) (envelope-from green@green.bikeshed.org) Message-Id: <200201151630.g0FGU5k77068@green.bikeshed.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Poul-Henning Kamp Cc: Max Khon , arch@FreeBSD.ORG Subject: Re: request for review In-Reply-To: Your message of "Tue, 15 Jan 2002 11:53:46 +0100." <19068.1011092026@critter.freebsd.dk> From: "Brian F. Feldman" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 15 Jan 2002 11:30:05 -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > I think it is bogus for devices other than disks to flout a > va_blocksize, but I am on the other hand not sure what the > relevant (if any) standards say. > > If this patch makes things better, I'm all for it. Posix specifies that st_size for non-regular files is "unspecified", however it doesn't specify st_blksize at all. In the "spirit" of POSIX I'd say st_blksize is free to be anything we want it to be, and non-0 for VCHR is certainly a reasonable value of anything. -- Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! / green@FreeBSD.org `------------------------------' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 8:34:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from glenfiddich.infospace.com (mail1.infospace.com [206.29.197.33]) by hub.freebsd.org (Postfix) with SMTP id 15D4E37B41C for ; Tue, 15 Jan 2002 08:34:32 -0800 (PST) Received: (qmail 2400 invoked from network); 15 Jan 2002 16:34:30 -0000 Received: from unknown (HELO stoli.inspinc.ad) (206.29.197.190) by mail1.infospace.com with SMTP; 15 Jan 2002 16:34:30 -0000 Received: (qmail 29077 invoked from network); 15 Jan 2002 16:34:30 -0000 Received: from gasket.inspinc.ad (HELO irishbreakfast.carrel.org) ([10.99.32.118]) (envelope-sender ) by stoli.inspinc.ad (qmail-ldap-1.03) with SMTP for ; 15 Jan 2002 16:34:30 -0000 Date: Tue, 15 Jan 2002 08:34:52 -0800 Reply-To: freebsd-questions@FreeBSD.ORG Mime-Version: 1.0 (Apple Message framework v480) Cc: freebsd-questions@FreeBSD.ORG Message-Id: In-Reply-To: <200201151526.g0FFQFX02180@grant.org> Content-Type: text/plain; charset=US-ASCII; format=flowed Subject: Re: kern/33904: secure mode bug From: William Carrel Content-Transfer-Encoding: 7bit To: FreeBSD@jovi.net X-Mailer: Apple Mail (2.480) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [This probably belongs to -questions.] On Tuesday, January 15, 2002, at 07:26 AM, FreeBSD@jovi.net wrote: > Thanks for your reply. > I suggest escalating the trouble. > Correct time is mission-critical on many systems > and this is an issue of unreliable service under FreeBSD. No. This is an issue of a user not reading the appropriate documentation before changing the securelevel and then being surprised when it exhibits exactly the behavior documented there. There are good reasons why time changes are clamped to one second under that securelevel. > A settimeofday other than that programmed is worse than doing nothing. > Three orders of magnitude is a complete failure by every reasonable > standard. > Breaking date, ntpdate, ntpd, ... is a reliable indication of severe > failure. > These programs now need rewriting to operate reliably. No. You need to run sync your clock before raising securelevel. Or keep your securelevel down below two. Of course, I'm sure you read this part of init(8)'s man page: 2 Highly secure mode - same as secure mode, plus disks may not be opened for writing (except by mount(2)) whether mounted or not. This level precludes tampering with filesystems by unmounting them, but also inhibits running newfs(8) while the system is multi-user. In addition, kernel time changes are restricted to less than or equal to one second. Attempts to change the time by more than this will log the message ``Time adjustment clamped to +1 second''. If you sync before raising securelevel and then run ntpd, unless your box has severe problems with clock drift (like NetBSD/mac68k 15mins/hour) it should stay in sync. Be sure not to adjust other knobs like securelevel without knowing what you're doing and consulting the appropriate manpages, it will save you lots of pain. Modifying these things to return ETIMEADJCLAMPED or some such seems a little silly, and would represent a pretty hairy delta into ntpd. -- Andy To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 9:22:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id C8CC037B405 for ; Tue, 15 Jan 2002 09:22:16 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id EAA03708; Wed, 16 Jan 2002 04:14:34 +1100 Date: Wed, 16 Jan 2002 04:15:43 +1100 (EST) From: Bruce Evans X-X-Sender: To: Cy Schubert - ITSD Open Systems Group Cc: Poul-Henning Kamp , Max Khon , Subject: Re: request for review In-Reply-To: <200201151434.g0FEYlK72569@cwsys.cwsent.com> Message-ID: <20020116035325.T2884-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 15 Jan 2002, Cy Schubert - ITSD Open Systems Group wrote: > In addition to making FreeBSD consistent with OpenBSD and Linux, as > stated by the originator of this thread, the patch also makes FreeBSD > consistent with Solaris and Tru64-UNIX. I'm for it. Please don't top post. This patch also makes FreeBSD almost compatible with FreeBSD (versions 1, 2, 3) that is. Old versions vn_stat() simply passed back the va_blocksize set by VOP_GETTATTR(). This at least gave individual filesystems a chance to set it right. It was too bad that they mostly set it wrong using slightly different hacks than the ones in the -current vn_stat(). The patch under review restores the old behaviour for non-disk character devices only. Unfortunately, it was character (and block) devices that had the most bogus values for va_blocksize. > In message <19068.1011092026@critter.freebsd.dk>, Poul-Henning Kamp > writes: > > > > I think it is bogus for devices other than disks to flout a > > va_blocksize, but I am on the other hand not sure what the > > relevant (if any) standards say. Please don't top post. POSIX says almost the same as the quote from www.opengroup.org in the sources: "A file system-specific [sic] preferred I/O block size for this object". 0 is wrong except possibly for /dev/null since it is a very bad I/O size. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 10:47:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id CDE4137B42A; Tue, 15 Jan 2002 10:47:26 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id FAA07957; Wed, 16 Jan 2002 05:47:24 +1100 Date: Wed, 16 Jan 2002 05:48:33 +1100 (EST) From: Bruce Evans X-X-Sender: To: "Brian F. Feldman" Cc: Poul-Henning Kamp , Max Khon , Subject: Re: request for review In-Reply-To: <200201151630.g0FGU5k77068@green.bikeshed.org> Message-ID: <20020116054555.H3135-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 15 Jan 2002, Brian F. Feldman wrote: > Posix specifies that st_size for non-regular files is "unspecified", however > it doesn't specify st_blksize at all. In the "spirit" of POSIX I'd say POSIX.1-2001 specifies st_blksize. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 12: 3:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from grant.org (grant.org [206.190.164.98]) by hub.freebsd.org (Postfix) with ESMTP id 4549237B417; Tue, 15 Jan 2002 12:03:43 -0800 (PST) Received: (from devon@localhost) by grant.org (8.11.6/8.11.6) id g0FK3gb04767; Tue, 15 Jan 2002 15:03:42 -0500 (EST) (envelope-from devon) Date: Tue, 15 Jan 2002 15:03:42 -0500 (EST) Message-Id: <200201152003.g0FK3gb04767@grant.org> From: FreeBSD@jovi.net To: freebsd-questions@FreeBSD.ORG Cc: FreeBSD@jovi.net, freebsd-arch@FreeBSD.ORG In-reply-to: (message from William Carrel on Tue, 15 Jan 2002 08:34:52 -0800) Subject: Re: kern/33904: secure mode bug References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG It would help to document or fix date/ntpdate/ntpd/... to warn of this "clock clamp" trouble or better yet fix settimeofday to return EPERM instead of trying to second-guess the intent of the programmer. If not, settimeofday documentation must clarify this silent lossage of setting a time other than what the calling program specified. If write were to sometimes read rather than report EROFS we'd expect that to be prominently and frequently documented! Cheers --Devon /"\ \ / ASCII Ribbon Campaign X Help Cure HTML Mail / \ PS: Yes, I read everything you quote before writing the initial bug report. While I was flogging ntpdate and tailing logs it was clear the kernel was clamping the clock but it offered no clue as to how or why. The fact that date/ntpdate/ntpd/... were silently failing was especially confusing. Silent failure can never be tolerated. Yes, my hardware has major clock drift. I am in the real world, I have more things to do than read up on every ill considered experiment that breaks stuff which used to work. I enabled no secure level, whoever configured the box did. Time was silently failing. At the very least, document trouble if you won't fix it. Date: Tue, 15 Jan 2002 08:34:52 -0800 Reply-To: freebsd-questions@FreeBSD.ORG From: William Carrel [This probably belongs to -questions.] On Tuesday, January 15, 2002, at 07:26 AM, FreeBSD@jovi.net wrote: > Thanks for your reply. > I suggest escalating the trouble. > Correct time is mission-critical on many systems > and this is an issue of unreliable service under FreeBSD. No. This is an issue of a user not reading the appropriate documentation before changing the securelevel and then being surprised when it exhibits exactly the behavior documented there. There are good reasons why time changes are clamped to one second under that securelevel. > A settimeofday other than that programmed is worse than doing nothing. > Three orders of magnitude is a complete failure by every reasonable > standard. > Breaking date, ntpdate, ntpd, ... is a reliable indication of severe > failure. > These programs now need rewriting to operate reliably. No. You need to run sync your clock before raising securelevel. Or keep your securelevel down below two. Of course, I'm sure you read this part of init(8)'s man page: 2 Highly secure mode - same as secure mode, plus disks may not be opened for writing (except by mount(2)) whether mounted or not. This level precludes tampering with filesystems by unmounting them, but also inhibits running newfs(8) while the system is multi-user. In addition, kernel time changes are restricted to less than or equal to one second. Attempts to change the time by more than this will log the message ``Time adjustment clamped to +1 second''. If you sync before raising securelevel and then run ntpd, unless your box has severe problems with clock drift (like NetBSD/mac68k 15mins/hour) it should stay in sync. Be sure not to adjust other knobs like securelevel without knowing what you're doing and consulting the appropriate manpages, it will save you lots of pain. Modifying these things to return ETIMEADJCLAMPED or some such seems a little silly, and would represent a pretty hairy delta into ntpd. -- Andy To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 12:27:32 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 74F6237B405 for ; Tue, 15 Jan 2002 12:27:29 -0800 (PST) Received: from dialup-209.244.107.183.dial1.sanjose1.level3.net ([209.244.107.183] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Qa93-0000Sf-00; Tue, 15 Jan 2002 12:25:22 -0800 Message-ID: <3C44902E.F9E0B6C@mindspring.com> Date: Tue, 15 Jan 2002 12:25:18 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Mats Lofkvist Cc: imp@village.org, bmilekic@technokratis.com, arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: <20020114105859.A24635@technokratis.com> <3C4305E5.65BB32A6@FreeBSD.org> <20020114114911.A24990@technokratis.com> <20020114.130705.84407599.imp@village.org> <3C439B25.27DCCB2D@mindspring.com> <20020115091235.23322.qmail@kairos.algonet.se> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mats Lofkvist wrote: > > The idea that some counter size will never be exceeded by > > technology by the time it is obsolete is what has people > > complaining about 32 bit counters in this thread already; > > are you prepared to make the same base assumption in the > > choice of 64 bit counters? A terabit network could push > > a 64 bit counter over in under 6 months. > > And it (a terabit network) would overflow the 1kB modular > counter solution in 35 seconds or so. Yes, I know, larger > blocks than 1kB should be used in the global counter, and > I guess multiple levels of (32-bit) modular counters can > be used to achieve higher limits, but will it still be > cheaper than just using a 64 bit counter? At that point, no. THat's the main reason I sugested counting something else instead of bytes. If you count something that doesn't increase with as steep a slope as the rate of increase of interface speeds (HP is sampling 10Gbit parts right now, even though the standard has not solidified because the optical people can't figure out how to get their parts that fast), all you do is put off the inevitable, so long as you keep the linear bound relationship between the measured values and the speed. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 12:33:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 0BC9537B417 for ; Tue, 15 Jan 2002 12:33:50 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0FKUqU28603; Tue, 15 Jan 2002 21:30:54 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Terry Lambert Cc: Mats Lofkvist , imp@village.org, bmilekic@technokratis.com, arch@FreeBSD.ORG Subject: Re: 64 bit counters again In-Reply-To: Your message of "Tue, 15 Jan 2002 12:25:18 PST." <3C44902E.F9E0B6C@mindspring.com> Date: Tue, 15 Jan 2002 21:30:52 +0100 Message-ID: <28601.1011126652@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <3C44902E.F9E0B6C@mindspring.com>, Terry Lambert writes: >THat's the main reason I sugested counting something else >instead of bytes. Terry, at this point I think we can conclude that this is not what the majority of people want. Now, please go away and stop wasting our time. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 12:37:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 8933E37B419; Tue, 15 Jan 2002 12:37:14 -0800 (PST) Received: from dialup-209.244.107.183.dial1.sanjose1.level3.net ([209.244.107.183] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QaKP-0000kj-00; Tue, 15 Jan 2002 12:37:05 -0800 Message-ID: <3C4492EE.5A60AD0B@mindspring.com> Date: Tue, 15 Jan 2002 12:37:02 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Michal Mertl Cc: Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Michal Mertl wrote: > I wrote it so it's just matter of one #define change to have all counters > switch to 32 or 64 bit. It also would be possible to have something like > atomic funtctions - different counter functions for various counter sizes, > maybe even something like "use counter at least 32 bits wide - If we need > atomicity on <586 we can't simply implement these ops" - I agree that some > counters would have sufficient precision and scale in 32 bit. If it is #if'ed (e.g. "#if (sizeof(long)>=8) || FORCE_64_BIT_CONUTERS"), then I have no problem with the code going in, so long as it is off by default on 32 bit machines. > I kind of measured the speed of just different addition implementation > (DUAL pIII cumine, ServerWorks and 440BX dual boards with SMP kernel) - [ ... ] > It seems you and others were right. SMP atomic implementations are a bit > expensive. But even the worst case 50 clocks for locked cmpxchg8b isn't > that bad but IMMV. Uh, that's a factor of 16 in the cases that I think most of us will care about, so I think MMDV. [ ... ] > 32 bit simple (compiler generates read-modify-write - wrong but used > throughout kernel!!!, only when counter variable is declared volatile > compiler generates addl change,(mem)): 3.00 Ugh. This looks like it needs to be fixed! Amazing what you find when you gice something a hard look... > --- > > I totally disagree with Terry that 64 bit counter is only slightly more > difficult to overflow then 32 bit. We will never (in near future - several > years) have terabit interfaces nor wirespeed 10Gbit (and 10Gbit fullduplex > would overflow in 500 years). If you don't like 64 bit, your fix (to count > in KB) only adds 10 bits of range, switch to 64 bit adds 32 bit. I > understand that global counter could be made 64 bit a the counter you're > actually counting on 32 bit so you would add 42 bits. That won't overflow > for sure :-). FWIW, HP is selling 10Gbit parts now; you can buy the cards from them online today, if you wanted to. It is very likely that we will see IA64 based networking hardware to support multiple ports on these things. 10 ports is 100Gbit. That puts us within one order of magnitude of terabit. Are you sure you want to challenge Moore's law like that? 8-). > I deeply apologise that the patch I posted was badly broken. I revamped > the code quite a lot before doing the diff and it resulted in lots of > cut-n-paste and similar errors :-(. Kernel&world build went without > problems but the code was wrong (in atomic.h and syscntr.h). New patch > will be available soon. If it conditionalizes the code so that the default disables 64 bit on 32 bit machines, I have no problem with it. If people really want 64 bit features, there's an easy fix for the problem: buy a 64 bit machine. Allowing people to use 64 bit features at 1500% inflation in cost is OK, if it is not on by default, and has to be willfully turned on (I would prefer that it has to be done by reading a warning first). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 12:43:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 1081F37B419 for ; Tue, 15 Jan 2002 12:43:19 -0800 (PST) Received: from dialup-209.244.107.183.dial1.sanjose1.level3.net ([209.244.107.183] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QaOM-0006Mo-00; Tue, 15 Jan 2002 12:41:10 -0800 Message-ID: <3C4493E3.5E322D19@mindspring.com> Date: Tue, 15 Jan 2002 12:41:07 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Max Khon , arch@FreeBSD.ORG Subject: Re: request for review References: <19068.1011092026@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > I think it is bogus for devices other than disks to flout a > va_blocksize, but I am on the other hand not sure what the > relevant (if any) standards say. > > If this patch makes things better, I'm all for it. Ugh. Hard call. I agree with you in principle, but looking at the "stat(2)" man page: u_int32_t st_blksize; /* optimal blocksize for I/O */ Sort of implies that you'll get useful information back for anything. For regular files, you'd expect this to be the FS block size, almost... 8-(. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 12:51:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from green.bikeshed.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 5A01737B416; Tue, 15 Jan 2002 12:51:09 -0800 (PST) Received: from localhost (green@localhost) by green.bikeshed.org (8.11.6/8.11.6) with ESMTP id g0FKp7l78189; Tue, 15 Jan 2002 15:51:08 -0500 (EST) (envelope-from green@green.bikeshed.org) Message-Id: <200201152051.g0FKp7l78189@green.bikeshed.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Bruce Evans Cc: "Brian F. Feldman" , Poul-Henning Kamp , Max Khon , arch@FreeBSD.ORG Subject: Re: request for review In-Reply-To: Your message of "Wed, 16 Jan 2002 05:48:33 +1100." <20020116054555.H3135-100000@gamplex.bde.org> From: "Brian F. Feldman" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 15 Jan 2002 15:51:07 -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bruce Evans wrote: > On Tue, 15 Jan 2002, Brian F. Feldman wrote: > > > Posix specifies that st_size for non-regular files is "unspecified", however > > it doesn't specify st_blksize at all. In the "spirit" of POSIX I'd say > > POSIX.1-2001 specifies st_blksize. Great, right after I get POSIX.1-1996 ;) So how does it specify st_blksize, then? -- Brian Fundakowski Feldman \ FreeBSD: The Power to Serve! / green@FreeBSD.org `------------------------------' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 12:57:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 85A4B37B405; Tue, 15 Jan 2002 12:57:05 -0800 (PST) Received: from dialup-209.244.107.183.dial1.sanjose1.level3.net ([209.244.107.183] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Qadh-0003K0-00; Tue, 15 Jan 2002 12:57:02 -0800 Message-ID: <3C44979A.63F600B0@mindspring.com> Date: Tue, 15 Jan 2002 12:56:58 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: FreeBSD@jovi.net Cc: cjc@FreeBSD.ORG, freebsd-gnats-submit@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug References: <200201142344.g0ENimK91227@freefall.freebsd.org> <20020115011230.D28767@blossom.cjclark.org> <200201151526.g0FFQFX02180@grant.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG FreeBSD@jovi.net wrote: > Thanks for your reply. > I suggest escalating the trouble. > Correct time is mission-critical on many systems Agreed. > and this is an issue of unreliable service under FreeBSD. Disagree: FreeBSD is ensuring the correctness of the time by disallowing a delta which is so large as to be useful to someone attempting to hack date stamps on files. If you want to "reliably" change the time by a large delta, then there are only three valid circumstances in which this should be done: 1) Initial installation of a system (in which case you have control over it to the level of being able to not set the secure mode up before you have made the adjustment). 2) You are the Pope, and you are changing the calendar again (I'd be happy to fix it for you, your eminence, particularly if I can let it be known you use FreeBSD). 3) You just got back from a relatavistic flight that resulted in the on board computer in your space ship (I will fix it for free for you, if I am allowed to do it on site, and take the vehicle out for a spin afterwards, to verify that the large delats are now allowed, when necessary). > A settimeofday other than that programmed is worse than doing nothing. Particularly if you are trying to hack someone... > Three orders of magnitude is a complete failure by every reasonable > standard. Breaking date, ntpdate, ntpd, ... is a reliable indication > of severe failure. These programs now need rewriting to operate reliably. I guess you've never used NTP when the drift exceeds an hour or so, without having intentially set your security mode to disable changes over one second? The change is refused if the delta exceeds a maximum delta time, as being "unreasonable" by the NTP code itself, and you have to manually synchronize the clock to get it "in the neighborhood" in the first place before it will start working again. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 13: 1:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from technokratis.com (modemcable099.144-201-24.mtl.mc.videotron.ca [24.201.144.99]) by hub.freebsd.org (Postfix) with ESMTP id D90A937B404 for ; Tue, 15 Jan 2002 13:01:45 -0800 (PST) Received: (from bmilekic@localhost) by technokratis.com (8.11.4/8.11.3) id g0FL3gl34638; Tue, 15 Jan 2002 16:03:42 -0500 (EST) (envelope-from bmilekic) Date: Tue, 15 Jan 2002 16:03:42 -0500 From: Bosko Milekic To: Poul-Henning Kamp Cc: Terry Lambert , Mats Lofkvist , imp@village.org, arch@FreeBSD.ORG Subject: Re: 64 bit counters again Message-ID: <20020115160342.A34519@technokratis.com> References: <3C44902E.F9E0B6C@mindspring.com> <28601.1011126652@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <28601.1011126652@critter.freebsd.dk>; from phk@critter.freebsd.dk on Tue, Jan 15, 2002 at 09:30:52PM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG OK, I think it's time to kill this discussion. I still think that counting bytes is pretty lame but if "most" (I would hardly call it "most," but whatever) people want a 64-bit counter on a 32-bit architecture just to do this, then whatever. When I posted my original concerns, it was only because I wanted to avoid having a huge sweep over all the network code, converting all counters to 64-bit - which would be ridiculous. If it's only a matter of one or two counters, as DG appeared to suggest in an earlier post, let's just drop the discussion and you can do/have someone else do whatever pleases "most" people. On Tue, Jan 15, 2002 at 09:30:52PM +0100, Poul-Henning Kamp wrote: > In message <3C44902E.F9E0B6C@mindspring.com>, Terry Lambert writes: > > >THat's the main reason I sugested counting something else > >instead of bytes. > > Terry, at this point I think we can conclude that this is not > what the majority of people want. > > Now, please go away and stop wasting our time. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. -- Bosko Milekic bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 13: 7:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id DBF0037B400 for ; Tue, 15 Jan 2002 13:07:36 -0800 (PST) Received: from dialup-209.244.107.183.dial1.sanjose1.level3.net ([209.244.107.183] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Qalt-0000QN-00; Tue, 15 Jan 2002 13:05:29 -0800 Message-ID: <3C449995.E91FDA27@mindspring.com> Date: Tue, 15 Jan 2002 13:05:25 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bruce Evans Cc: Cy Schubert - ITSD Open Systems Group , Poul-Henning Kamp , Max Khon , arch@FreeBSD.ORG Subject: Re: request for review References: <20020116035325.T2884-100000@gamplex.bde.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bruce Evans wrote: > POSIX says almost the same as the quote from www.opengroup.org in the > sources: "A file system-specific [sic] preferred I/O block size for > this object". 0 is wrong except possibly for /dev/null since it is > a very bad I/O size. Hahahaha! What if the FS prefers you to not read or write the object? 8-) 8-) -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 13:22: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 9A2B037B400 for ; Tue, 15 Jan 2002 13:21:54 -0800 (PST) Received: from dialup-209.244.107.183.dial1.sanjose1.level3.net ([209.244.107.183] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Qb1N-0001X7-00; Tue, 15 Jan 2002 13:21:29 -0800 Message-ID: <3C449D56.696CBA9A@mindspring.com> Date: Tue, 15 Jan 2002 13:21:26 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Mats Lofkvist , imp@village.org, bmilekic@technokratis.com, arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: <28601.1011126652@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > >THat's the main reason I sugested counting something else > >instead of bytes. > > Terry, at this point I think we can conclude that this is not > what the majority of people want. > > Now, please go away and stop wasting our time. Apparently you didn't realize that "suggested" is the past tense of "suggest", and that I wasn't in the process of suggesting it right then. PS: THanks for throwing away everything else and taking that one sentence out of context; I'm sure you've clarified for everyone that that was the incredibly mist important sentence in the posting to which you are replying, instead of just a side note in response to Mats statement. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 13:33:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id E989537B41C for ; Tue, 15 Jan 2002 13:33:54 -0800 (PST) Received: (qmail 6100 invoked from network); 15 Jan 2002 21:33:54 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([64.81.54.73]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 15 Jan 2002 21:33:54 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <3C449D56.696CBA9A@mindspring.com> Date: Tue, 15 Jan 2002 13:33:10 -0800 (PST) From: John Baldwin To: Terry Lambert Subject: Re: 64 bit counters again Cc: arch@FreeBSD.ORG, bmilekic@technokratis.com, imp@village.org, Mats Lofkvist , Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 15-Jan-02 Terry Lambert wrote: > Poul-Henning Kamp wrote: >> >THat's the main reason I sugested counting something else >> >instead of bytes. >> >> Terry, at this point I think we can conclude that this is not >> what the majority of people want. >> >> Now, please go away and stop wasting our time. > > Apparently you didn't realize that "suggested" is the past > tense of "suggest", and that I wasn't in the process of > suggesting it right then. > > PS: THanks for throwing away everything else and taking that > one sentence out of context; I'm sure you've clarified for > everyone that that was the incredibly mist important sentence > in the posting to which you are replying, instead of just a > side note in response to Mats statement. FWIW, I agree that doing 64-bit counters on IA32 is more of a pain than its worth, so I think it's not quite the Terry vs. the world some people would like to believe. Terry is trying to get people to use solutions that scale. Bumping the size of the counter for bytes doesn't scale, it just postpones the problem a little. If you really want a 64-bit counter, go get an alpha or when 5.0 comes out an ultrasparc or ia64 box. > -- Terry -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 13:34:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from swan.prod.itd.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by hub.freebsd.org (Postfix) with ESMTP id 5E81937B41A; Tue, 15 Jan 2002 13:34:07 -0800 (PST) Received: from dialup-209.245.128.158.dial1.sanjose1.level3.net ([209.245.128.158] helo=blossom.cjclark.org) by swan.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QbDM-0000T4-00; Tue, 15 Jan 2002 13:33:55 -0800 Received: (from cjc@localhost) by blossom.cjclark.org (8.11.6/8.11.3) id g0FLXdd32181; Tue, 15 Jan 2002 13:33:39 -0800 (PST) (envelope-from cjc) Date: Tue, 15 Jan 2002 13:33:39 -0800 From: "Crist J . Clark" To: FreeBSD@jovi.net Cc: freebsd-gnats-submit@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug Message-ID: <20020115133339.A31328@blossom.cjclark.org> Reply-To: cjclark@alum.mit.edu References: <200201142344.g0ENimK91227@freefall.freebsd.org> <20020115011230.D28767@blossom.cjclark.org> <200201151526.g0FFQFX02180@grant.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200201151526.g0FFQFX02180@grant.org>; from FreeBSD@jovi.net on Tue, Jan 15, 2002 at 10:26:15AM -0500 X-URL: http://people.freebsd.org/~cjc/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jan 15, 2002 at 10:26:15AM -0500, FreeBSD@jovi.net wrote: > Thanks for your reply. > I suggest escalating the trouble. > Correct time is mission-critical on many systems > and this is an issue of unreliable service under FreeBSD. > > A settimeofday other than that programmed is worse than doing nothing. > Three orders of magnitude is a complete failure by every reasonable standard. > Breaking date, ntpdate, ntpd, ... is a reliable indication of severe failure. The behavior is specifically designed not to break something like ntpd(8) which only makes tiny changes to the system time. > These programs now need rewriting to operate reliably. At securelevel(8) >= 2, the ability to change the clock by more than +/-1 s is removed for security reasons. Things like date(1) and ntpdate(8) _will_ be broken. The only question is, how to best handle the breakage? There are basically two approaches, 1) Have system calls that request an illegal time change completely fail. 2) Have system calls that request an illegal time change have the time changed as much as allowed and not return failure. When calls to change system time fail in option (1), we may see programs bailing out or reporting strange failures (failure to set the time is not usually expected). When option (2) is being used, programs will not be bailing out because things failed, but we might get odd behavior... and I think we are more likely to have some confused users in case (2). Finally, nothing about this prevents accurate timekeeping. First, if this is a really big problem for you, don't run at securelevel(8) >= 2. By design, securelevel(8) breaks stuff. If it breaks things you want or need, don't use it. If you do want accurate timekeeping and securelevel(8), do something like run ntpdate(8) during system startup before securelevel(8) is raised and run ntpd(8). Generally, ntpd(8)'s time adjustments will not be hampered by securelevel(8) settings. > Date: Tue, 15 Jan 2002 01:12:30 -0800 > From: "Crist J . Clark" > Cc: freebsd-gnats-submit@FreeBSD.ORG > > On Mon, Jan 14, 2002 at 03:44:48PM -0800, D. McCullough wrote: > [snip] > > > >Description: > > settimeofday silently fails in secure mode. > > > > I wasted a lot of time baffled -- > > this makes all the various date > > apps and clients silently fail. > > The problem is that it doesn't _completely_ fail. > > > >Fix: > > When an operation prohibited by secure mode fails, > > this failure must be accurately reported by the API. > > If you run your test program, > > # ./testtime > gettimeofday = 0; t = 1011071149.590024 > settimeofday = 0; t = 1011067549.590024 > gettimeofday = 0; t = 1011071148.591402 > > At securelevel == 2, we see that the time _has_ been changed by the > call, but like the log message says, > > Jan 14 21:05:44 bubbles kernel: Time adjustment clamped to -1 second > > So the call of settimeofday(2) didn't actually fail due to "permission > denied," but the result is not exactly what the caller expected either. > > Now, whether this whole approach is wrong is more of a discussion for > freebsd-arch@freebsd.org. -- "It's always funny until someone gets hurt. Then it's hilarious." Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 13:44:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id AE40437B416 for ; Tue, 15 Jan 2002 13:44:10 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id HAA01765; Wed, 16 Jan 2002 07:26:03 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37641) with ESMTP id <01KD4PV219U8VFMGZL@cim.alcatel.com.au>; Wed, 16 Jan 2002 07:26:19 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.6/8.11.6) id g0FKQ0572158; Wed, 16 Jan 2002 07:26:00 +1100 Content-return: prohibited Date: Wed, 16 Jan 2002 07:26:00 +1100 From: Peter Jeremy Subject: Re: 64 bit counters again In-reply-to: <20020114201018.GA64041@voi.aagh.net>; from tom.hurst@clara.net on Mon, Jan 14, 2002 at 08:10:18PM +0000 To: Thomas Hurst Cc: arch@FreeBSD.ORG Mail-Followup-To: Thomas Hurst , arch@FreeBSD.ORG Message-id: <20020116072600.A72142@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <3C41F3FD.4ECC8CD@mindspring.com> <20020113231459.GA30349@voi.aagh.net> <3C42390A.F9E9F533@mindspring.com> <3C42E899.CB21BD0A@FreeBSD.org> <3C431EE5.1CFF557B@mindspring.com> <20020114201018.GA64041@voi.aagh.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Jan 14, 2002 at 08:10:18PM +0000, Thomas Hurst wrote: >I'm just interested in how much bandwidth I'm using, but if the counters >are going to overflow this easily, they cease being terribly useful, >unless I'm going to actively count them and watch for overflows in >userspace. All this needs is a counter big enough to represent the number of bits/bytes/whatever transmitted/received/whatever during the sample period. It doesn't matter if the absolute counter value overflows during the sample period (as long as it doesn't overflow more than once). [This assumes an unsigned 2's complement representation with arithmetic overflows ignored - which covers virtually all current hardware]. For example, the following function will report the average bandwidth since it was last called: static ulong prev_val; struct { ulong low, high; } total_val; double bytes_per_second() { static timeval prev_time; timeval cur_time; ulong cur_val; double bps; cur_val = read_counter(); gettimeofday(&cur_time, NULL); bps = (double)(cur_val - prev_val) / (cur_time.tv_sec - prev_time.tv_sec + (cur_time.tv_usec - prev_time.tv_usec) / 1e6); #ifdef CALCULATE_TOTAL_VALUE total_val.low = cur_val; if (cur_val < prev_val) total_val.high++; #endif prev_time = cur_time; prev_val = cur_val; return (bps); } The optional code inside CALCULATE_TOTAL_VALUE will give you an accurate total units count with an extended range as long as the sample rate is high enough. >I'm not really bothered if they're totally accurate, since they're just >a guide - how about a float that counts kb? ;) Two major downsides: 1) The FreeBSD kernel (as well as most other Un*x kernels) avoid the use of FP for various reasons. The use of FP within interrupt handlers would seriously affect interrupt latency - by a factor or two or more. 2) FP trades accuracy for range. After a while your counters will lose more and more precision and eventually stop counting at all: 33554432.0f + 1.0f == 33554432.0f Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 14: 2:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id C8A2B37B400; Tue, 15 Jan 2002 14:02:13 -0800 (PST) Received: from gateway.posi.net ([12.236.90.177]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020115220208.PZFF5944.rwcrmhc51.attbi.com@gateway.posi.net>; Tue, 15 Jan 2002 22:02:08 +0000 Received: from localhost (kbyanc@localhost) by gateway.posi.net (8.11.6/8.11.6) with ESMTP id g0FM26C21862; Tue, 15 Jan 2002 14:02:06 -0800 (PST) (envelope-from kbyanc@posi.net) X-Authentication-Warning: gateway.posi.net: kbyanc owned process doing -bs Date: Tue, 15 Jan 2002 14:02:06 -0800 (PST) From: Kelly Yancey To: John Baldwin Cc: Terry Lambert , arch@FreeBSD.ORG, bmilekic@technokratis.com, imp@village.org, Mats Lofkvist , Poul-Henning Kamp Subject: Re: 64 bit counters again In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 15 Jan 2002, John Baldwin wrote: > FWIW, I agree that doing 64-bit counters on IA32 is more of a pain > than its worth, so I think it's not quite the Terry vs. the world > some people would like to believe. Terry is trying to get people > to use solutions that scale. Bumping the size of the counter for > bytes doesn't scale, it just postpones the problem a little. > > If you really want a 64-bit counter, go get an alpha or when 5.0 > comes out an ultrasparc or ia64 box. > I'm sorry, but that's akin to saying that implementing SMP on i386 is more of a pain than it is worth and if you really want a good SMP implementation you should just buy a box from Sun. A legitimate engineering problem was presented, presumably there is a legitimate engineering solution. Finding one that everyone can agree on is another thing entirely... :) Kelly To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 14: 2:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from grant.org (grant.org [206.190.164.98]) by hub.freebsd.org (Postfix) with ESMTP id 5A10C37B400; Tue, 15 Jan 2002 14:02:49 -0800 (PST) Received: (from devon@localhost) by grant.org (8.11.6/8.11.6) id g0FM2lE05944; Tue, 15 Jan 2002 17:02:47 -0500 (EST) (envelope-from devon) Date: Tue, 15 Jan 2002 17:02:47 -0500 (EST) Message-Id: <200201152202.g0FM2lE05944@grant.org> From: Devon@jovi.net To: cjclark@alum.mit.edu Cc: FreeBSD@jovi.net, freebsd-gnats-submit@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG In-reply-to: <20020115133339.A31328@blossom.cjclark.org> (cristjc@earthlink.net) Subject: Re: kern/33904: secure mode bug References: <200201142344.g0ENimK91227@freefall.freebsd.org> <20020115011230.D28767@blossom.cjclark.org> <200201151526.g0FFQFX02180@grant.org> <20020115133339.A31328@blossom.cjclark.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG (failure to set the time is not usually expected) Programs seem to handle it ok when user is not root. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 14:26:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id 1A75337B416 for ; Tue, 15 Jan 2002 14:26:31 -0800 (PST) Received: (qmail 16194 invoked from network); 15 Jan 2002 22:26:29 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([64.81.54.73]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 15 Jan 2002 22:26:29 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Tue, 15 Jan 2002 14:25:46 -0800 (PST) From: John Baldwin To: Kelly Yancey Subject: Re: 64 bit counters again Cc: Poul-Henning Kamp , Mats Lofkvist , imp@village.org, bmilekic@technokratis.com, arch@FreeBSD.ORG, Terry Lambert Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 15-Jan-02 Kelly Yancey wrote: > On Tue, 15 Jan 2002, John Baldwin wrote: > >> FWIW, I agree that doing 64-bit counters on IA32 is more of a pain >> than its worth, so I think it's not quite the Terry vs. the world >> some people would like to believe. Terry is trying to get people >> to use solutions that scale. Bumping the size of the counter for >> bytes doesn't scale, it just postpones the problem a little. >> >> If you really want a 64-bit counter, go get an alpha or when 5.0 >> comes out an ultrasparc or ia64 box. >> > > I'm sorry, but that's akin to saying that implementing SMP on i386 is more > of a pain than it is worth and if you really want a good SMP implementation > you should just buy a box from Sun. A legitimate engineering problem was > presented, presumably there is a legitimate engineering solution. Finding one > that everyone can agree on is another thing entirely... :) Yes, the legit solution is not to just band-aid the problem by bumping your counter size. Peter Jeremy's post which highlighted the type of implementation Terry was talking about is quite straightforward. We can get away with bumping time_t when the time comes because time dosen't increase exponentially, but hardware speeds do increease exponentially. Might as well solve the problem for real. :) > Kelly -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 14:48:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from camelot.spsl.nsc.ru (camelot.spsl.nsc.ru [194.226.174.80]) by hub.freebsd.org (Postfix) with ESMTP id 53F2437B404 for ; Tue, 15 Jan 2002 14:48:27 -0800 (PST) Received: (from root@localhost) by camelot.spsl.nsc.ru (8.11.6/8.11.6) id g0FMmE394699 for arch@FreeBSD.ORG.AVP; Wed, 16 Jan 2002 04:48:14 +0600 (NOVT) (envelope-from fjoe@iclub.nsu.ru) Received: from iclub.nsu.ru (root@iclub.nsu.ru [193.124.222.66]) by camelot.spsl.nsc.ru (8.11.6/8.11.6) with ESMTP id g0FMmBJ94691; Wed, 16 Jan 2002 04:48:11 +0600 (NOVT) (envelope-from fjoe@iclub.nsu.ru) Received: (from fjoe@localhost) by iclub.nsu.ru (8.11.6/8.11.6) id g0FI69u11161; Wed, 16 Jan 2002 00:06:09 +0600 (NS) (envelope-from fjoe) Date: Wed, 16 Jan 2002 00:06:08 +0600 From: Max Khon To: Bruce Evans Cc: Cy Schubert - ITSD Open Systems Group , Poul-Henning Kamp , arch@FreeBSD.ORG Subject: Re: request for review Message-ID: <20020116000607.B10938@iclub.nsu.ru> References: <200201151434.g0FEYlK72569@cwsys.cwsent.com> <20020116035325.T2884-100000@gamplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020116035325.T2884-100000@gamplex.bde.org>; from bde@zeta.org.au on Wed, Jan 16, 2002 at 04:15:43AM +1100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG hi, there! On Wed, Jan 16, 2002 at 04:15:43AM +1100, Bruce Evans wrote: > This patch also makes FreeBSD almost compatible with FreeBSD (versions > 1, 2, 3) that is. Old versions vn_stat() simply passed back the > va_blocksize set by VOP_GETTATTR(). NetBSD/OpenBSD do exactly the same > > > I think it is bogus for devices other than disks to flout a > > > va_blocksize, but I am on the other hand not sure what the > > > relevant (if any) standards say. > > POSIX says almost the same as the quote from www.opengroup.org in the > sources: "A file system-specific [sic] preferred I/O block size for > this object". 0 is wrong except possibly for /dev/null since it is > a very bad I/O size. do you think we should remove setting sb->st_blksize to 0 in vn_stat() at all? /fjoe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 17: 6:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id B5B4E37B400; Tue, 15 Jan 2002 17:06:23 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id MAA01630; Wed, 16 Jan 2002 12:06:17 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37640) with ESMTP id <01KD4ZN364F4VM9P2R@cim.alcatel.com.au>; Wed, 16 Jan 2002 12:06:14 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.6/8.11.6) id g0G16BJ73440; Wed, 16 Jan 2002 12:06:11 +1100 Content-return: prohibited Date: Wed, 16 Jan 2002 12:06:11 +1100 From: Peter Jeremy Subject: Re: 64 bit counters again In-reply-to: <3C4492EE.5A60AD0B@mindspring.com>; from tlambert2@mindspring.com on Tue, Jan 15, 2002 at 12:37:02PM -0800 To: Terry Lambert Cc: Michal Mertl , Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.ORG Mail-Followup-To: Terry Lambert , Michal Mertl , Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.ORG Message-id: <20020116120611.A72285@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <3C4492EE.5A60AD0B@mindspring.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-Jan-15 12:37:02 -0800, Terry Lambert wrote: >Michal Mertl wrote: >> I wrote it so it's just matter of one #define change to have all counters >> switch to 32 or 64 bit. > >If it is #if'ed (e.g. "#if (sizeof(long)>=8) || FORCE_64_BIT_CONUTERS"), >then I have no problem with the code going in, so long as it is off by >default on 32 bit machines. I tend to agree with this point. Most people probably don't care about "bytes since reboot" and average bandwidth calculations don't need more than 32 bits for most installations. (Gig-Ethernet interfaces running close to wire speed on an IA32 is not a common configuration and I doubt it ever will be). >> I kind of measured the speed of just different addition implementation >> (DUAL pIII cumine, ServerWorks and 440BX dual boards with SMP kernel) - >[ ... ] >> It seems you and others were right. SMP atomic implementations are a bit >> expensive. But even the worst case 50 clocks for locked cmpxchg8b isn't >> that bad but IMMV. > >Uh, that's a factor of 16 in the cases that I think most of >us will care about, so I think MMDV. Comparing an unlocked 32-bit operation to a locked 64-bit SMP operation is unrealistic. The comparison is either ~2.7:1 (simple) or ~2.4:1 (locked) - neither of which are outrageous. Also, you need to take into account the impact of this change on the overall times - I suspect the extra ~30 cycles for the locked case is still only a small fraction of the total processing time. Note that in the UP case, you don't need atomic ops to access or update a multi-word object. The only requirements are that RMW primitives are used and the words are always updated in the same order (logically, LSW to MSW) - ie addl change,(mem);adcl 0,4(mem). For the update case "addl, interrupt, addl, adcl, reti, adcl" is safe. The counter value in memory will be out by 2^32 during the interrupt but that doesn't matter to the interrupt and will be corrected by the second adcl. For the read case, the reader uses something like: loop: movl 4(mem),%edx movl (mem),%eax cmpl 4(mem),%edx jnz loop If an interrupt updates the MSW then you take another pass around the loop, otherwise you always read the correct value. For the SMP case, you either need to use locks or you need to use per-CPU counters. (And the per-CPU counters can be read by another CPU using the above trick). >FWIW, HP is selling 10Gbit parts now; you can buy the cards >from them online today, if you wanted to. It is very likely >that we will see IA64 based networking hardware to support >multiple ports on these things. 10 ports is 100Gbit. That >puts us within one order of magnitude of terabit. IA64 is 64-bit. The current argument is about 64-bit operations on a 32-bit machine. Multi-port 10Gbit cards are going to need a re-designed bus as well. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 17:22:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id 014FE37B400 for ; Tue, 15 Jan 2002 17:22:21 -0800 (PST) Received: (qmail 18344 invoked from network); 16 Jan 2002 01:22:20 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([64.81.54.73]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 16 Jan 2002 01:22:20 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020116120611.A72285@gsmx07.alcatel.com.au> Date: Tue, 15 Jan 2002 17:21:37 -0800 (PST) From: John Baldwin To: Peter Jeremy Subject: Re: 64 bit counters again Cc: arch@FreeBSD.ORG, Thomas Hurst , "James E. Housley" , Bosko Milekic , Michal Mertl , Terry Lambert Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 16-Jan-02 Peter Jeremy wrote: > For the read case, the reader uses something like: > > loop: movl 4(mem),%edx > movl (mem),%eax > cmpl 4(mem),%edx > jnz loop > > If an interrupt updates the MSW then you take another pass around the > loop, otherwise you always read the correct value. > > For the SMP case, you either need to use locks or you need to use > per-CPU counters. (And the per-CPU counters can be read by another > CPU using the above trick). Well, SMP on Pentium's maybe, but not on Alpha, sparc64, or ia64, all of which support OOE and looser memory models than x86, meaning that you really need locks unless you are going to have 386-specific code all over the place. I suppose you can wrap it behind an MI API but that seems like a lot of work for fairly small gain that can end up making the code uglier. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 18:25: 8 2002 Delivered-To: freebsd-arch@freebsd.org Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by hub.freebsd.org (Postfix) with ESMTP id 00C1437B405; Tue, 15 Jan 2002 18:25:06 -0800 (PST) Received: (from dan@localhost) by dan.emsphone.com (8.11.6/8.11.6) id g0G2OfD58828; Tue, 15 Jan 2002 20:24:41 -0600 (CST) (envelope-from dan) Date: Tue, 15 Jan 2002 20:24:40 -0600 From: Dan Nelson To: Peter Jeremy Cc: Terry Lambert , Michal Mertl , Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.ORG Subject: Re: 64 bit counters again Message-ID: <20020116022440.GB71726@dan.emsphone.com> References: <3C4492EE.5A60AD0B@mindspring.com> <20020116120611.A72285@gsmx07.alcatel.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020116120611.A72285@gsmx07.alcatel.com.au> User-Agent: Mutt/1.3.25i X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Jan 16), Peter Jeremy said: > I tend to agree with this point. Most people probably don't care > about "bytes since reboot" and average bandwidth calculations don't > need more than 32 bits for most installations. (Gig-Ethernet > interfaces running close to wire speed on an IA32 is not a common > configuration and I doubt it ever will be). You don't even need gigabit. Plain old fast Ethernet at wire speed will roll over its byte count every 6 minutes, which means that for simple monitoring purposes, you have to sample at least every 5 minutes. Anyhow, I thought we had a solution to this "64-bit counters are slow on SMP" issue; per-cpu counters with aggregation at sysctl-query time. -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 20:54:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id EF9CC37B402 for ; Tue, 15 Jan 2002 20:54:10 -0800 (PST) Received: from dialup-209.245.128.158.dial1.sanjose1.level3.net ([209.245.128.158] helo=blossom.cjclark.org) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Qi5C-0004tH-00; Tue, 15 Jan 2002 20:53:55 -0800 Received: (from cjc@localhost) by blossom.cjclark.org (8.11.6/8.11.3) id g0G4rPT33431; Tue, 15 Jan 2002 20:53:25 -0800 (PST) (envelope-from cjc) Date: Tue, 15 Jan 2002 20:53:21 -0800 From: "Crist J . Clark" To: Devon@jovi.net Cc: FreeBSD@jovi.net, freebsd-arch@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug Message-ID: <20020115205321.D31328@blossom.cjclark.org> References: <200201142344.g0ENimK91227@freefall.freebsd.org> <20020115011230.D28767@blossom.cjclark.org> <200201151526.g0FFQFX02180@grant.org> <20020115133339.A31328@blossom.cjclark.org> <200201152202.g0FM2lE05944@grant.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200201152202.g0FM2lE05944@grant.org>; from Devon@jovi.net on Tue, Jan 15, 2002 at 05:02:47PM -0500 X-URL: http://people.freebsd.org/~cjc/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jan 15, 2002 at 05:02:47PM -0500, Devon@jovi.net wrote: > (failure to set the time is not usually expected) > > Programs seem to handle it ok when user is not root. If the program checks if it is running under the superuser uid on its own, it might not have exceptions to deal with these types of problems. It's also fun to have programs tell you you need to be root to do something when you are root. -- "It's always funny until someone gets hurt. Then it's hilarious." Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 20:57:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 0C1A437B405; Tue, 15 Jan 2002 20:57:11 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id PAA11368; Wed, 16 Jan 2002 15:57:08 +1100 Date: Wed, 16 Jan 2002 15:58:19 +1100 (EST) From: Bruce Evans X-X-Sender: To: "Brian F. Feldman" Cc: Poul-Henning Kamp , Max Khon , Subject: Re: request for review In-Reply-To: <200201152051.g0FKp7l78189@green.bikeshed.org> Message-ID: <20020116155420.N487-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 15 Jan 2002, Brian F. Feldman wrote: > Bruce Evans wrote: > > POSIX.1-2001 specifies st_blksize. > > Great, right after I get POSIX.1-1996 ;) So how does it specify st_blksize, > then? The same as the quote from www.opengroup.org in the comment in vn_stat(). It says nothing important that isn't in that quote. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 21: 3:57 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 184F637B419; Tue, 15 Jan 2002 21:03:47 -0800 (PST) Received: from dialup-209.245.128.158.dial1.sanjose1.level3.net ([209.245.128.158] helo=blossom.cjclark.org) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QiEY-00037r-00; Tue, 15 Jan 2002 21:03:38 -0800 Received: (from cjc@localhost) by blossom.cjclark.org (8.11.6/8.11.3) id g0G534P33468; Tue, 15 Jan 2002 21:03:04 -0800 (PST) (envelope-from cjc) Date: Tue, 15 Jan 2002 21:03:03 -0800 From: "Crist J . Clark" To: FreeBSD@jovi.net Cc: freebsd-questions@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG, bug-followup@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug Message-ID: <20020115210303.E31328@blossom.cjclark.org> References: <200201152003.g0FK3gb04767@grant.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200201152003.g0FK3gb04767@grant.org>; from FreeBSD@jovi.net on Tue, Jan 15, 2002 at 03:03:42PM -0500 X-URL: http://people.freebsd.org/~cjc/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jan 15, 2002 at 03:03:42PM -0500, FreeBSD@jovi.net wrote: > It would help to document or fix date/ntpdate/ntpd/... ntpdate(8) and ntpd(8) are contributed software. As far as I can tell, FreeBSD uses "virgin" imports. There is no local customization of the code. Maintaining local modifications to code maintained by outside vendors is expensive and therefore needs a lot of justification. > to warn of this "clock clamp" trouble You do get a log message generated at . That tends to be something people notice. > or better yet > fix settimeofday to return EPERM instead of trying > to second-guess the intent of the programmer. That might be a valid option. Would you be willing to patch settime() (in sys/kern/kern_time.c)? Would you be willing to test patches to see if they don't break things even worse? > If not, settimeofday documentation must clarify this silent lossage > of setting a time other than what the calling program specified. This is true. The statements in settimeofday(2) about "secure mode" are actually incorrect. I'll commit a fix shortly. -- "It's always funny until someone gets hurt. Then it's hilarious." Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 21:13:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id 2BB8537B405; Tue, 15 Jan 2002 21:13:07 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id QAA03959; Wed, 16 Jan 2002 16:12:55 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37641) with ESMTP id <01KD58A8B3WWVFKCPE@cim.alcatel.com.au>; Wed, 16 Jan 2002 16:13:10 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.6/8.11.6) id g0G5Cp374428; Wed, 16 Jan 2002 16:12:51 +1100 Content-return: prohibited Date: Wed, 16 Jan 2002 16:12:51 +1100 From: Peter Jeremy Subject: Re: 64 bit counters again In-reply-to: ; from jhb@FreeBSD.org on Tue, Jan 15, 2002 at 05:21:37PM -0800 To: John Baldwin Cc: arch@FreeBSD.org, Thomas Hurst , "James E. Housley" , Bosko Milekic , Michal Mertl , Terry Lambert Mail-Followup-To: John Baldwin , arch@FreeBSD.org, Thomas Hurst , "James E. Housley" , Bosko Milekic , Michal Mertl , Terry Lambert Message-id: <20020116161251.E72285@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <20020116120611.A72285@gsmx07.alcatel.com.au> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-Jan-15 17:21:37 -0800, John Baldwin wrote: > >On 16-Jan-02 Peter Jeremy wrote: >> For the read case, the reader uses something like: >> >> loop: movl 4(mem),%edx >> movl (mem),%eax >> cmpl 4(mem),%edx >> jnz loop >> >> If an interrupt updates the MSW then you take another pass around the >> loop, otherwise you always read the correct value. >> >> For the SMP case, you either need to use locks or you need to use >> per-CPU counters. (And the per-CPU counters can be read by another >> CPU using the above trick). My parenthetical comment also neglects the effect of per-CPU caches :-(. >Well, SMP on Pentium's maybe, but not on Alpha, sparc64, or ia64, all of which >support OOE and looser memory models than x86, Good point. The multi-precision counters part is probably only relevant to IA32 (at least until we need 128-bit counters), but combining per-CPU counters will be needed in all SMP cases. I think I'll retract my suggestion that it's possible to read the counters on any CPU without locking :-). Locking per-CPU counters defeats most of the justification for having per-CPU counters in the first place. (There is still some advantage - the counters hopefully stay in the local CPU cache instead of continually jumping between caches). How about the following: Have a per-CPU thread to read the per-CPU counters and update them into common counters. This could be done either by having the counter read operation schedule a read on each CPU and then returning the total, or having a "counter update" thread that runs in each CPU every (say) second [with a degree of time skew] and does locked updates into a common central counter. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 21:30: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 99C1337B417 for ; Tue, 15 Jan 2002 21:29:53 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id QAA13871; Wed, 16 Jan 2002 16:17:02 +1100 Date: Wed, 16 Jan 2002 16:18:12 +1100 (EST) From: Bruce Evans X-X-Sender: To: Max Khon Cc: Cy Schubert - ITSD Open Systems Group , Poul-Henning Kamp , Subject: Re: request for review In-Reply-To: <20020116000607.B10938@iclub.nsu.ru> Message-ID: <20020116155828.T487-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002, Max Khon wrote: > On Wed, Jan 16, 2002 at 04:15:43AM +1100, Bruce Evans wrote: > > > This patch also makes FreeBSD almost compatible with FreeBSD (versions > > 1, 2, 3) that is. Old versions vn_stat() simply passed back the > > va_blocksize set by VOP_GETTATTR(). > > NetBSD/OpenBSD do exactly the same The main point of the differences in FreeBSD is to set st_blksize to a reasonable value for disks (since individual filesystems didn't do it). The changes really shouldn't have affected non-disks. > > > > I think it is bogus for devices other than disks to flout a > > > > va_blocksize, but I am on the other hand not sure what the > > > > relevant (if any) standards say. > > > > POSIX says almost the same as the quote from www.opengroup.org in the > > sources: "A file system-specific [sic] preferred I/O block size for > > this object". 0 is wrong except possibly for /dev/null since it is > > a very bad I/O size. > > do you think we should remove setting sb->st_blksize to 0 in vn_stat() at all? Yes. We should also fix any filesystems that set va_blocksize to 0, or convert a va_blocksize of 0 to a more sensible st_blksize. I just noticed the following comment in vn_stat(): * Default to zero to catch bogus uses of this field. This causes many bogus block sizes in the same way that a fixed default in the kernel does. An st_blksize of 0 means stdio's BUFSIZ in many cases, since stdio prefers to use st_blksize but has to invent a block size if st_blksize is not useful. BUFSIZ is 1024, which is rather small these days. It's good for slow cdevs but not much else. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 22:29:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from enigma.trueimpact.net (enigma.trueimpact.net [209.82.45.201]) by hub.freebsd.org (Postfix) with ESMTP id 27E9637B402 for ; Tue, 15 Jan 2002 22:29:19 -0800 (PST) Received: from spirit.jaded.net (unknown [24.141.6.76]) by enigma.trueimpact.net (Postfix) with ESMTP id 9D67866B09 for ; Wed, 16 Jan 2002 01:29:09 -0500 (EST) Received: (from dan@localhost) by spirit.jaded.net (8.11.6/8.11.6) id g0G6ToY09921 for arch@freebsd.org; Wed, 16 Jan 2002 01:29:50 -0500 (EST) (envelope-from dan) Date: Wed, 16 Jan 2002 01:29:50 -0500 From: Dan Moschuk To: arch@freebsd.org Subject: M_SHORTWAIT, the md driver and errno Message-ID: <20020116012950.C7831@spirit.jaded.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I'm fixing up the md driver to use the M_SHORTWAIT flag when allocating memory. In the event that our request times out, what errno should the driver return? Cheers, -Dan -- Build a man a fire and he'll be warm for a day. Set a man on fire and he'll be warm for the rest of his life. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 23:14:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 845DE37B416; Tue, 15 Jan 2002 23:14:39 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0G7C7U43820; Wed, 16 Jan 2002 08:12:08 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Dan Moschuk Cc: arch@FreeBSD.ORG Subject: Re: M_SHORTWAIT, the md driver and errno In-Reply-To: Your message of "Wed, 16 Jan 2002 01:29:50 EST." <20020116012950.C7831@spirit.jaded.net> Date: Wed, 16 Jan 2002 08:12:07 +0100 Message-ID: <43818.1011165127@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020116012950.C7831@spirit.jaded.net>, Dan Moschuk writes: > >I'm fixing up the md driver to use the M_SHORTWAIT flag when allocating >memory. In the event that our request times out, what errno should the >driver return? Good question, I guess EIO is the "make sure not to break anything" choice, but obviously ENOMEM would make more sense. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 15 23:55:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 00DF837B419 for ; Tue, 15 Jan 2002 23:55:09 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0G7slle098693; Wed, 16 Jan 2002 08:54:47 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0G7slEM098690; Wed, 16 Jan 2002 08:54:47 +0100 (CET) Date: Wed, 16 Jan 2002 08:54:47 +0100 (CET) From: Michal Mertl To: John Baldwin Cc: Peter Jeremy , , Thomas Hurst , "James E. Housley" , Bosko Milekic , Terry Lambert Subject: Re: 64 bit counters again In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 15 Jan 2002, John Baldwin wrote: > > On 16-Jan-02 Peter Jeremy wrote: > > For the read case, the reader uses something like: > > > > loop: movl 4(mem),%edx > > movl (mem),%eax > > cmpl 4(mem),%edx > > jnz loop > > > > If an interrupt updates the MSW then you take another pass around the > > loop, otherwise you always read the correct value. > > > > For the SMP case, you either need to use locks or you need to use > > per-CPU counters. (And the per-CPU counters can be read by another > > CPU using the above trick). > > Well, SMP on Pentium's maybe, but not on Alpha, sparc64, or ia64, all of which > support OOE and looser memory models than x86, meaning that you really need > locks unless you are going to have 386-specific code all over the place. I > suppose you can wrap it behind an MI API but that seems like a lot of work for > fairly small gain that can end up making the code uglier. > This ugly hack is supposed to be required only on 32 bit platforms. For me it's i386 (anything else with >0.1% user base?). On 64 bit platforms you just use native operand size. The code, which is present in the kernel now is most probably broken (at least on i386 compiler generates RMW). If we use something like what I proposed, we can fix it easily. At least I identified lots of places (not that it was difficult to do) where network counters are updated with possibly unsafe operation. -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 0:54:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id 1C69F37B417; Wed, 16 Jan 2002 00:54:10 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16Qlrw-000Phw-00; Wed, 16 Jan 2002 10:56:28 +0200 From: Sheldon Hearn To: "Crist J . Clark" Cc: FreeBSD@jovi.net, freebsd-questions@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG, bug-followup@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug In-reply-to: Your message of "Tue, 15 Jan 2002 21:03:03 PST." <20020115210303.E31328@blossom.cjclark.org> Date: Wed, 16 Jan 2002 10:56:28 +0200 Message-ID: <98823.1011171388@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 15 Jan 2002 21:03:03 PST, "Crist J . Clark" wrote: > On Tue, Jan 15, 2002 at 03:03:42PM -0500, FreeBSD@jovi.net wrote: > > It would help to document or fix date/ntpdate/ntpd/... > > ntpdate(8) and ntpd(8) are contributed software. As far as I can tell, > FreeBSD uses "virgin" imports. There is no local customization of the > code. Maintaining local modifications to code maintained by outside > vendors is expensive and therefore needs a lot of justification. And that's not the point. :-) Basically, we're being asked to compensate for poor conduct on the part of an administrator who changed the securelevel without knowing what he was doing. We can't compensate for poor change control in our documentation. This is akin to documenting in the manual pages of _every_ utility that writes to some file in the /var hierarchy, that the /var filesystem must not be mounted read-only. Let's just close the PR already. Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 0:56:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id 2572637B404; Wed, 16 Jan 2002 00:56:18 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16Qlu7-000Pim-00; Wed, 16 Jan 2002 10:58:43 +0200 From: Sheldon Hearn To: "Crist J . Clark" Cc: FreeBSD@jovi.net, freebsd-questions@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG, bug-followup@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug In-reply-to: Your message of "Wed, 16 Jan 2002 10:56:28 +0200." <98823.1011171388@axl.seasidesoftware.co.za> Date: Wed, 16 Jan 2002 10:58:43 +0200 Message-ID: <98875.1011171523@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002 10:56:28 +0200, Sheldon Hearn wrote: > Let's just close the PR already. And before anyone gets hysterical, I meant close it as per ru's request to nsayer, not just summarily close it without _any_ action. :-) Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 1:10: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from avocet.prod.itd.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by hub.freebsd.org (Postfix) with ESMTP id 95E3F37B427; Wed, 16 Jan 2002 01:09:51 -0800 (PST) Received: from dialup-209.245.128.158.dial1.sanjose1.level3.net ([209.245.128.158] helo=blossom.cjclark.org) by avocet.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16Qm4l-0006Wb-00; Wed, 16 Jan 2002 01:09:44 -0800 Received: (from cjc@localhost) by blossom.cjclark.org (8.11.6/8.11.3) id g0G99dV34320; Wed, 16 Jan 2002 01:09:39 -0800 (PST) (envelope-from cjc) Date: Wed, 16 Jan 2002 01:09:37 -0800 From: "Crist J . Clark" To: Sheldon Hearn Cc: FreeBSD@jovi.net, freebsd-questions@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG, bug-followup@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug Message-ID: <20020116010937.K31328@blossom.cjclark.org> Reply-To: cjclark@alum.mit.edu References: <20020115210303.E31328@blossom.cjclark.org> <98823.1011171388@axl.seasidesoftware.co.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <98823.1011171388@axl.seasidesoftware.co.za>; from sheldonh@starjuice.net on Wed, Jan 16, 2002 at 10:56:28AM +0200 X-URL: http://people.freebsd.org/~cjc/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 10:56:28AM +0200, Sheldon Hearn wrote: > > > On Tue, 15 Jan 2002 21:03:03 PST, "Crist J . Clark" wrote: > > > On Tue, Jan 15, 2002 at 03:03:42PM -0500, FreeBSD@jovi.net wrote: > > > It would help to document or fix date/ntpdate/ntpd/... > > > > ntpdate(8) and ntpd(8) are contributed software. As far as I can tell, > > FreeBSD uses "virgin" imports. There is no local customization of the > > code. Maintaining local modifications to code maintained by outside > > vendors is expensive and therefore needs a lot of justification. > > And that's not the point. :-) > > Basically, we're being asked to compensate for poor conduct on the > part of an administrator who changed the securelevel without knowing > what he was doing. We can't compensate for poor change control in our > documentation. I think there is a valid point. The settimeofday(2) call returns success even though the change requested by the call is not really done. This is somewhat questionable behavior. The documentation for settimeofday(2) was not clear about how this works under elevated securelevel(8), and in fact, the documentation is actually wrong (which I will fix shortly). -- "It's always funny until someone gets hurt. Then it's hilarious." Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 1:16:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id 1B18237B416; Wed, 16 Jan 2002 01:16:26 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16QmDP-000Pno-00; Wed, 16 Jan 2002 11:18:39 +0200 From: Sheldon Hearn To: cjclark@alum.mit.edu Cc: FreeBSD@jovi.net, freebsd-questions@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG, bug-followup@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug In-reply-to: Your message of "Wed, 16 Jan 2002 01:09:37 PST." <20020116010937.K31328@blossom.cjclark.org> Date: Wed, 16 Jan 2002 11:18:39 +0200 Message-ID: <99187.1011172719@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002 01:09:37 PST, "Crist J . Clark" wrote: > I think there is a valid point. The settimeofday(2) call returns > success even though the change requested by the call is not really > done. This is somewhat questionable behavior. The documentation for > settimeofday(2) was not clear about how this works under elevated > securelevel(8), and in fact, the documentation is actually wrong > (which I will fix shortly). I'm not disputing the need to correct settimeofday(2). The PR is already assigned to nsayer with a request to do this. But that's it. The notion of changing the documentation of all userland consumers is what I'm challenging. Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 5:31:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 641D337B416; Wed, 16 Jan 2002 05:31:29 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id AAA21917; Thu, 17 Jan 2002 00:31:11 +1100 Date: Thu, 17 Jan 2002 00:32:23 +1100 (EST) From: Bruce Evans X-X-Sender: To: Cc: Sheldon Hearn , , , , Subject: Re: kern/33904: secure mode bug In-Reply-To: <20020116010937.K31328@blossom.cjclark.org> Message-ID: <20020117002038.L3563-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002, Crist J . Clark wrote: > On Wed, Jan 16, 2002 at 10:56:28AM +0200, Sheldon Hearn wrote: > > Basically, we're being asked to compensate for poor conduct on the > > part of an administrator who changed the securelevel without knowing > > what he was doing. We can't compensate for poor change control in our > > documentation. > > I think there is a valid point. The settimeofday(2) call returns > success even though the change requested by the call is not really > done. This is somewhat questionable behavior. The documentation for > settimeofday(2) was not clear about how this works under elevated > securelevel(8), and in fact, the documentation is actually wrong > (which I will fix shortly). Don't forget to "fix" the Standard way to set times (clock_settime(2)) and the standard that says that clock_settime() shall set the time or fail (POSIX). The man pages are also misleading at best about adjtime(2) setting the time backwards. adjtime(2) can't go backwards fast enough to do more than retard the advance of the time, except possibly for bugs. The original "fix" may be moot because ntpd now uses the kernel pll automagically by default. Large steps by ntpdate and ntpd can't be replaced by the kernel pll, but the don't work anyway when securelevel > 1. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 8:37:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 8E1FE37B405; Wed, 16 Jan 2002 08:37:25 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g0GGbCa29120; Wed, 16 Jan 2002 18:37:12 +0200 (EET) (envelope-from ru) Date: Wed, 16 Jan 2002 18:37:12 +0200 From: Ruslan Ermilov To: Joerg Wunsch , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org Cc: arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020116183712.G13904@sunbay.com> References: <20020116132917.K78030@wantadilla.lemis.com> <20020116154210.A74132@uriah.heep.sax.de> <20020116174352.C13904@sunbay.com> <20020116171144.C18043@uriah.heep.sax.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020116171144.C18043@uriah.heep.sax.de> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 05:11:44PM +0100, Joerg Wunsch wrote: > As Ruslan Ermilov wrote: > > > All you need to do is to change the ownership on catpages holding > > directories back to ``man'', and install man(1) setuid ``man''. > > ...until the next "make installworld". That's why i'm asking for > a knob in /etc/make.conf. setuidperl can get its suid bit `sticky' > by the same way. > Hmm, can't you live with a custom gnu/usr.bin/man/man/Makefile? :-) > > But because it was proven to be insecure in many ways (the most > > important leak is a customized environment), I don't like the idea > > of even putting the required knobs back to src/. > > But all that can be compromised is user "man", isn't it? I don't > care much about this. > Not user "man", but the contents of the system manpages. Try this: ln -s /usr/bin/true /tmp/troff rm /usr/share/man/cat1/cat.1* /usr/bin/env GROFF_BIN_PATH=/tmp man 1 cat > Get me right, it's OK by me to put a big warning above that knob > into /etc/defaults/make.conf. But for sites less concerned about > security (like my home computer), compromising the "man" account > by the only other user on my machine (that's my wife :) isn't > anything i would care much about. Even though the machine is > pretty fast, i prefer the cached catpages for efficiency. (setuid > root binaries are a totally different matter, since they might > always serve as a target by a potential intruder. But i don't > think such an intruder would gain much by breaking a setuid man > man(1), and then replace all my catpages. ;-) > OK, what do others think about this? > > > I hope man(1) > > > is smart enough to handle that situation, and would reformat > > > the more recent man source instead of displaying the stale > > > catpage then. > > > Yes, man(1) handles this. > > OK, fine. > > > Also, catman(1) doesn't re-catman > > the entire tree by default: > > I know, this was one of Wolfram's design goals when he rewrote > it. ;-) > > -- > cheers, J"org .-.-. --... ...-- -.. . DL8DTL > > http://www.sax.de/~joerg/ NIC: JW11-RIPE > Never trust an operating system you don't have sources for. ;-) -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 8:47:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from enigma.trueimpact.net (enigma.trueimpact.net [209.82.45.201]) by hub.freebsd.org (Postfix) with ESMTP id 44E9537B41F for ; Wed, 16 Jan 2002 08:47:43 -0800 (PST) Received: from spirit.jaded.net (unknown [24.141.6.76]) by enigma.trueimpact.net (Postfix) with ESMTP id 4AB5266B09; Wed, 16 Jan 2002 11:47:31 -0500 (EST) Received: (from dan@localhost) by spirit.jaded.net (8.11.6/8.11.6) id g0GGm5I17260; Wed, 16 Jan 2002 11:48:05 -0500 (EST) (envelope-from dan) Date: Wed, 16 Jan 2002 11:48:05 -0500 From: Dan Moschuk To: Poul-Henning Kamp Cc: arch@FreeBSD.ORG Subject: Re: M_SHORTWAIT, the md driver and errno Message-ID: <20020116114805.L7831@spirit.jaded.net> References: <20020116012950.C7831@spirit.jaded.net> <43818.1011165127@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <43818.1011165127@critter.freebsd.dk>; from phk@critter.freebsd.dk on Wed, Jan 16, 2002 at 08:12:07AM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG | >I'm fixing up the md driver to use the M_SHORTWAIT flag when allocating | >memory. In the event that our request times out, what errno should the | >driver return? | | Good question, I guess EIO is the "make sure not to break anything" | choice, but obviously ENOMEM would make more sense. ENOMEM would make the most sense, I just wasn't sure you were "allowed" to return that for write calls. EIO kind of sucks; seeing an "Input/output error" while writing to a memory disk will probably result in people incorrectly thinking that they have bad ram. Cheers, -Dan -- Build a man a fire and he'll be warm for a day. Set a man on fire and he'll be warm for the rest of his life. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 9:23:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by hub.freebsd.org (Postfix) with ESMTP id 9175837B41A; Wed, 16 Jan 2002 09:23:39 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.9.3/8.9.3) with UUCP id SAA04804; Wed, 16 Jan 2002 18:23:38 +0100 (CET) Received: (from j@localhost) by uriah.heep.sax.de (8.11.6/8.11.6) id g0GHGPm00972; Wed, 16 Jan 2002 18:16:25 +0100 (MET) (envelope-from j) Date: Wed, 16 Jan 2002 18:16:25 +0100 From: Joerg Wunsch To: Ruslan Ermilov Cc: cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020116181625.B757@uriah.heep.sax.de> Reply-To: Joerg Wunsch Mail-Followup-To: Joerg Wunsch , Ruslan Ermilov , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org References: <20020116132917.K78030@wantadilla.lemis.com> <20020116154210.A74132@uriah.heep.sax.de> <20020116174352.C13904@sunbay.com> <20020116171144.C18043@uriah.heep.sax.de> <20020116183712.G13904@sunbay.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020116183712.G13904@sunbay.com>; from ru@FreeBSD.org on Wed, Jan 16, 2002 at 06:37:12PM +0200 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG As Ruslan Ermilov wrote: > > ...until the next "make installworld". That's why i'm asking for > > a knob in /etc/make.conf. setuidperl can get its suid bit `sticky' > > by the same way. > > > Hmm, can't you live with a custom gnu/usr.bin/man/man/Makefile? :-) Not really. (OK, i see the smiley. ;-) > Not user "man", but the contents of the system manpages. Try this: > > ln -s /usr/bin/true /tmp/troff > rm /usr/share/man/cat1/cat.1* > /usr/bin/env GROFF_BIN_PATH=/tmp man 1 cat OK, someone can cause garbage to go into my cat page. He could pretend that the options "-r" and "-f" to rm(1) would be something harmless :). Well, i'd like to see two things: . Variables like FOO_BIN_PATH need to be ignored when running with raised prvileges, no question asked. We used to ignore LD_LIBRARY_PATH for the same reason. I hope this is something that is fixable. . Then turn off the setuid bit, but offer the option to re-enable it for those who value the feature more than the risk, much in the same sense as we do for suidperl (which i still think is a lot less risky than someone (like me :) writing a buggy setuid wrapper in C). -- cheers, J"org .-.-. --... ...-- -.. . DL8DTL http://www.sax.de/~joerg/ NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 9:30:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by hub.freebsd.org (Postfix) with ESMTP id 5BD7B37B400; Wed, 16 Jan 2002 09:30:05 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.9.3/8.9.3) with UUCP id SAA04917; Wed, 16 Jan 2002 18:30:04 +0100 (CET) Received: (from j@localhost) by uriah.heep.sax.de (8.11.6/8.11.6) id g0GHMeN01036; Wed, 16 Jan 2002 18:22:40 +0100 (MET) (envelope-from j) Date: Wed, 16 Jan 2002 18:22:40 +0100 From: Joerg Wunsch To: cvs-committers@FreeBSD.ORG, cvs-all@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020116182240.C757@uriah.heep.sax.de> Reply-To: Joerg Wunsch Mail-Followup-To: Joerg Wunsch , cvs-committers@FreeBSD.ORG, cvs-all@FreeBSD.ORG, arch@FreeBSD.ORG References: <20020116183712.G13904@sunbay.com> <20020117053925.A18072-100000@lists.unixathome.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020117053925.A18072-100000@lists.unixathome.org>; from dan@langille.org on Thu, Jan 17, 2002 at 05:57:23AM +1300 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG As Dan Langille wrote: > > OK, what do others think about this? > > We've had users coming into IRC channels asking about the man page > changes. I don't fully understand the implications yet. From the > comments above, it sounds like each time I issue 'man ls', it's going to > decompress the man page unless I do some tweaking. Is that correct? It's even worse. Decompressing takes relatively little time, and happens for cat pages as well (they are stored as .gz files, too). But it needs to run the troff toolchain each time you want to look at a man page (gzcat .gz | tbl | nroff -mandoc). For larger pages this can take a good bit of time. The largest page i could find here was avr-gcc (basically a full plain GNU gcc man page), which takes 3.5 seconds to format on this 400 MHz K6 machine here. > Please don't force the majority of people to tweak their systems in order > to avoid a theoretical problem for a small minority. I think you might neglect the security problem here. Although i'd like to have the knob in make.conf as well, i wouldn't turn it on unless after trying to understand /which/ security risk i am at. ;-) -- cheers, J"org .-.-. --... ...-- -.. . DL8DTL http://www.sax.de/~joerg/ NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 9:34: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id A437D37B404; Wed, 16 Jan 2002 09:34:00 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id EAA01541; Thu, 17 Jan 2002 04:33:57 +1100 Date: Thu, 17 Jan 2002 04:35:10 +1100 (EST) From: Bruce Evans X-X-Sender: To: Ruslan Ermilov Cc: Joerg Wunsch , , , Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist In-Reply-To: <20020116183712.G13904@sunbay.com> Message-ID: <20020117042612.H380-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002, Ruslan Ermilov wrote: > On Wed, Jan 16, 2002 at 05:11:44PM +0100, Joerg Wunsch wrote: > > As Ruslan Ermilov wrote: > > > > > All you need to do is to change the ownership on catpages holding > > > directories back to ``man'', and install man(1) setuid ``man''. > > > > ...until the next "make installworld". That's why i'm asking for > > a knob in /etc/make.conf. setuidperl can get its suid bit `sticky' > > by the same way. > > > Hmm, can't you live with a custom gnu/usr.bin/man/man/Makefile? :-) No, since installworld will change the ownership of the catpages directories back to root. You also need to modify src/etc/mtree/*. Then you will probably find that installing packages changes the ownership of the catpage directories back to root, unless the packages are old. People without local changes will probably find that installing old packages changes the ownership of the catpage directories back to man :-). I'm used to installing packages clobbering my policy of not having any catpage directories. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 9:55: 3 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id EC5D337B404; Wed, 16 Jan 2002 09:54:40 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g0GHsTU39957; Wed, 16 Jan 2002 19:54:29 +0200 (EET) (envelope-from ru) Date: Wed, 16 Jan 2002 19:54:29 +0200 From: Ruslan Ermilov To: Joerg Wunsch , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020116195429.J13904@sunbay.com> References: <20020116132917.K78030@wantadilla.lemis.com> <20020116154210.A74132@uriah.heep.sax.de> <20020116174352.C13904@sunbay.com> <20020116171144.C18043@uriah.heep.sax.de> <20020116183712.G13904@sunbay.com> <20020116181625.B757@uriah.heep.sax.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020116181625.B757@uriah.heep.sax.de> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 06:16:25PM +0100, Joerg Wunsch wrote: > As Ruslan Ermilov wrote: > > > > ...until the next "make installworld". That's why i'm asking for > > > a knob in /etc/make.conf. setuidperl can get its suid bit `sticky' > > > by the same way. > > > > > Hmm, can't you live with a custom gnu/usr.bin/man/man/Makefile? :-) > > Not really. (OK, i see the smiley. ;-) > > > Not user "man", but the contents of the system manpages. Try this: > > > > ln -s /usr/bin/true /tmp/troff > > rm /usr/share/man/cat1/cat.1* > > /usr/bin/env GROFF_BIN_PATH=/tmp man 1 cat > > OK, someone can cause garbage to go into my cat page. He could > pretend that the options "-r" and "-f" to rm(1) would be something > harmless :). > > Well, i'd like to see two things: > > . Variables like FOO_BIN_PATH need to be ignored when running > with raised prvileges, no question asked. We used to ignore > LD_LIBRARY_PATH for the same reason. I hope this is something > that is fixable. > Hmm. In this case, the actual problem is with groff(1). man(1) executes groff(1) in secure (-S) mode (actually it is now the default mode), and the latter should not respect these environment variables while running in this mode. I will see tomorrow if this is easily fixable -- it should be. OK, one big problem shot. > . Then turn off the setuid bit, but offer the option to re-enable > it for those who value the feature more than the risk, much in > the same sense as we do for suidperl (which i still think is a > lot less risky than someone (like me :) writing a buggy setuid > wrapper in C). > There's still problem exists with following symbolic links (please see the PR for an example exploit). I tried a quick patch that should solve this, but Robert Watson pointed out that it is subject to a race between lstat(2)'ting a directory holding a catpage and creating a file in that directory. Unfortunately, O_NOFOLLOW only works for the last component of the pathname passed to open(2). If we could find a solution to this problem, I would be more than happy to restore this functionality of man(1). Cheers, Ruslan (who now thinks that the only reliable way to get useful feedback is to commit something). To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 10: 2:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 0CA4837B426; Wed, 16 Jan 2002 10:02:19 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g0GI1St41047; Wed, 16 Jan 2002 20:01:28 +0200 (EET) (envelope-from ru) Date: Wed, 16 Jan 2002 20:01:28 +0200 From: Ruslan Ermilov To: Joerg Wunsch , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020116200128.L13904@sunbay.com> References: <20020116132917.K78030@wantadilla.lemis.com> <20020116154210.A74132@uriah.heep.sax.de> <20020116174352.C13904@sunbay.com> <20020116171144.C18043@uriah.heep.sax.de> <20020116183712.G13904@sunbay.com> <20020116181625.B757@uriah.heep.sax.de> <20020116195429.J13904@sunbay.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020116195429.J13904@sunbay.com> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 07:54:29PM +0200, Ruslan Ermilov wrote: > On Wed, Jan 16, 2002 at 06:16:25PM +0100, Joerg Wunsch wrote: > > As Ruslan Ermilov wrote: > > > > > > ...until the next "make installworld". That's why i'm asking for > > > > a knob in /etc/make.conf. setuidperl can get its suid bit `sticky' > > > > by the same way. > > > > > > > Hmm, can't you live with a custom gnu/usr.bin/man/man/Makefile? :-) > > > > Not really. (OK, i see the smiley. ;-) > > > > > Not user "man", but the contents of the system manpages. Try this: > > > > > > ln -s /usr/bin/true /tmp/troff > > > rm /usr/share/man/cat1/cat.1* > > > /usr/bin/env GROFF_BIN_PATH=/tmp man 1 cat > > > > OK, someone can cause garbage to go into my cat page. He could > > pretend that the options "-r" and "-f" to rm(1) would be something > > harmless :). > > > > Well, i'd like to see two things: > > > > . Variables like FOO_BIN_PATH need to be ignored when running > > with raised prvileges, no question asked. We used to ignore > > LD_LIBRARY_PATH for the same reason. I hope this is something > > that is fixable. > > > Hmm. In this case, the actual problem is with groff(1). man(1) > executes groff(1) in secure (-S) mode (actually it is now the > default mode), and the latter should not respect these environment > variables while running in this mode. I will see tomorrow if this > is easily fixable -- it should be. OK, one big problem shot. > > > . Then turn off the setuid bit, but offer the option to re-enable > > it for those who value the feature more than the risk, much in > > the same sense as we do for suidperl (which i still think is a > > lot less risky than someone (like me :) writing a buggy setuid > > wrapper in C). > > > There's still problem exists with following symbolic links (please > see the PR for an example exploit). I tried a quick patch that > should solve this, but Robert Watson pointed out that it is subject > to a race between lstat(2)'ting a directory holding a catpage and > creating a file in that directory. Unfortunately, O_NOFOLLOW only > works for the last component of the pathname passed to open(2). > If we could find a solution to this problem, I would be more than > happy to restore this functionality of man(1). > No, not actually happy unless you tell me how to solve the GZIP environment variable problem mentioned here: http://security-archive.merton.ox.ac.uk/security-audit-200010/0022.html Cheers, -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 10:31:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail12.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by hub.freebsd.org (Postfix) with ESMTP id BA53537B404 for ; Wed, 16 Jan 2002 10:31:35 -0800 (PST) Received: (qmail 17492 invoked from network); 16 Jan 2002 18:31:34 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([64.81.54.73]) (envelope-sender ) by mail12.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 16 Jan 2002 18:31:34 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Wed, 16 Jan 2002 10:30:51 -0800 (PST) From: John Baldwin To: Michal Mertl Subject: Re: 64 bit counters again Cc: Terry Lambert , Cc: Terry Lambert , Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.ORG, Peter Jeremy Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 16-Jan-02 Michal Mertl wrote: > On Tue, 15 Jan 2002, John Baldwin wrote: > >> >> On 16-Jan-02 Peter Jeremy wrote: >> > For the read case, the reader uses something like: >> > >> > loop: movl 4(mem),%edx >> > movl (mem),%eax >> > cmpl 4(mem),%edx >> > jnz loop >> > >> > If an interrupt updates the MSW then you take another pass around the >> > loop, otherwise you always read the correct value. >> > >> > For the SMP case, you either need to use locks or you need to use >> > per-CPU counters. (And the per-CPU counters can be read by another >> > CPU using the above trick). >> >> Well, SMP on Pentium's maybe, but not on Alpha, sparc64, or ia64, all of >> which >> support OOE and looser memory models than x86, meaning that you really need >> locks unless you are going to have 386-specific code all over the place. I >> suppose you can wrap it behind an MI API but that seems like a lot of work >> for >> fairly small gain that can end up making the code uglier. >> > > This ugly hack is supposed to be required only on 32 bit platforms. For me > it's i386 (anything else with >0.1% user base?). On 64 bit platforms you > just use native operand size. > > The code, which is present in the kernel now is most probably broken (at > least on i386 compiler generates RMW). Other platforms don't _have_ a single RMW version of add for example. Most of them do a load, add, store. So are you now going to be using atomic ops for all counters? It's either that or a lock. And if you use u_int64_t for your counters, you would just need to add support for 64-bit atomic ops to the atomic API. Reading the values becomes a bit hard then though as that would require an atomic op to ensure you aren't reading a stale value. So which approach are you going to take? (They will need one of these once SMPng gets down to that level anyways.) Also, the powerpc port will be 32-bit once it gets off the ground and will likely have a fairly good userbase, as well. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 10:38:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from columbus.cris.net (columbus.cris.net [212.110.128.65]) by hub.freebsd.org (Postfix) with ESMTP id A0D9D37B400; Wed, 16 Jan 2002 10:38:45 -0800 (PST) Received: from ark.cris.net (ark.cris.net [212.110.128.68]) by columbus.cris.net (8.9.3/8.9.3) with ESMTP id UAA26252; Wed, 16 Jan 2002 20:38:41 +0200 (EET) Received: (from phantom@localhost) by ark.cris.net (8.11.1/8.11.1) id g0GIcH600910; Wed, 16 Jan 2002 20:38:17 +0200 (EET) Date: Wed, 16 Jan 2002 20:38:16 +0200 From: Alexey Zelkin To: Ruslan Ermilov Cc: Joerg Wunsch , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020116203816.B99450@ark.cris.net> References: <20020116132917.K78030@wantadilla.lemis.com> <20020116154210.A74132@uriah.heep.sax.de> <20020116174352.C13904@sunbay.com> <20020116171144.C18043@uriah.heep.sax.de> <20020116183712.G13904@sunbay.com> <20020116181625.B757@uriah.heep.sax.de> <20020116195429.J13904@sunbay.com> <20020116200128.L13904@sunbay.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <20020116200128.L13904@sunbay.com>; from ru@FreeBSD.org on Wed, Jan 16, 2002 at 08:01:28PM +0200 X-Operating-System: FreeBSD 3.5-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG hi, On Wed, Jan 16, 2002 at 08:01:28PM +0200, Ruslan Ermilov wrote: > No, not actually happy unless you tell me how to solve the GZIP > environment variable problem mentioned here: > > http://security-archive.merton.ox.ac.uk/security-audit-200010/0022.html unsetenv(3) ? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 11: 9:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id AAF8937B419; Wed, 16 Jan 2002 11:09:19 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id 8022310DE06; Wed, 16 Jan 2002 11:09:19 -0800 (PST) Date: Wed, 16 Jan 2002 11:09:19 -0800 From: Alfred Perlstein To: Dan Moschuk Cc: Poul-Henning Kamp , arch@FreeBSD.ORG Subject: Re: M_SHORTWAIT, the md driver and errno Message-ID: <20020116110919.D26067@elvis.mu.org> References: <20020116012950.C7831@spirit.jaded.net> <43818.1011165127@critter.freebsd.dk> <20020116114805.L7831@spirit.jaded.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020116114805.L7831@spirit.jaded.net>; from dan@FreeBSD.ORG on Wed, Jan 16, 2002 at 11:48:05AM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Dan Moschuk [020116 08:48] wrote: > > | >I'm fixing up the md driver to use the M_SHORTWAIT flag when allocating > | >memory. In the event that our request times out, what errno should the > | >driver return? > | > | Good question, I guess EIO is the "make sure not to break anything" > | choice, but obviously ENOMEM would make more sense. > > ENOMEM would make the most sense, I just wasn't sure you were "allowed" to > return that for write calls. > > EIO kind of sucks; seeing an "Input/output error" while writing to a > memory disk will probably result in people incorrectly thinking that they > have bad ram. I think EIO is probably more correct with a rate limited console message explaining what happened. Also, I think that the timeout should be configurable and users should have the option to turn it off. I also think that the timeout should be relatively long, on the order of 30 seconds at least. -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 11:33:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by hub.freebsd.org (Postfix) with ESMTP id 236FC37B417 for ; Wed, 16 Jan 2002 11:33:33 -0800 (PST) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.11.4/8.11.4) id g0GJXGn72831; Wed, 16 Jan 2002 14:33:16 -0500 (EST) (envelope-from wollman) Date: Wed, 16 Jan 2002 14:33:16 -0500 (EST) From: Garrett Wollman Message-Id: <200201161933.g0GJXGn72831@khavrinen.lcs.mit.edu> To: Alexey Zelkin Cc: arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist In-Reply-To: <20020116203816.B99450@ark.cris.net> References: <20020116132917.K78030@wantadilla.lemis.com> <20020116154210.A74132@uriah.heep.sax.de> <20020116174352.C13904@sunbay.com> <20020116171144.C18043@uriah.heep.sax.de> <20020116183712.G13904@sunbay.com> <20020116181625.B757@uriah.heep.sax.de> <20020116195429.J13904@sunbay.com> <20020116200128.L13904@sunbay.com> <20020116203816.B99450@ark.cris.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > unsetenv(3) ? I'd like to suggest that man(1) should clean out the environment before running groff. The normal policy of the security weenies is ``deny by default'' and I don't see any reason why it should be different in this case. -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 11:43:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 1EC0937B404; Wed, 16 Jan 2002 11:43:09 -0800 (PST) Received: from pool0454.cvx21-bradley.dialup.earthlink.net ([209.179.193.199] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QvxN-0005K6-00; Wed, 16 Jan 2002 11:42:45 -0800 Message-ID: <3C45D7B2.A1F66BA7@mindspring.com> Date: Wed, 16 Jan 2002 11:42:42 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Peter Jeremy Cc: Michal Mertl , Bosko Milekic , "James E. Housley" , Thomas Hurst , arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: <3C4492EE.5A60AD0B@mindspring.com> <20020116120611.A72285@gsmx07.alcatel.com.au> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Jeremy wrote: > IA64 is 64-bit. The current argument is about 64-bit operations > on a 32-bit machine. Multi-port 10Gbit cards are going to need > a re-designed bus as well. Infiniband? HCI? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 12: 1:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 7127637B427; Wed, 16 Jan 2002 12:00:51 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g0GK0iD75891; Wed, 16 Jan 2002 15:00:44 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Wed, 16 Jan 2002 15:00:41 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Ruslan Ermilov Cc: Joerg Wunsch , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist In-Reply-To: <20020116195429.J13904@sunbay.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002, Ruslan Ermilov wrote: > There's still problem exists with following symbolic links (please see > the PR for an example exploit). I tried a quick patch that should solve > this, but Robert Watson pointed out that it is subject to a race between > lstat(2)'ting a directory holding a catpage and creating a file in that > directory. Unfortunately, O_NOFOLLOW only works for the last component > of the pathname passed to open(2). If we could find a solution to this > problem, I would be more than happy to restore this functionality of > man(1). Part of the problem here is that man's behavior is very complex, and the UNIX inheritence model makes things rather messy. Simply eliminating dynamically cached catpages eliminates the risk associated with the model, and is my preferred solution. It's not hard to imagine tactics to escalate privilege from user man to user root in the event that the man program or any children running as uid of man are compromised. I'm happy with the behavior being available and turned off by default, but personally my feeling is that the performance/correctness tradeoff leans towards correctness given the risk. And to be honest, people don't usually benchmark systems based on the time it takes to render a man page. :-) Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 12: 2:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 09E8537B405; Wed, 16 Jan 2002 12:02:51 -0800 (PST) Received: from pool0454.cvx21-bradley.dialup.earthlink.net ([209.179.193.199] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QwGo-0004Po-00; Wed, 16 Jan 2002 12:02:50 -0800 Message-ID: <3C45DC67.D92C32B9@mindspring.com> Date: Wed, 16 Jan 2002 12:02:47 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Crist J . Clark" Cc: FreeBSD@jovi.net, freebsd-questions@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG, bug-followup@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug References: <200201152003.g0FK3gb04767@grant.org> <20020115210303.E31328@blossom.cjclark.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Crist J . Clark" wrote: > On Tue, Jan 15, 2002 at 03:03:42PM -0500, FreeBSD@jovi.net wrote: > > or better yet > > fix settimeofday to return EPERM instead of trying > > to second-guess the intent of the programmer. > > That might be a valid option. Would you be willing to patch settime() > (in sys/kern/kern_time.c)? Would you be willing to test patches to see > if they don't break things even worse? Bad idea. Specifically, the clamping doesn't prevent small deltas from being applied, and it does not prevent larger deltas from being applied iteratively in order to actually get a larger delta. This means that a delta of several minutes *will* eventually cause the clock to slowly approach the correct value, until it is eventually correct. Returning an error all the time wouldn't let this happen, since you would prevent, not clam, the adjustment. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 12: 6:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from ns.yogotech.com (ns.yogotech.com [206.127.123.66]) by hub.freebsd.org (Postfix) with ESMTP id AAF6037B41C; Wed, 16 Jan 2002 12:06:28 -0800 (PST) Received: from caddis.yogotech.com (caddis.yogotech.com [206.127.123.130]) by ns.yogotech.com (8.9.3/8.9.3) with ESMTP id NAA10995; Wed, 16 Jan 2002 13:06:21 -0700 (MST) (envelope-from nate@yogotech.com) Received: (from nate@localhost) by caddis.yogotech.com (8.11.6/8.11.6) id g0GK6LD12894; Wed, 16 Jan 2002 13:06:21 -0700 (MST) (envelope-from nate) From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15429.56636.984304.733778@caddis.yogotech.com> Date: Wed, 16 Jan 2002 13:06:20 -0700 To: Robert Watson Cc: Ruslan Ermilov , Joerg Wunsch , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist In-Reply-To: References: <20020116195429.J13904@sunbay.com> X-Mailer: VM 6.96 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Reply-To: nate@yogotech.com (Nate Williams) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > > There's still problem exists with following symbolic links (please see > > the PR for an example exploit). I tried a quick patch that should solve > > this, but Robert Watson pointed out that it is subject to a race between > > lstat(2)'ting a directory holding a catpage and creating a file in that > > directory. Unfortunately, O_NOFOLLOW only works for the last component > > of the pathname passed to open(2). If we could find a solution to this > > problem, I would be more than happy to restore this functionality of > > man(1). > > Part of the problem here is that man's behavior is very complex, and the > UNIX inheritence model makes things rather messy. Simply eliminating > dynamically cached catpages eliminates the risk associated with the model, > and is my preferred solution. It's not hard to imagine tactics to > escalate privilege from user man to user root in the event that the man > program or any children running as uid of man are compromised. My thinking is that it's just as easy to get root from a normal user as it is to get it from man, so we're really not gaining a whole lot (from the point of view of root compromises). Regardless, there are still other concerns with over-writing files and such that are annoying, if not necessarily security holes in the sense of getting root access. Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 12: 8: 1 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id B7AE737B404; Wed, 16 Jan 2002 12:07:52 -0800 (PST) Received: from pool0454.cvx21-bradley.dialup.earthlink.net ([209.179.193.199] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QwLX-0004Zq-00; Wed, 16 Jan 2002 12:07:43 -0800 Message-ID: <3C45DD8C.E3C3414E@mindspring.com> Date: Wed, 16 Jan 2002 12:07:40 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Peter Jeremy Cc: John Baldwin , arch@FreeBSD.org, Thomas Hurst , "James E. Housley" , Bosko Milekic , Michal Mertl Subject: Re: 64 bit counters again References: <20020116120611.A72285@gsmx07.alcatel.com.au> <20020116161251.E72285@gsmx07.alcatel.com.au> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Jeremy wrote: > I think I'll retract my suggestion that it's possible to read the > counters on any CPU without locking :-). > > Locking per-CPU counters defeats most of the justification for having > per-CPU counters in the first place. (There is still some advantage - > the counters hopefully stay in the local CPU cache instead of > continually jumping between caches). This shouldn't be a problem, if the global total counter is a real total and not an incremental total, since reading these values is not a problem, only writing them, even if your cache coherency is only MEI, instead of MESI. Statistics are only ever snapshots, anyway. > How about the following: Have a per-CPU thread to read the per-CPU > counters and update them into common counters. This could be done either > by having the counter read operation schedule a read on each CPU and > then returning the total, or having a "counter update" thread that runs > in each CPU every (say) second [with a degree of time skew] and > does locked updates into a common central counter. Complicated; I think the cure is worse than the disease; I'm still very fond (if one *must* have 64 bit counters) of letting people build kernels with 64 bit counters on 32 bit processors, but leaving them 32 bit by default on those platforms. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 12:21:20 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 1CC2E37B400; Wed, 16 Jan 2002 12:21:17 -0800 (PST) Received: from pool0454.cvx21-bradley.dialup.earthlink.net ([209.179.193.199] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16QwYX-0006uS-00; Wed, 16 Jan 2002 12:21:10 -0800 Message-ID: <3C45E0B2.A092CB4E@mindspring.com> Date: Wed, 16 Jan 2002 12:21:06 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: cjclark@alum.mit.edu Cc: Sheldon Hearn , FreeBSD@jovi.net, freebsd-questions@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG, bug-followup@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug References: <20020115210303.E31328@blossom.cjclark.org> <98823.1011171388@axl.seasidesoftware.co.za> <20020116010937.K31328@blossom.cjclark.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Crist J . Clark" wrote: > The settimeofday(2) call returns > success even though the change requested by the call is not really > done. This is somewhat questionable behavior. The documentation for > settimeofday(2) was not clear about how this works under elevated > securelevel(8), and in fact, the documentation is actually wrong > (which I will fix shortly). This is BS. It's not documented how it works in jails, either, or under vmware. THe call is done, it's just that the caller was, in context, requesting something other than what they thought they were requesting. In fact, any delta in excess of 1 is a request for 1, at that secure level. The call is doing *exactly* what it is supposed to do. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 12:30:14 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by hub.freebsd.org (Postfix) with ESMTP id F1DD837B402; Wed, 16 Jan 2002 12:30:06 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.9.3/8.9.3) with UUCP id VAA08829; Wed, 16 Jan 2002 21:30:05 +0100 (CET) Received: (from j@localhost) by uriah.heep.sax.de (8.11.6/8.11.6) id g0GKSRQ05435; Wed, 16 Jan 2002 21:28:27 +0100 (MET) (envelope-from j) Date: Wed, 16 Jan 2002 21:28:27 +0100 From: Joerg Wunsch To: Ruslan Ermilov Cc: cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020116212827.B3262@uriah.heep.sax.de> Reply-To: Joerg Wunsch Mail-Followup-To: Joerg Wunsch , Ruslan Ermilov , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org References: <20020116132917.K78030@wantadilla.lemis.com> <20020116154210.A74132@uriah.heep.sax.de> <20020116174352.C13904@sunbay.com> <20020116171144.C18043@uriah.heep.sax.de> <20020116183712.G13904@sunbay.com> <20020116181625.B757@uriah.heep.sax.de> <20020116195429.J13904@sunbay.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020116195429.J13904@sunbay.com>; from ru@FreeBSD.org on Wed, Jan 16, 2002 at 07:54:29PM +0200 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG As Ruslan Ermilov wrote: > There's still problem exists with following symbolic links (please > see the PR for an example exploit). But that one either requires write permission to the directory holding the cat* directories, or it requires the user to run man -M or otherwise against a modified $MANPATH. Sure, it can clobber files that are writable by user man. It's fine by me to have suidness turned off by default (and then probably also to ship a system that doesn't even have the cat directories -- what are they good for if we don't store something there? catman can handle creation of the directories by itself). I'll probably even leave it turned off on my workstation at work, but would simply like to have it as a knob on some machines. -- cheers, J"org .-.-. --... ...-- -.. . DL8DTL http://www.sax.de/~joerg/ NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 13: 9:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id DC48237B419; Wed, 16 Jan 2002 13:09:09 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g0GL8tD80819; Wed, 16 Jan 2002 16:08:56 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Wed, 16 Jan 2002 16:08:55 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Nate Williams Cc: Ruslan Ermilov , Joerg Wunsch , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist In-Reply-To: <15429.56636.984304.733778@caddis.yogotech.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002, Nate Williams wrote: > My thinking is that it's just as easy to get root from a normal user as > it is to get it from man, so we're really not gaining a whole lot (from > the point of view of root compromises). I'm not 100% sure that's true. One of the problems with UUCP was that it was special in a way normal accounts are not: the root user may routinely run binaries that are setuid uucp, or setuid man, whereas the superuser doesn't routinely run binaries that are setuid nate or setuid robert. The result is that extra care must be taken to determine that the privileges inherited by the binary are properly handled. Right now, the binaries are set schg, which provides quite a bit of improvement over the alternative (where the man user or uucp user could simply modify the binary and gain root privilege the next time it was run). Of course, that's only true if the filesystem you're running off of is UFS. :-) There are still many potential sources of weakness due to the unix inheritence model associated with exec and setuid applications. The lowest risk solution (which doesn't cut back much on functionality) is to avoid this sort of arrangement as much as possible. We've shot ourselves in the feet with both setuid man and setuid uucp in the past. Our feet are sore. Let's shoot something else. > Regardless, there are still other concerns with over-writing files and > such that are annoying, if not necessarily security holes in the sense > of getting root access. Those can be used in social engineering attacks quite easily: many system administrators rely on the man pages to provide an accurate description of how a utility works. It's easy to imagine minor tweaks to common command lines that open up unnecessary privileges to users, even for experienced administrators. For example, the nohup command has documented behavior of creating output files in the CWD of the process. By tweaking the man page appropriately and describing normal use of nohup, I might leave you with the impression it was safe to run nohup in a directory that you don't own, or that is world writable. Likewise, I might tweak man pages to have you incorrectly configure utilities, use the wrong command line argument to specify files (target vs. configuration file, for example). The correctness and safety of man pages directly affects the security of the system. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 13:10:29 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id 8E00637B423; Wed, 16 Jan 2002 13:09:40 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.11.3/8.11.3) with ESMTP id g0GL9cF27430; Wed, 16 Jan 2002 16:09:38 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: References: Date: Wed, 16 Jan 2002 16:09:37 -0500 To: Robert Watson , Ruslan Ermilov From: Garance A Drosihn Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Cc: Joerg Wunsch , arch@FreeBSD.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.1 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG (I am bcc:-ing this to cvs-committers and cvs-all, with the idea that this discussion is also going on in freebsd-arch and thus we could drop it from those two cvs lists...) At 3:00 PM -0500 1/16/02, Robert Watson wrote: >I'm happy with the behavior being available and turned off by default, >but personally my feeling is that the performance/correctness tradeoff >leans towards correctness given the risk. And to be honest, people >don't usually benchmark systems based on the time it takes to render >a man page. :-) But it is one of those things that will make the system "seem slower" to them, in day-to-day use. I think the security issue is a good enough reason to turn off the current behavior of 'man', but I do wish there was some middle-ground option which was between 'zero cat pages on disk' and 'automatically generate all cat pages for all existing man pages'. In my case, I have about a dozen man pages that I reference a lot, and a lot of man pages that I never reference. If something could keep track of which pages were actually referenced a lot, then some system daemon could generate cat-versions of just those man pages. I realize that's probably a large hammer to be invoking to solve such a little nail of a problem, but I couldn't help but wonder if there was some other way to handle this. -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 13:15:29 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 88C4837B400 for ; Wed, 16 Jan 2002 13:15:24 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0GLFEle074809; Wed, 16 Jan 2002 22:15:16 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0GLFDWT074806; Wed, 16 Jan 2002 22:15:13 +0100 (CET) Date: Wed, 16 Jan 2002 22:15:13 +0100 (CET) From: Michal Mertl To: John Baldwin Cc: Terry Lambert , Bosko Milekic , "James E. Housley" , Thomas Hurst , , Peter Jeremy Subject: Re: 64 bit counters again In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002, John Baldwin wrote: > > On 16-Jan-02 Michal Mertl wrote: > > On Tue, 15 Jan 2002, John Baldwin wrote: > > > >> > >> On 16-Jan-02 Peter Jeremy wrote: > >> > For the read case, the reader uses something like: > >> > > >> > loop: movl 4(mem),%edx > >> > movl (mem),%eax > >> > cmpl 4(mem),%edx > >> > jnz loop > >> > > >> > If an interrupt updates the MSW then you take another pass around the > >> > loop, otherwise you always read the correct value. > >> > > >> > For the SMP case, you either need to use locks or you need to use > >> > per-CPU counters. (And the per-CPU counters can be read by another > >> > CPU using the above trick). > >> > >> Well, SMP on Pentium's maybe, but not on Alpha, sparc64, or ia64, all of > >> which > >> support OOE and looser memory models than x86, meaning that you really need > >> locks unless you are going to have 386-specific code all over the place. I > >> suppose you can wrap it behind an MI API but that seems like a lot of work > >> for > >> fairly small gain that can end up making the code uglier. > >> > > > > This ugly hack is supposed to be required only on 32 bit platforms. For me > > it's i386 (anything else with >0.1% user base?). On 64 bit platforms you > > just use native operand size. > > > > The code, which is present in the kernel now is most probably broken (at > > least on i386 compiler generates RMW). > > Other platforms don't _have_ a single RMW version of add for example. Most of > them do a load, add, store. So are you now going to be using atomic ops for > all counters? It's either that or a lock. And if you use u_int64_t for My fault - I meant you wouldn't have problems with 2 partial accesses (high and low part). But you're right - it doesn't solve the problem. I made an API which makes it easy to choose whatever version of operation is required on chosen platform. No locks though. I don't know how expensive these are, but if atomicity is required and is very expensive on "strange :-)" hardware, the best way to go probably would be to have some per-cpu data and some collector kernel thread or something. Access to these counters could be also covered by my functions. But I didn't programme it nor the collecting code which would be needed then. > your counters, you would just need to add support for 64-bit atomic ops to the > atomic API. Reading the values becomes a bit hard then though as that would In my patch there's also support for 64 bit atomic ops on >=i586. That's one thing which shouldn't hurt anyone. That needs review - I wasn't able to find good documentation examples for gcc inline asm constraints so I ended up mostly telling gcc exactly which register to use. > require an atomic op to ensure you aren't reading a stale value. So which > approach are you going to take? (They will need one of these once SMPng gets With my patch it's compile time decision - the default should be probably simple 32 bit operations. That's how the counters work now (they're defined u_long so on 64 bit it's simple 64 bit operation). It should be fairly easy to make it MI. > down to that level anyways.) > > Also, the powerpc port will be 32-bit once it gets off the ground and will > likely have a fairly good userbase, as well. That would have to be delt with when it comes but I don't expect problems (is it English sentence ? :-) ). -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 13:16:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id A769337B400; Wed, 16 Jan 2002 13:16:09 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id B9EF010DE00; Wed, 16 Jan 2002 13:16:08 -0800 (PST) Date: Wed, 16 Jan 2002 13:16:08 -0800 From: Alfred Perlstein To: Garance A Drosihn Cc: Robert Watson , Ruslan Ermilov , Joerg Wunsch , arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020116131608.J26067@elvis.mu.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from drosih@rpi.edu on Wed, Jan 16, 2002 at 04:09:37PM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Garance A Drosihn [020116 13:12] wrote: > (I am bcc:-ing this to cvs-committers and cvs-all, with the idea > that this discussion is also going on in freebsd-arch and thus we > could drop it from those two cvs lists...) > > At 3:00 PM -0500 1/16/02, Robert Watson wrote: > >I'm happy with the behavior being available and turned off by default, > >but personally my feeling is that the performance/correctness tradeoff > >leans towards correctness given the risk. And to be honest, people > >don't usually benchmark systems based on the time it takes to render > >a man page. :-) > > But it is one of those things that will make the system "seem slower" > to them, in day-to-day use. > > I think the security issue is a good enough reason to turn off the > current behavior of 'man', but I do wish there was some middle-ground > option which was between 'zero cat pages on disk' and 'automatically > generate all cat pages for all existing man pages'. > > In my case, I have about a dozen man pages that I reference a lot, and > a lot of man pages that I never reference. If something could keep > track of which pages were actually referenced a lot, then some system > daemon could generate cat-versions of just those man pages. > > I realize that's probably a large hammer to be invoking to solve such > a little nail of a problem, but I couldn't help but wonder if there > was some other way to handle this. If $HOME/.foo has proper permissions man should cache the pages there. -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 15: 1:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 43F0537B404 for ; Wed, 16 Jan 2002 15:01:16 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0GN1Fle079313 for ; Thu, 17 Jan 2002 00:01:15 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0GN1F5C079310 for ; Thu, 17 Jan 2002 00:01:15 +0100 (CET) Date: Wed, 16 Jan 2002 23:56:54 +0100 (CET) From: Michal Mertl To: Peter Jeremy Subject: i386 atomic quad implementation (was: 64 bit counters again) In-Reply-To: <20020117090710.I72285@gsmx07.alcatel.com.au> Message-ID: MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-1934547448-1011221814=:78763" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --0-1934547448-1011221814=:78763 Content-Type: TEXT/PLAIN; charset=US-ASCII On Thu, 17 Jan 2002, Peter Jeremy wrote: > On 2002-Jan-16 22:15:13 +0100, Michal Mertl wrote: > >In my patch there's also support for 64 bit atomic ops on >=i586. That's > >one thing which shouldn't hurt anyone. That needs review - I wasn't able > >to find good documentation examples for gcc inline asm constraints > > I agree that this makes inline asm difficult to use for non-trivial > functions. Another problem is that the allowable constraint > combinations change between gcc revisions and even with different > optimization levels. Have a look at old versions of i386 atomic.h > and the regular "it doesn't work at -O0" threads for details. > I don't think we need to bother with older than 2.95 gcc. From comments in atomic.h it seems the change you're talking about was at 2.8, which is veru old. > > so I > >ended up mostly telling gcc exactly which register to use. > > As long as you did this within the constraints this should be OK. > You shouldn't need to specify register names in the actual assembler. > > I suggest you post your patch (or a pointer if it's big) to -arch or > -audit with a request for review. Before this can be committed, you'll > need MD backends for each architecture supported. > Here it is. --- atomic.h Fri Jul 7 02:38:47 2000 +++ atomic.h.new Wed Jan 16 23:54:56 2002 @@ -148,4 +148,165 @@ #endif +#if 1 +#define _ATOMIC_HAVE_QUAD +#endif + +#ifdef _ATOMIC_HAVE_QUAD +#define atomic_set_acq_quad atomic_set_quad +#define atomic_set_rel_quad atomic_set_quad +#define atomic_clear_acq_quad atomic_clear_quad +#define atomic_clear_rel_quad atomic_clear_quad +#define atomic_add_acq_quad atomic_add_quad +#define atomic_add_rel_quad atomic_add_quad +#define atomic_subtract_acq_quad atomic_subtract_quad +#define atomic_subtract_rel_quad atomic_subtract_quad +#define atomic_cmpset_acq_quad atomic_cmpset_quad +#define atomic_cmpset_rel_quad atomic_cmpset_quad +#define atomic_readandclear_quad atomic_readandclear_quad + +#if !defined(KLD_MODULE) +static __inline void atomic_store_rel_quad(volatile u_int64_t *p,u_int64_t v){ + __asm __volatile ("#atomic_store_rel_quad + movl (%%esi),%%eax + movl 4(%%esi),%%edx + 1:"MPLOCKED "cmpxchg8b (%%esi) + jnz 1b" + : + : "S" (p), "b" ((u_long)v), "c" ((u_long)((u_long *)&v)[1]) + : "eax", "edx"); +} + +static __inline void atomic_add_quad(volatile u_int64_t *p, u_int64_t v) +{ + __asm __volatile ("#atomic_add_quad + movl (%%esi),%%eax + movl 4(%%esi),%%edx + 1:movl (%%edi),%%ebx + movl 4(%%edi),%%ecx + addl %%eax,%%ebx + adcl %%edx,%%ecx + "MPLOCKED "cmpxchg8b (%%esi) + jnz 1b" + : + : "S" (p), "D" (&v) + : "eax", "ebx", "ecx", "edx"); +} + +static __inline void atomic_subtract_quad(volatile u_int64_t *p, u_int64_t v) +{ + atomic_add_quad(p,-v); +} + +static __inline void atomic_clear_quad(volatile u_int64_t *p, u_int64_t v) +{ + __asm __volatile ("#atomic_clear_quad + movl (%%esi),%%eax + movl 4(%%esi),%%edx + 1: movl (%%edi),%%ebx + movl 4(%%edi),%%ecx + notl %%eax + notl %%edx + andl %%eax,%%ebx + andl %%edx,%%ecx + movl (%%edi),%%eax + movl 4(%%edi),%%edx + "MPLOCKED "cmpxchg8b (%%esi) + jnz 1b" + : + : "S" (p), "D" (&v) + : "eax", "ebx", "ecx", "edx"); +} + +static __inline void atomic_set_quad(volatile u_int64_t *p, u_int64_t v) +{ + __asm __volatile ("#atomic_set_quad + movl (%%esi),%%eax + movl 4(%%esi),%%edx + 1: movl (%%edi),%%ebx + movl 4(%%edi),%%ecx + orl %%eax,%%ebx + orl %%edx,%%ecx + "MPLOCKED "cmpxchg8b (%%esi) + jnz 1b" + : + : "S" (p), "D" (&v) + : "eax", "ebx", "ecx", "edx"); +} + +static __inline u_int64_t atomic_load_acq_quad(volatile u_int64_t *p) +{ + u_int64_t tmp; + + __asm __volatile ("#atomic_load_acq_quad + xorl %%eax,%%eax + xorl %%ebx,%%ebx + xorl %%ecx,%%ecx + xorl %%edx,%%edx + "MPLOCKED "cmpxchg8b (%%esi) + jnz 1f + movl $0,(%%edi) + movl $0,4(%%edi) + jmp 2f +1: movl %%eax,(%%edi) + movl %%edx,4(%%edi) +2:" + : + : "D" (&tmp), "S" (p) + : "eax", "ebx", "ecx", "edx"); + return tmp; +} + +static __inline int atomic_cmpset_quad(volatile u_int64_t *dst,u_int64_t exp,u_int64_t src) +{ + int res=exp; + + __asm __volatile (MPLOCKED "#atomic_cmpset_quad + cmpxchg8b (%%esi) + setz %%al + movzbl %%al,%0" + : "=a" (res) /* 0 */ + : "S" (dst) /* 1 */, "eax" ((u_long)exp), "edx" ((u_long)((u_long *)&exp)[1]), + "ebx" ((u_long)src), "ecx" ((u_long)((u_long *)&src)[1]) + ); + return res; +} + +static __inline u_int64_t atomic_readandclear_quad(volatile u_int64_t *p) +{ + u_int64_t tmp; + + __asm __volatile ("#atomic_readandclear_quad + xorl %%eax,%%eax + xorl %%ebx,%%ebx + xorl %%ecx,%%ecx + xorl %%edx,%%edx + "MPLOCKED "cmpxchg8b (%%esi) + jnz 1f + movl $0,(%%edi) + movl $0,4(%%edi) + jmp 2f +1: movl %%eax,(%%edi) + movl %%edx,4(%%edi) +2: "MPLOCKED "cmpxchg8b (%%esi)" + : + : "D" (&tmp), "S" (p) + : "eax", "ebx", "ecx", "edx"); + return tmp; +} + +#else /* define(KLD_MODULE) */ + +void atomic_set_quad(volatile u_int64_t *, u_int64_t); +void atomic_store_rel_quad(volatile u_int64_t *, u_int64_t); +void atomic_clear_quad(volatile u_int64_t *, u_int64_t); +void atomic_add_quad(volatile u_int64_t *, u_int64_t); +void atomic_subtract_quad(volatile u_int64_t *, u_int64_t); +u_int64_t atomic_load_acq_quad(volatile u_int64_t *); +int atomic_cmpset_quad(volatile u_int64_t *,u_int64_t,u_int64_t); +u_int64_t atomic_readandclear_quad(volatile u_int64_t *); +#endif + +#endif /*defined(ATOMIC_HAVE_QUAD)*/ + #endif /* ! _MACHINE_ATOMIC_H_ */ --------------------------- 8< cut here ------ -- Michal Mertl mime@traveller.cz --0-1934547448-1011221814=:78763 X-Content-Type: TEXT/PLAIN; charset=US-ASCII; name="atomic.h.diff" X-Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: X-Content-Disposition: attachment; filename="atomic.h.diff" Content-Type: TEXT/PLAIN The following attachment was sent, but NOT saved in the Fcc copy: A Text/PLAIN segment of about 4,186 bytes. --0-1934547448-1011221814=:78763-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 16:54:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from canonware.com (dsl081-058-209.sfo1.dsl.speakeasy.net [64.81.58.209]) by hub.freebsd.org (Postfix) with ESMTP id 9801D37B402 for ; Wed, 16 Jan 2002 16:54:41 -0800 (PST) Received: by canonware.com (Postfix, from userid 1001) id 7D8C0BA; Wed, 16 Jan 2002 16:59:26 -0800 (PST) Date: Wed, 16 Jan 2002 16:59:26 -0800 From: Jason Evans To: arch@freebsd.org Subject: termcap versus terminfo Message-ID: <20020116165926.N451@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Currently, FreeBSD uses termcap for storage of terminal attributes. In order to make this work, we have had to hack up ncurses, which is normally based on terminfo. I have several problems with this: 1) It requires extra work in tracking ncurses, which is contributed source code. 2) We end up having to maintain the termcap file, whereas if we were using terminfo and submitting our changes back to the ncurses maintainer, there would be very little divergence from the pristine contrib sources. 3) With the likely exception of some of (but not all of) the other BSDs, I don't think any other modern systems use termcap. Linux and SYSV-based OSes use terminfo. In fact, Solaris does not even support the termcap APIs. 4) terminfo is standardized. It is documented in SUSv2, and in all likelihood, the Austin Group Specification (IEEE 1003.1). 5) terminfo is generally better designed. termcap has scalability issues, which we have partially solved by creating termcap.db. However, the speedups afforded by the db file only help if using the termcap library API, since termcap.db does not have a public API to allow direct access. By contrast, the terminfo database is in a compiled format that is formally documented, which allows applications to quickly access terminfo entries without using the terminfo library API. I've been thinking for over a month about this, trying to intuit the rationale for sticking with termcap, and nothing really comes to mind, except NIH. ncurses provides a compatibility shim for termcap functions, which means that unless an application wants to access the termcap file directly, using terminfo isn't even visible to applications. As mentioned above, directly accessing termcap has scalability issues, unlike terminfo. I would like to move FreeBSD to a much less hacked ncurses, which means converting to terminfo. Are there any good reasons for not making this change? Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 17:49:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from avocet.prod.itd.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by hub.freebsd.org (Postfix) with ESMTP id 5EE5737B404 for ; Wed, 16 Jan 2002 17:49:37 -0800 (PST) Received: from pool0199.cvx22-bradley.dialup.earthlink.net ([209.179.198.199] helo=mindspring.com) by avocet.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16R1gH-0006vc-00; Wed, 16 Jan 2002 17:49:29 -0800 Message-ID: <3C462D9C.D81306FD@mindspring.com> Date: Wed, 16 Jan 2002 17:49:16 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jason Evans Cc: arch@freebsd.org Subject: Re: termcap versus terminfo References: <20020116165926.N451@canonware.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jason Evans wrote: > Currently, FreeBSD uses termcap for storage of terminal attributes. In > order to make this work, we have had to hack up ncurses, which is normally > based on terminfo. I have several problems with this: > > 1) It requires extra work in tracking ncurses, which is contributed source > code. > > 2) We end up having to maintain the termcap file, whereas if we were using > terminfo and submitting our changes back to the ncurses maintainer, > there would be very little divergence from the pristine contrib sources. There are tools to convert between them. Perhaps we should use them, instead? The captoinfo/infotocap/toe/tic/infocmp family of commands were written specifically for this. ESR even had his paws in a couple of them... http://www.csclub.uwaterloo.ca/u/r-ridge/ ftp://ftp.uu.net/usenet/comp.sources.unix/volume26/mytinfo/ > 3) With the likely exception of some of (but not all of) the other BSDs, I > don't think any other modern systems use termcap. Linux and SYSV-based > OSes use terminfo. In fact, Solaris does not even support the termcap > APIs. That's because these systems don't trust the users to correct their own problems, and provide binary only terminfo files, with only the compilation tools, and no decompilation tools for modifications. This is because corporate entities like to reduce support overhead, even if it means that a clued user can't fix a problem with a vendor supplied description file (or enhance it, when a new capability exists in a terminal that the vendor chose not to use, didn't know about, or a new version of a terminal ROM came out. The other benefit to termcap entries is that they can be pushed around in environments for things like SH and xterm (and are). > 4) terminfo is standardized. It is documented in SUSv2, and in all > likelihood, the Austin Group Specification (IEEE 1003.1). POSIX signals not restarting system calls by default, and the first close of an fd to a file destroying all locks, even in the presence of multiple open instances for the file in the same program are also standardized and documented in both those places. > 5) terminfo is generally better designed. termcap has scalability issues, > which we have partially solved by creating termcap.db. However, the > speedups afforded by the db file only help if using the termcap library > API, since termcap.db does not have a public API to allow direct access. > By contrast, the terminfo database is in a compiled format that is > formally documented, which allows applications to quickly access > terminfo entries without using the terminfo library API. The way you are supposed to do this is to use "tset(1)" to export the termcap entry as an envirponment variable "TERMCAP", which prevents the lookup overhead from ocurring every again: it happens once at login, and never again. Avoiding doing this correctly causing delays in running programs is not really a valid argument against termcap. If you think the design is well thought out, consider that it uses a struct, the size of which is compiled into the programs that use its version of the library, so if you want to add a new arbitrary attribute, you basically have to recompile all code that uses the damn thing, after you extend the structure to include the new element. > I've been thinking for over a month about this, trying to intuit the > rationale for sticking with termcap, and nothing really comes to mind, > except NIH. ncurses provides a compatibility shim for termcap functions, > which means that unless an application wants to access the termcap file > directly, using terminfo isn't even visible to applications. As mentioned > above, directly accessing termcap has scalability issues, unlike terminfo. Printcap? Gettytab? XXXFOOtab? > I would like to move FreeBSD to a much less hacked ncurses, which means > converting to terminfo. Are there any good reasons for not making this > change? I think the place to move the hack boundary is still inside the ncurses code. This is a problem with the ncurses code (IMO), not a problem with BSD. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 18: 3: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from swan.prod.itd.earthlink.net (swan.mail.pas.earthlink.net [207.217.120.123]) by hub.freebsd.org (Postfix) with ESMTP id C637237B419; Wed, 16 Jan 2002 18:02:48 -0800 (PST) Received: from dialup-209.244.107.170.dial1.sanjose1.level3.net ([209.244.107.170] helo=blossom.cjclark.org) by swan.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16R1t8-000680-00; Wed, 16 Jan 2002 18:02:47 -0800 Received: (from cjc@localhost) by blossom.cjclark.org (8.11.6/8.11.3) id g0H22eV37376; Wed, 16 Jan 2002 18:02:40 -0800 (PST) (envelope-from cjc) Date: Wed, 16 Jan 2002 18:02:40 -0800 From: "Crist J . Clark" To: Terry Lambert Cc: Sheldon Hearn , FreeBSD@jovi.net, freebsd-questions@FreeBSD.ORG, freebsd-arch@FreeBSD.ORG, bug-followup@FreeBSD.ORG Subject: Re: kern/33904: secure mode bug Message-ID: <20020116180239.F35910@blossom.cjclark.org> References: <20020115210303.E31328@blossom.cjclark.org> <98823.1011171388@axl.seasidesoftware.co.za> <20020116010937.K31328@blossom.cjclark.org> <3C45E0B2.A092CB4E@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3C45E0B2.A092CB4E@mindspring.com>; from tlambert2@mindspring.com on Wed, Jan 16, 2002 at 12:21:06PM -0800 X-URL: http://people.freebsd.org/~cjc/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 12:21:06PM -0800, Terry Lambert wrote: > "Crist J . Clark" wrote: > > The settimeofday(2) call returns > > success even though the change requested by the call is not really > > done. This is somewhat questionable behavior. The documentation for > > settimeofday(2) was not clear about how this works under elevated > > securelevel(8), and in fact, the documentation is actually wrong > > (which I will fix shortly). > > This is BS. I don't think so. > It's not documented how it works in jails, either, > or under vmware. But settimeofday(2) _does_ claim to document what happens, but it is flat out wrong (out of date), Only the super-user may set the time of day or time zone. If the system is running in secure mode (see init(8)), the time may only be advanced. This limitation is imposed to prevent a malicious super-user from setting arbitrary time stamps on files. The system time can still be adjusted backwards using the adjtime(2) system call even when the system is secure. Read the comments for settime() and the rest of the code in kern_time.c, and test it. This is clearly not how things really work. -- Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 22:12:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by hub.freebsd.org (Postfix) with ESMTP id 45B7137B402 for ; Wed, 16 Jan 2002 22:12:41 -0800 (PST) Received: (from dan@localhost) by dan.emsphone.com (8.11.6/8.11.6) id g0H6Cbh10073; Thu, 17 Jan 2002 00:12:37 -0600 (CST) (envelope-from dan) Date: Thu, 17 Jan 2002 00:11:57 -0600 From: Dan Nelson To: Terry Lambert Cc: Jason Evans , arch@FreeBSD.ORG Subject: Re: termcap versus terminfo Message-ID: <20020117061157.GA84018@dan.emsphone.com> References: <20020116165926.N451@canonware.com> <3C462D9C.D81306FD@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3C462D9C.D81306FD@mindspring.com> User-Agent: Mutt/1.3.25i X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Jan 16), Terry Lambert said: > Jason Evans wrote: > > I would like to move FreeBSD to a much less hacked ncurses, which > > means converting to terminfo. Are there any good reasons for not > > making this change? [...] > > 5) terminfo is generally better designed. termcap has scalability > > issues, which we have partially solved by creating termcap.db. > > However, the speedups afforded by the db file only help if using > > the termcap library API, since termcap.db does not have a public > > API to allow direct access. By contrast, the terminfo database > > is in a compiled format that is formally documented, which > > allows applications to quickly access terminfo entries without > > using the terminfo library API. Isn't using an API better than codifying a file format? And how is an application even going to know where the system put its terminfo files when it decides to bypass the API? And the .05 second saved by having the terminal description precompiled will be in the noise when we stop caching catpages. Another drawback with terminfo is there's no provision for non-disk-based terminal descriptions. With termcap, a program simulating its own termcal (screen, or emacs for example) can simply set TERMCAP to the terminal description itself and not worry about whether the system's termcap/terminfo is right. Same thing for telnet and ssh. They both have the ability to pass TERMCAP to a remote system. Handy when connecting to OSes that don't have a description for cons25. [...] > I think the place to move the hack boundary is still inside the > ncurses code. This is a problem with the ncurses code (IMO), not a > problem with BSD. How much hacking is really involved? Every terminfo-using OS here at work (Linux, tru64, Solaris, SCO) has a different on-disk terminfo structure (surprise surprise), so the first thing I do is install ncurses with the following to generate a nice /etc/termcap-friendly lib: ./configure --prefix=/usr/local/ --enable-termcap --enable-getcap --disable-overwrite I would assume that whatever things get set by these --flags are simply #defines in a config.h file. Are there any other changes that are needed to get a true termcap-only ncurses? If you're worried about the maintaining the source terminal description file, you can always keep "terminfo.src" in CVS and run infotocap on it during the build to generate a terminfo file. -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 23: 1:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from canonware.com (dsl081-058-209.sfo1.dsl.speakeasy.net [64.81.58.209]) by hub.freebsd.org (Postfix) with ESMTP id 348B337B402 for ; Wed, 16 Jan 2002 23:01:05 -0800 (PST) Received: by canonware.com (Postfix, from userid 1001) id 80914BA; Wed, 16 Jan 2002 23:05:52 -0800 (PST) Date: Wed, 16 Jan 2002 23:05:52 -0800 From: Jason Evans To: arch@freebsd.org Subject: Re: termcap versus terminfo Message-ID: <20020116230552.O451@canonware.com> References: <20020116165926.N451@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020116165926.N451@canonware.com>; from jasone@canonware.com on Wed, Jan 16, 2002 at 04:59:26PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 04:59:26PM -0800, Jason Evans wrote: > Are there any good reasons for not making this change? A NetBSD developer I was chatting with this evening pointed out that terminfo is not extensible in a portable manner, since the compiled terminfo files use a pre-defined structure, where each element corresponds to a particular capability. While this doesn't make the compiled format useless, it does make portable access of vendor extensions (non-standard capability keys) more or less impossible. termcap doesn't have this problem. Interestingly, he also mentioned that NetBSD developers have continued to work on the original BSD curses code, and they have implemented the majority of the extensions documented in SUSv2 at this point. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 23:25:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from hawk.prod.itd.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by hub.freebsd.org (Postfix) with ESMTP id 60B1C37B416 for ; Wed, 16 Jan 2002 23:25:52 -0800 (PST) Received: from pool0014.cvx22-bradley.dialup.earthlink.net ([209.179.198.14] helo=mindspring.com) by hawk.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16R6vj-0002JJ-00; Wed, 16 Jan 2002 23:25:47 -0800 Message-ID: <3C467C45.2D7A3381@mindspring.com> Date: Wed, 16 Jan 2002 23:24:53 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jason Evans Cc: arch@freebsd.org Subject: Re: termcap versus terminfo References: <20020116165926.N451@canonware.com> <20020116230552.O451@canonware.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jason Evans wrote: > > On Wed, Jan 16, 2002 at 04:59:26PM -0800, Jason Evans wrote: > > Are there any good reasons for not making this change? > > A NetBSD developer I was chatting with this evening pointed out that > terminfo is not extensible in a portable manner, since the compiled > terminfo files use a pre-defined structure, where each element corresponds > to a particular capability. While this doesn't make the compiled format > useless, it does make portable access of vendor extensions (non-standard > capability keys) more or less impossible. termcap doesn't have this > problem. One of the same objections I raised... BSD minds think alike... 8-) > Interestingly, he also mentioned that NetBSD developers have continued to > work on the original BSD curses code, and they have implemented the > majority of the extensions documented in SUSv2 at this point. It's worth looking at; I think that abstracting the capability code from the ncurses code would be a good idea, if it could be done, though, since t's so widely used and active maintained, so let's not throw the baby out with the bathwater... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 23:43: 2 2002 Delivered-To: freebsd-arch@freebsd.org Received: from hawk.prod.itd.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by hub.freebsd.org (Postfix) with ESMTP id 33CE337B419 for ; Wed, 16 Jan 2002 23:42:53 -0800 (PST) Received: from pool0014.cvx22-bradley.dialup.earthlink.net ([209.179.198.14] helo=mindspring.com) by hawk.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16R7CD-0003pI-00; Wed, 16 Jan 2002 23:42:50 -0800 Message-ID: <3C468042.8791FBD4@mindspring.com> Date: Wed, 16 Jan 2002 23:41:54 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Dan Nelson Cc: Jason Evans , arch@FreeBSD.ORG Subject: Re: termcap versus terminfo References: <20020116165926.N451@canonware.com> <3C462D9C.D81306FD@mindspring.com> <20020117061157.GA84018@dan.emsphone.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Nelson wrote: [ ... definition of file formats ... ] > Isn't using an API better than codifying a file format? And how is an > application even going to know where the system put its terminfo files > when it decides to bypass the API? And the .05 second saved by having > the terminal description precompiled will be in the noise when we stop > caching catpages. You're actually attributing something Jason said to me, but I'll answer anyway: yes, a procedural interface is almost always better than a data interface, since it avoids things like "proc size mismatch", and the extension issue... with one exception: performance requirements constrin interface design. The precompilation is actually not the win, it's the parsing of every newline looking for a non-newline+tab combo, and a non-newline-+# combo, indicating a continuation line or a comment. After that, you have to compare every string in the set up to the delimiter for the name to see if it matches the value in the TERM environment variable. Jason was not wrong when he pointed at the overhead, it's just that there's a way around it that's not often used (and he's right that the termcap.db helps somewhat). > Another drawback with terminfo is there's no provision for > non-disk-based terminal descriptions. With termcap, a program > simulating its own termcal (screen, or emacs for example) can simply > set TERMCAP to the terminal description itself and not worry about > whether the system's termcap/terminfo is right. Same thing for telnet > and ssh. They both have the ability to pass TERMCAP to a remote > system. Handy when connecting to OSes that don't have a description > for cons25. I already noted that as one of my objections. 8-). > [...] > > I think the place to move the hack boundary is still inside the > > ncurses code. This is a problem with the ncurses code (IMO), not a > > problem with BSD. > > How much hacking is really involved? Every terminfo-using OS here at > work (Linux, tru64, Solaris, SCO) has a different on-disk terminfo > structure (surprise surprise), so the first thing I do is install > ncurses with the following to generate a nice /etc/termcap-friendly > lib: Basically, you have to seperate the capability lookup from the curses implementation itself. You also have to hack up the curses itself, or provide an inverse alias to two character capability string array so that the termcap lookup of values that would normally be in a terminfo database entry will be found. It's a little gross, in that the reference by name is to the address of a structure element that is parsed as a fixed length string out of the on disk structure, so you have to manufacture one of those from the termcap entry, or you have to make the interface more general (which is somewhat harder). If the string exceeds the allowed length, you have to also truncate the thing. It means that the resulting code is gross, unless it's seperated at the capability-by-name boundary, and then two sets of routines are written, along with the inverse lookup to handle the use of long names in terminfo-expecting code that uses the long name instead of the two character alias name (an incredibly cheap way to deaal with this would be to make a "long name" and "short name" entry in the database, but of course that would only work for the termcap, since the terminfo structure elements are fixed length, and ints can be too small for the names of themselves... and then, the termcap is slower than the terminfo, for the lookup; I guess you could put the entries first in the termcap database...). > I would assume that whatever things get set by these --flags are simply > #defines in a config.h file. Are there any other changes that are > needed to get a true termcap-only ncurses? You don't want that: you want both, or the possibility of either, on the same system. Perhaps the NetBSD code s the right way to go, but the ncurses code is far from knocked out (IMO). SYSV handled both by having a "termcap" library that had tgetXXX() functions, and it worked because the system default was terminfo, and short name aliases existed (and could be fit into a terminfo entery, since two characters will fit in an int, even if it's only 16 bits. 8-)). This worked by linking the termcap library before the curses library (otherwise, you got the terminfo implementations). The problem with "just doing this" is that (1) it's not BSD-like to require two libraries, or we'd already have a "libns" for the resolver, and (2) by the default being termcap instead of terminfo like SYSV, handling long names in reverse translation is a bit problematic (it works the other way because termcap-expecting code will always use the shor two cahracter names, but terminfo-expecting code could use either the short names or the longer names, and one of the reasons for the longer names in the first place was self-documentation). > If you're worried about the maintaining the source terminal description > file, you can always keep "terminfo.src" in CVS and run infotocap on it > during the build to generate a terminfo file. Nope. The problem is maintaining the modified ncurses code, not the database. THe termcap database is already well maintained, both by terminal vendors, and centrally. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 16 23:59:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 129B237B400; Wed, 16 Jan 2002 23:59:31 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g0H7wjs29428; Thu, 17 Jan 2002 09:58:45 +0200 (EET) (envelope-from ru) Date: Thu, 17 Jan 2002 09:58:45 +0200 From: Ruslan Ermilov To: Alfred Perlstein Cc: Garance A Drosihn , Robert Watson , Joerg Wunsch , arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020117095845.A27310@sunbay.com> References: <20020116131608.J26067@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020116131608.J26067@elvis.mu.org> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 01:16:08PM -0800, Alfred Perlstein wrote: > * Garance A Drosihn [020116 13:12] wrote: > > (I am bcc:-ing this to cvs-committers and cvs-all, with the idea > > that this discussion is also going on in freebsd-arch and thus we > > could drop it from those two cvs lists...) > > > > At 3:00 PM -0500 1/16/02, Robert Watson wrote: > > >I'm happy with the behavior being available and turned off by default, > > >but personally my feeling is that the performance/correctness tradeoff > > >leans towards correctness given the risk. And to be honest, people > > >don't usually benchmark systems based on the time it takes to render > > >a man page. :-) > > > > But it is one of those things that will make the system "seem slower" > > to them, in day-to-day use. > > > > I think the security issue is a good enough reason to turn off the > > current behavior of 'man', but I do wish there was some middle-ground > > option which was between 'zero cat pages on disk' and 'automatically > > generate all cat pages for all existing man pages'. > > > > In my case, I have about a dozen man pages that I reference a lot, and > > a lot of man pages that I never reference. If something could keep > > track of which pages were actually referenced a lot, then some system > > daemon could generate cat-versions of just those man pages. > > > > I realize that's probably a large hammer to be invoking to solve such > > a little nail of a problem, but I couldn't help but wonder if there > > was some other way to handle this. > > If $HOME/.foo has proper permissions man should cache the pages > there. > It's easy: cd ~ mkdir .foo cd .foo ln -s /usr/share/man/man? . (cd /usr/share/man; echo cat?) | xargs mkdir man -M ~/.foo cat man(1) will create private catpages for you. Cheers, -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 0: 3:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 3318437B405; Thu, 17 Jan 2002 00:03:27 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g0H838U29790; Thu, 17 Jan 2002 10:03:08 +0200 (EET) (envelope-from ru) Date: Thu, 17 Jan 2002 10:03:08 +0200 From: Ruslan Ermilov To: Garance A Drosihn Cc: Robert Watson , Joerg Wunsch , arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020117100308.B27310@sunbay.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.23i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 04:09:37PM -0500, Garance A Drosihn wrote: > (I am bcc:-ing this to cvs-committers and cvs-all, with the idea > that this discussion is also going on in freebsd-arch and thus we > could drop it from those two cvs lists...) > > At 3:00 PM -0500 1/16/02, Robert Watson wrote: > >I'm happy with the behavior being available and turned off by default, > >but personally my feeling is that the performance/correctness tradeoff > >leans towards correctness given the risk. And to be honest, people > >don't usually benchmark systems based on the time it takes to render > >a man page. :-) > > But it is one of those things that will make the system "seem slower" > to them, in day-to-day use. > > I think the security issue is a good enough reason to turn off the > current behavior of 'man', but I do wish there was some middle-ground > option which was between 'zero cat pages on disk' and 'automatically > generate all cat pages for all existing man pages'. > periodic.conf(5)'s weekly_catman_enable=YES seens to be a good option here, no? > In my case, I have about a dozen man pages that I reference a lot, and > a lot of man pages that I never reference. If something could keep > track of which pages were actually referenced a lot, then some system > daemon could generate cat-versions of just those man pages. > catpages take less space than manpages, so I don't see a reson to fine grain this. For tonight's -CURRENT: du -skc /usr/share/man/man* | tail -1 7800 total du -skc /usr/share/man/cat* | tail -1 6599 total Cheers, -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 0:21:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id E62D337B405 for ; Thu, 17 Jan 2002 00:21:18 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16R7pO-0005SP-00; Thu, 17 Jan 2002 10:23:18 +0200 From: Sheldon Hearn To: Terry Lambert Cc: Dan Nelson , Jason Evans , arch@FreeBSD.ORG Subject: Re: termcap versus terminfo In-reply-to: Your message of "Wed, 16 Jan 2002 23:41:54 PST." <3C468042.8791FBD4@mindspring.com> Date: Thu, 17 Jan 2002 10:23:18 +0200 Message-ID: <20980.1011255798@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Could you gentlemen take a quick look at PR misc/33906 and confirm that you agree with the approach proposed for handling the way we've hacked up ncurses? Thanks, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 0:55: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from snipe.prod.itd.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by hub.freebsd.org (Postfix) with ESMTP id 9C34537B416 for ; Thu, 17 Jan 2002 00:55:01 -0800 (PST) Received: from pool0014.cvx22-bradley.dialup.earthlink.net ([209.179.198.14] helo=mindspring.com) by snipe.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16R8Jx-0003cT-00; Thu, 17 Jan 2002 00:54:53 -0800 Message-ID: <3C46911C.9D267BE@mindspring.com> Date: Thu, 17 Jan 2002 00:53:48 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Sheldon Hearn Cc: Dan Nelson , Jason Evans , arch@FreeBSD.ORG Subject: Re: termcap versus terminfo References: <20980.1011255798@axl.seasidesoftware.co.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Sheldon Hearn wrote: > Could you gentlemen take a quick look at PR misc/33906 and confirm that > you agree with the approach proposed for handling the way we've hacked > up ncurses? THere is already a tput, tset, clear, and reset, so they are not a good ideac to install in /usr/local as a result of the prot, since they coulde conflict. The tic, toe, infocmp, captoinfo, and infotocap are OK, and should be part of the port. The share/terminfo should probably not be included at all. The reason the original ncurses is marked "FORBIDDEN" is the danger that it would displace the modified curses library in the hearts and minds of programs everywhere, and end up causing the use of terminfo. THis argues against the terminfo database being sidtributes as part of the port in binary form, which is what the share/terminfo target does (though I would not object to the terminfo source file being distributed in uncompiled form, which is not addressed by the change to leave it out entirely...). So my suggestions: 1) remove the installation of tput, tset, clear, and reset 2) Install the /share/terminfo/terminfosrc (or whatever it's called; I didn't bother to look). It's kind of ugly that tic will still create real terminfo files, and that it doesn't do the conversion only, by default, but since most access will be via the infotocap script, that is OK. As far as the "migrate to" argument is concerned, there's some merit to the idea of leaving out the "captoinfo" (as in, "I can convert that file to Word format for you, but I can't convert it back into Word Perfect format, thanks!"). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 1: 4:29 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id 4B6A437B404 for ; Thu, 17 Jan 2002 01:04:26 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16R8VD-0005i1-00; Thu, 17 Jan 2002 11:06:31 +0200 From: Sheldon Hearn To: Terry Lambert Cc: Dan Nelson , Jason Evans , arch@FreeBSD.ORG Subject: Re: termcap versus terminfo In-reply-to: Your message of "Thu, 17 Jan 2002 00:53:48 PST." <3C46911C.9D267BE@mindspring.com> Date: Thu, 17 Jan 2002 11:06:31 +0200 Message-ID: <21948.1011258391@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Jan 2002 00:53:48 PST, Terry Lambert wrote: > THere is already a tput, tset, clear, and reset, so they > are not a good ideac to install in /usr/local as a result of > the prot, since they coulde conflict. > [...] Thanks for the feedback, Terry. I've added your comments to the audit trail of the PR. Oh, and thanks for not cross-posting directly to gnats and a public mailing list. :-) Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 1:15:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from snipe.prod.itd.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by hub.freebsd.org (Postfix) with ESMTP id BCEC837B402 for ; Thu, 17 Jan 2002 01:15:49 -0800 (PST) Received: from pool0014.cvx22-bradley.dialup.earthlink.net ([209.179.198.14] helo=mindspring.com) by snipe.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16R8eB-00057O-00; Thu, 17 Jan 2002 01:15:47 -0800 Message-ID: <3C4695FF.3E04F8CB@mindspring.com> Date: Thu, 17 Jan 2002 01:14:39 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Sheldon Hearn Cc: Dan Nelson , Jason Evans , arch@FreeBSD.ORG Subject: Re: termcap versus terminfo References: <21948.1011258391@axl.seasidesoftware.co.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Sheldon Hearn wrote: > On Thu, 17 Jan 2002 00:53:48 PST, Terry Lambert wrote: > > THere is already a tput, tset, clear, and reset, so they > > are not a good ideac to install in /usr/local as a result of > > the prot, since they coulde conflict. > > [...] > > Thanks for the feedback, Terry. I've added your comments to the audit > trail of the PR. If I'd known you were going to do that, I would have typed slower, and the spelling would have been correct as a result... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 1:44:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.freebsd-corp-net-guide.com (mail.freebsd-corp-net-guide.com [206.29.169.15]) by hub.freebsd.org (Postfix) with ESMTP id 7012E37B417 for ; Thu, 17 Jan 2002 01:44:41 -0800 (PST) Received: from tedm.placo.com (nat-rtr.freebsd-corp-net-guide.com [206.29.168.154]) by mail.freebsd-corp-net-guide.com (8.11.1/8.11.1) with SMTP id g0H9idR78655; Thu, 17 Jan 2002 01:44:40 -0800 (PST) (envelope-from tedm@toybox.placo.com) From: "Ted Mittelstaedt" To: Cc: Subject: Re: termcap versus terminfo Date: Thu, 17 Jan 2002 01:44:39 -0800 Message-ID: <001501c19f3b$94c35280$1401a8c0@tedm.placo.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3155.0 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG For starters I'm not particularly a terminfo supporter, my main concern is seeing something that's easy to use and works. But I feel the current FreeBSD scheme doesen't work - at least not from an administration standpoint. The current FreeBSD scheme with the compiled termcap.db has terrible documentation. In fact the only mention of the need to use cap_mkdb to build termcap.db is in the cap_mkdb man page, and it's not even a mention, it's just a link in SEE ALSO. It's not mentioned in the man page for termcap. I don't see as how any admin is going to figure out how to add a terminal description other than trial and error so what "user friendliness" gained by holding to the human-readable /etc/termcap format is lost in the current scheme and really shouldn't be an issue to use in deciding between termcap and terminfo. With regards to Terry's comments about corporate entities wanting to reduce support overhead, I think this is a bit pedantic. Sun USED to keep infocmp separate in the Toolbox, but in Solaris 8, infocmp is included and can be used to extract terminfo source, and tic which is also included can be used to compile it back in. How many other commercial UNIX vendors are still unbundling infocmp I would ask - my guess is very few. I don't see that the infocmp-edit-tic way of modifing the termcap entry is superior or inferior to the edit-termcap-and-run-cap_mkdb, from an admin's point of view. The principle need admins have to tamper with terminfo or termcap entries is to work around deviations from whatever standard emulation their display devices are supposed to be following. In the old days when terminal manufacturers would fix bugs and release new ROMS, the cost to an enterprise of running around and replacing them in their terminals pretty much guarenteed that once a terminal was installed, it would never be touched. This meshed well with a central termcap authority controlling termcap entries in all UNIX everywhere because things didn't change much in the displays people were using, so it made sense to leverage effort to figure out the changes in new ROMS. Today though, most people use terminal emulation on PC's or hardware devices with upgradable hardware, so there's much wider deviation from terminal emulation "standards" like the vt102, ANSI, Wyse60 and so on. And the deviations occur not just from emulation program to emulation program but from version to version. So, having a single unified set of terminal entries that's kept maintained centrally isn't as important as it once was, because everyone has to change everything for their own stuff. What's more important is the default termcap supplied with the system have a set of common emulations, that people can use as starting points for their local mods. Ted Mittelstaedt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 1:54:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id 12ADC37B400 for ; Thu, 17 Jan 2002 01:54:40 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16R9Ho-0005qu-00; Thu, 17 Jan 2002 11:56:44 +0200 From: Sheldon Hearn To: "Ted Mittelstaedt" Cc: arch@freebsd.org, jasone@canonware.com Subject: Re: termcap versus terminfo In-reply-to: Your message of "Thu, 17 Jan 2002 01:44:39 PST." <001501c19f3b$94c35280$1401a8c0@tedm.placo.com> Date: Thu, 17 Jan 2002 11:56:44 +0200 Message-ID: <22499.1011261404@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Jan 2002 01:44:39 PST, "Ted Mittelstaedt" wrote: > The current FreeBSD scheme with the compiled termcap.db > has terrible documentation. In fact the only mention of the > need to use cap_mkdb to build termcap.db is in the cap_mkdb > man page, and it's not even a mention, it's just a link in > SEE ALSO. It's not mentioned in the man page for termcap. Are there any objections to rebuilding termcap.db once a day from the periodic scripts? Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 2: 7:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 0290037B405 for ; Thu, 17 Jan 2002 02:07:23 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g0HA3Jv47634; Thu, 17 Jan 2002 12:03:19 +0200 (EET) (envelope-from ru) Date: Thu, 17 Jan 2002 12:03:19 +0200 From: Ruslan Ermilov To: Ted Mittelstaedt Cc: arch@FreeBSD.ORG, jasone@canonware.com Subject: Re: termcap versus terminfo Message-ID: <20020117120319.L27310@sunbay.com> References: <001501c19f3b$94c35280$1401a8c0@tedm.placo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <001501c19f3b$94c35280$1401a8c0@tedm.placo.com> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Jan 17, 2002 at 01:44:39AM -0800, Ted Mittelstaedt wrote: > > For starters I'm not particularly a terminfo supporter, my main > concern is seeing something that's easy to use and works. But > I feel the current FreeBSD scheme doesen't work - at least not > from an administration standpoint. > > The current FreeBSD scheme with the compiled termcap.db > has terrible documentation. In fact the only mention of the > need to use cap_mkdb to build termcap.db is in the cap_mkdb > man page, and it's not even a mention, it's just a link in > SEE ALSO. It's not mentioned in the man page for termcap. > Not true. It's referenced from the FILES section of termcap(5). > I don't see as how any admin is going to figure out how to add a terminal > description other than trial and error so what "user friendliness" gained by > holding to the human-readable /etc/termcap format is lost in the current > scheme and really shouldn't be an issue to use in deciding between > termcap and terminfo. > What's wrong with reading termcap(5)? :-) Cheers, -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 2:14: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id 4814337B404 for ; Thu, 17 Jan 2002 02:13:31 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g0HA9Af48562; Thu, 17 Jan 2002 12:09:10 +0200 (EET) (envelope-from ru) Date: Thu, 17 Jan 2002 12:09:10 +0200 From: Ruslan Ermilov To: Garrett Wollman Cc: Alexey Zelkin , arch@FreeBSD.ORG Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020117120910.M27310@sunbay.com> References: <20020116154210.A74132@uriah.heep.sax.de> <20020116174352.C13904@sunbay.com> <20020116171144.C18043@uriah.heep.sax.de> <20020116183712.G13904@sunbay.com> <20020116181625.B757@uriah.heep.sax.de> <20020116195429.J13904@sunbay.com> <20020116200128.L13904@sunbay.com> <20020116203816.B99450@ark.cris.net> <200201161933.g0GJXGn72831@khavrinen.lcs.mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200201161933.g0GJXGn72831@khavrinen.lcs.mit.edu> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 16, 2002 at 02:33:16PM -0500, Garrett Wollman wrote: > < said: > > > unsetenv(3) ? > > I'd like to suggest that man(1) should clean out the environment > before running groff. The normal policy of the security weenies is > ``deny by default'' and I don't see any reason why it should be > different in this case. > It's moot whether it should be done always, or when man(1) runs with setuid privileges turned on. I think man(1) should allow user to provide custom GROFF_TMAC_PATH for his own manpages, no? Garrett, please don't exclude my email from replies, I didn't ask for it by setting up Mail-Followup-To:. Thanks. Cheers, -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 2:53:25 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id BF12A37B405; Thu, 17 Jan 2002 02:52:55 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16RACZ-00064D-00; Thu, 17 Jan 2002 12:55:23 +0200 From: Sheldon Hearn To: arch@freebsd.org Cc: brian@FreeBSD.org Subject: Re: termcap versus terminfo In-reply-to: Your message of "Thu, 17 Jan 2002 11:56:44 +0200." <22499.1011261404@axl.seasidesoftware.co.za> Date: Thu, 17 Jan 2002 12:55:23 +0200 Message-ID: <23324.1011264923@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Jan 2002 11:56:44 +0200, Sheldon Hearn wrote: > Are there any objections to rebuilding termcap.db once a day from the > periodic scripts? I should know better than to ask questions like that on this list without supporting patches. See attached diff output for an idea of what I'm proposing. Brian, I've copied you on this because you're the periodic(8) maintainer. Ciao, Sheldon. Index: etc/defaults/periodic.conf =================================================================== RCS file: /home/ncvs/src/etc/defaults/periodic.conf,v retrieving revision 1.18 diff -u -d -r1.18 periodic.conf --- etc/defaults/periodic.conf 7 Dec 2001 23:57:38 -0000 1.18 +++ etc/defaults/periodic.conf 17 Jan 2002 10:48:30 -0000 @@ -64,6 +64,12 @@ daily_clean_hoststat_days=3 # If not modified for daily_clean_hoststat_verbose="YES" # Mention files deleted +# 160.rebuild-db +daily_rebuild_db_aliases="YES" +daily_rebuild_db_login_conf="YES" +daily_rebuild_db_passwd="YES" +daily_rebuild_db_termcap="YES" + # 200.backup-passwd daily_backup_passwd_enable="YES" # Backup passwd & group Index: etc/periodic/daily/Makefile =================================================================== RCS file: /home/ncvs/src/etc/periodic/daily/Makefile,v retrieving revision 1.8 diff -u -d -r1.8 Makefile --- etc/periodic/daily/Makefile 1 Oct 2001 06:27:44 -0000 1.8 +++ etc/periodic/daily/Makefile 17 Jan 2002 10:48:30 -0000 @@ -6,6 +6,7 @@ 130.clean-msgs \ 140.clean-rwho \ 150.clean-hoststat \ + 160.rebuild-db \ 200.backup-passwd \ 210.backup-aliases \ 220.backup-distfile \ Index: etc/periodic/daily/160.rebuild-db =================================================================== RCS file: etc/periodic/daily/160.rebuild-db diff -N etc/periodic/daily/160.rebuild-db --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ etc/periodic/daily/160.rebuild-db 17 Jan 2002 10:48:30 -0000 @@ -0,0 +1,54 @@ +#!/bin/sh +# +# $FreeBSD$ +# +# Rebuild db files in /etc that exist for performance reasons. +# + +# If there is a global system configuration file, suck it in. +# +if [ -r /etc/defaults/periodic.conf ] +then + . /etc/defaults/periodic.conf + source_periodic_confs +fi + +rc=0 + +for i in login.conf termcap; do + eval opt=\$daily_rebuild_db_`echo $i | tr . _` + case "$opt" in + [Yy][Ee][Ss]) + cap_mkdb /etc/$i + if [ $? -ne 0 ]; then + rc=1 + fi + ;; + *) + ;; + esac +done + +case "$daily_rebuild_db_aliases" in + [Yy][Ee][Ss]) + newaliases + if [ $? -ne 0 ]; then + rc=1 + fi + ;; + *) + ;; +esac + +case "$daily_rebuild_db_passwd" in + [Yy][Ee][Ss]) + pwd_mkdb /etc/master.passwd + if [ $? -ne 0 ]; then + rc=1 + fi + ;; + *) + ;; +esac + +exit $rc Index: share/man/man5/periodic.conf.5 =================================================================== RCS file: /home/ncvs/src/share/man/man5/periodic.conf.5,v retrieving revision 1.39 diff -u -d -r1.39 periodic.conf.5 --- share/man/man5/periodic.conf.5 17 Jan 2002 10:35:48 -0000 1.39 +++ share/man/man5/periodic.conf.5 17 Jan 2002 10:48:32 -0000 @@ -334,6 +334,40 @@ .Dq YES if you want to run .Pa /etc/news.expire . +.It Va daily_rebuild_db_aliases +.Pq Vt bool +Set to +.Dq YES +to rebuild +.Pa /etc/aliases.db +from +.Pa /etc/aliases . +.It Va daily_rebuild_db_login_conf +.Pq Vt bool +Set to +.Dq YES +to rebuild +.Pa /etc/login.conf.db +from +.Pa /etc/login.conf . +.It Va daily_rebuild_db_passwd +.Pq Vt bool +Set to +.Dq YES +to rebuild +.Pa /etc/pwd.db +and +.Pa /etc/spwd.db +from +.Pa /etc/master.passwd . +.It Va daily_rebuild_db_termcap +.Pq Vt bool +Set to +.Dq YES +to rebuild +.Pa /etc/termcap.db +from +.Pa /etc/termcap . .It Va daily_status_disks_enable .Pq Vt bool Set to To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 3:12: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.freebsd-corp-net-guide.com (mail.freebsd-corp-net-guide.com [206.29.169.15]) by hub.freebsd.org (Postfix) with ESMTP id 1829037B417; Thu, 17 Jan 2002 03:12:01 -0800 (PST) Received: from tedm.placo.com (nat-rtr.freebsd-corp-net-guide.com [206.29.168.154]) by mail.freebsd-corp-net-guide.com (8.11.1/8.11.1) with SMTP id g0HBC0R79202; Thu, 17 Jan 2002 03:12:00 -0800 (PST) (envelope-from tedm@toybox.placo.com) From: "Ted Mittelstaedt" To: "Ruslan Ermilov" Cc: , Subject: RE: termcap versus terminfo Date: Thu, 17 Jan 2002 03:12:00 -0800 Message-ID: <002001c19f47$c8aa8f80$1401a8c0@tedm.placo.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3155.0 Importance: Normal In-Reply-To: <20020117120319.L27310@sunbay.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >-----Original Message----- >From: Ruslan Ermilov [mailto:ru@FreeBSD.ORG] >Sent: Thursday, January 17, 2002 2:03 AM >To: Ted Mittelstaedt >Cc: arch@FreeBSD.ORG; jasone@canonware.com >Subject: Re: termcap versus terminfo > > >Not true. It's referenced from the FILES section of termcap(5). > You mean right above the sentence about termcap being replaced by terminfo. (on System V, yet) Uh huh. Just gives you a warm fuzzy about the applicability of the manpage, doesen't it? :-) >What's wrong with reading termcap(5)? :-) > Hey, you got a point there - Small is Beautiful. The kernel configuration instructions should be so concise. :-) Ted To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 3:16:25 2002 Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (gw.Awfulhak.org [217.204.245.18]) by hub.freebsd.org (Postfix) with ESMTP id E6CB137B402; Thu, 17 Jan 2002 03:16:20 -0800 (PST) Received: from hak.lan.Awfulhak.org (root@hak.lan.Awfulhak.org [fec0::1:12]) by Awfulhak.org (8.11.6/8.11.6) with ESMTP id g0HBGJN53414; Thu, 17 Jan 2002 11:16:19 GMT (envelope-from brian@freebsd-services.com) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.6/8.11.6) with ESMTP id g0HBGCL32118; Thu, 17 Jan 2002 11:16:12 GMT (envelope-from brian@freebsd-services.com) Message-Id: <200201171116.g0HBGCL32118@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Sheldon Hearn Cc: arch@freebsd.org, brian@freebsd.org, brian@freebsd-services.com Subject: Re: termcap versus terminfo In-Reply-To: Message from Sheldon Hearn of "Thu, 17 Jan 2002 12:55:23 +0200." <23324.1011264923@axl.seasidesoftware.co.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 17 Jan 2002 11:16:12 +0000 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I think it might be better to have a job like this that just checks the db source times against the db times and complains when the sources are newer. If people are actually updating the sources without running the relevant _mkdb program, then maybe that needs to be addressed (more loudly) rather than fixing the problem ? > On Thu, 17 Jan 2002 11:56:44 +0200, Sheldon Hearn wrote: > > > Are there any objections to rebuilding termcap.db once a day from the > > periodic scripts? > > I should know better than to ask questions like that on this list > without supporting patches. See attached diff output for an idea of > what I'm proposing. > > Brian, I've copied you on this because you're the periodic(8) > maintainer. > > Ciao, > Sheldon. [patch elided] -- Brian http://www.freebsd-services.com/ Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 3:25:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id CEF8B37B404; Thu, 17 Jan 2002 03:25:56 -0800 (PST) Received: from pool0014.cvx22-bradley.dialup.earthlink.net ([209.179.198.14] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16RAg0-0004iF-00; Thu, 17 Jan 2002 03:25:48 -0800 Message-ID: <3C46B45A.27C8AB4D@mindspring.com> Date: Thu, 17 Jan 2002 03:24:10 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Brian Somers Cc: Sheldon Hearn , arch@freebsd.org, brian@freebsd.org Subject: Re: termcap versus terminfo References: <200201171116.g0HBGCL32118@hak.lan.Awfulhak.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Brian Somers wrote: > I think it might be better to have a job like this that just checks > the db source times against the db times and complains when the > sources are newer. > > If people are actually updating the sources without running the > relevant _mkdb program, then maybe that needs to be addressed (more > loudly) rather than fixing the problem ? I was actually thinking that if the db was not newer than the termcap, it should just have the application complain, and do the search the old, slow way. In fact, I'm not sure that it doesn't. 8-). Someone might want to look, and while they are there, add it, if it's not there already... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 3:36:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id B239637B404 for ; Thu, 17 Jan 2002 03:36:16 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.6/8.11.2) id g0HBa4h60393; Thu, 17 Jan 2002 13:36:04 +0200 (EET) (envelope-from ru) Date: Thu, 17 Jan 2002 13:36:04 +0200 From: Ruslan Ermilov To: Ted Mittelstaedt Cc: arch@FreeBSD.ORG, jasone@canonware.com Subject: Re: termcap versus terminfo Message-ID: <20020117133604.A59128@sunbay.com> References: <20020117120319.L27310@sunbay.com> <002001c19f47$c8aa8f80$1401a8c0@tedm.placo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <002001c19f47$c8aa8f80$1401a8c0@tedm.placo.com> User-Agent: Mutt/1.3.23i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Jan 17, 2002 at 03:12:00AM -0800, Ted Mittelstaedt wrote: > >-----Original Message----- > >From: Ruslan Ermilov [mailto:ru@FreeBSD.ORG] > >Sent: Thursday, January 17, 2002 2:03 AM > >To: Ted Mittelstaedt > >Cc: arch@FreeBSD.ORG; jasone@canonware.com > >Subject: Re: termcap versus terminfo > > > > > >Not true. It's referenced from the FILES section of termcap(5). > > > > You mean right above the sentence about termcap being replaced by > terminfo. (on System V, yet) Uh huh. Just gives you a warm fuzzy > about the applicability of the manpage, doesen't it? :-) > You wrote: : The current FreeBSD scheme with the compiled termcap.db : has terrible documentation. In fact the only mention of the : need to use cap_mkdb to build termcap.db is in the cap_mkdb : man page, and it's not even a mention, it's just a link in : SEE ALSO. It's not mentioned in the man page for termcap. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ It _is_ mentioned here: : TERMCAP(5) FreeBSD File Formats Manual TERMCAP(5) : : NAME : termcap - terminal capability data base [...] : FILES : /usr/share/misc/termcap File containing terminal descriptions. : /usr/share/misc/termcap.db Hash database file containing terminal : descriptions (see cap_mkdb(1)). ^^^^^^^^ : : SEE ALSO : cap_mkdb(1), ex(1), more(1), tset(1), ul(1), vi(1), ncurses(3), ^^^^^^^^ : printf(3), termcap(3), term(5) Looking at the wrong section manpage? :-) Cheers, -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 3:37:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailhub.fokus.gmd.de (mailhub.fokus.gmd.de [193.174.154.14]) by hub.freebsd.org (Postfix) with ESMTP id 6E6A037B400; Thu, 17 Jan 2002 03:37:55 -0800 (PST) Received: from beagle (beagle [193.175.132.100]) by mailhub.fokus.gmd.de (8.11.6/8.11.6) with ESMTP id g0HBbqR11244; Thu, 17 Jan 2002 12:37:52 +0100 (MET) Date: Thu, 17 Jan 2002 12:37:52 +0100 (CET) From: Harti Brandt To: Ruslan Ermilov Cc: Ted Mittelstaedt , , Subject: Re: termcap versus terminfo In-Reply-To: <20020117120319.L27310@sunbay.com> Message-ID: <20020117123219.R67162-100000@beagle.fokus.gmd.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Jan 2002, Ruslan Ermilov wrote: RE>On Thu, Jan 17, 2002 at 01:44:39AM -0800, Ted Mittelstaedt wrote: RE>> RE>> For starters I'm not particularly a terminfo supporter, my main RE>> concern is seeing something that's easy to use and works. But RE>> I feel the current FreeBSD scheme doesen't work - at least not RE>> from an administration standpoint. RE>> RE>> The current FreeBSD scheme with the compiled termcap.db RE>> has terrible documentation. In fact the only mention of the RE>> need to use cap_mkdb to build termcap.db is in the cap_mkdb RE>> man page, and it's not even a mention, it's just a link in RE>> SEE ALSO. It's not mentioned in the man page for termcap. RE>> RE>Not true. It's referenced from the FILES section of termcap(5). RE> RE>> I don't see as how any admin is going to figure out how to add a terminal RE>> description other than trial and error so what "user friendliness" gained by RE>> holding to the human-readable /etc/termcap format is lost in the current RE>> scheme and really shouldn't be an issue to use in deciding between RE>> termcap and terminfo. RE>> RE>What's wrong with reading termcap(5)? :-) It's misleading because of numerous references to terminfo(5). So we have a terminfo man page, but no terminfo. terminfo(5) in turn refers to term(5) which describes files we don't use. harti NB: neither terminfo nor termcap is in the new POSIX. The only mention there is to 'historic implementations [of displays] which use either termcap or terminfo' :-) So, from the standard point of view, we are free to do it either way. -- harti brandt, http://www.fokus.gmd.de/research/cc/cats/employees/hartmut.brandt/private brandt@fokus.fhg.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 4:58:13 2002 Delivered-To: freebsd-arch@freebsd.org Received: from livroceres.com.br (c62196.upc-c.chello.nl [212.187.62.196]) by hub.freebsd.org (Postfix) with SMTP id DA3D937B405; Thu, 17 Jan 2002 04:54:39 -0800 (PST) From: "Livroceres" Subject: Informativo Date: Thu, 17 Jan 2002 10:52:49 -0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_000C_01C19F45.1EAD05A0" X-Priority: 3 X-MSMail-Priority: Normal X-Unsent: 1 X-MIMEOLE: Produced By Microsoft MimeOLE V5.00.2314.1300 Message-Id: <20020117125439.DA3D937B405@hub.freebsd.org> To: undisclosed-recipients:; Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. ------=_NextPart_000_000C_01C19F45.1EAD05A0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable LIVROCERES LTDA=20 AGRONOMIA - VETERIN=C1RIA - ZOOTECNIA Fone / Fax: (0xx19) 3433-3733 & 3433-2277 Email: livroceres@livroceres.com.br=20 HomePage: http://www.livroceres.com.br Estaremos enviando quinzenalmente informa=E7=F5es de livros com assuntos = t=E9cnicos sobre: =20 - arquitetura / paisagismo / jardinagem - agropecu=E1ria - agricultura - entomologia - fitopatologia - controle qu=EDmico de plantas infestantes - silvicultura (florestas) - veterin=E1ria e zootecnia Caso n=E3o deseje receber esses informativos, clique aqui para = excluirmos seu e-mail de nosso cadastro. =20 (obs: n=E3o clique em responder (reply), clique no link indicado) Obrigado xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Esta mensagem =E9 enviada com a complac=EAncia da nova legisla=E7=E3o = sobre o Correio Eletr=F4nico, Se=E7=E3o 301, Par=E1grafo (a) (3) (c) = Decreto =A7 1618, T=EDtulo Terceiro aprovado pelo 105=BA Congresso Base = das Normativas Internacionais sobre o SPAM. Um e-mail n=E3o poder=E1 ser = considerado SPAM quando incluir uma forma de ser removido. E-MARKETING = N=C3O =C9 SPAM. ------=_NextPart_000_000C_01C19F45.1EAD05A0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
LIVROCERES LTDA
AGRONOMIA - = VETERIN=C1RIA -=20 ZOOTECNIA
Fone / = Fax: (0xx19)=20 3433-3733 & 3433-2277
HomePage: http://www.livroceres.com.br
 
Estaremos = enviando=20 quinzenalmente informa=E7=F5es de livros com assuntos = t=E9cnicos=20 sobre:
 
- arquitetura = / paisagismo=20 / jardinagem
-=20 agropecu=E1ria
 -=20 agricultura
-=20 entomologia
-=20 fitopatologia
- = controle qu=EDmico de=20 plantas infestantes
- = silvicultura=20 (florestas)
- = veterin=E1ria e=20 zootecnia
 
Caso = n=E3o deseje receber=20 esses informativos, clique aqui para excluirmos seu = e-mail de=20 nosso cadastro.
 
(obs: n=E3o clique = em responder=20 (reply), clique no link indicado)
 
Obrigado
 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx= xxxxxxxxxx
Esta mensagem =E9 = enviada com a=20 complac=EAncia da nova legisla=E7=E3o sobre o Correio Eletr=F4nico, = Se=E7=E3o 301, Par=E1grafo=20 (a) (3) (c) Decreto =A7 1618, T=EDtulo Terceiro aprovado pelo 105=BA = Congresso Base=20 das Normativas Internacionais sobre o SPAM. Um e-mail n=E3o poder=E1 ser = considerado=20 SPAM quando incluir uma forma de ser removido. E-MARKETING N=C3O = =C9=20 SPAM.
------=_NextPart_000_000C_01C19F45.1EAD05A0-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 7:27:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from nagual.pp.ru (pobrecita.freebsd.ru [194.87.13.42]) by hub.freebsd.org (Postfix) with ESMTP id 1CBD837B405 for ; Thu, 17 Jan 2002 07:27:14 -0800 (PST) Received: (from ache@localhost) by nagual.pp.ru (8.11.6/8.11.6) id g0HFQTw77142; Thu, 17 Jan 2002 18:26:29 +0300 (MSK) (envelope-from ache) Date: Thu, 17 Jan 2002 18:26:25 +0300 From: "Andrey A. Chernov" To: Sheldon Hearn Cc: Ted Mittelstaedt , arch@FreeBSD.ORG, jasone@canonware.com Subject: Re: termcap versus terminfo Message-ID: <20020117152624.GA76988@nagual.pp.ru> References: <001501c19f3b$94c35280$1401a8c0@tedm.placo.com> <22499.1011261404@axl.seasidesoftware.co.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <22499.1011261404@axl.seasidesoftware.co.za> User-Agent: Mutt/1.3.24i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Jan 17, 2002 at 11:56:44 +0200, Sheldon Hearn wrote: > > Are there any objections to rebuilding termcap.db once a day from the > periodic scripts? Yes. Why not once a months? Once a year? Once a minute? or second? If this check ever should exist, it must be in the cgetent() itself, returning proper error code for application. -- Andrey A. Chernov http://ache.pp.ru/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 7:35:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id CBAED37B41A for ; Thu, 17 Jan 2002 07:35:25 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16REbS-0006kJ-00; Thu, 17 Jan 2002 17:37:22 +0200 From: Sheldon Hearn To: "Andrey A. Chernov" Cc: Ted Mittelstaedt , arch@FreeBSD.ORG, jasone@canonware.com Subject: Re: termcap versus terminfo In-reply-to: Your message of "Thu, 17 Jan 2002 18:26:25 +0300." <20020117152624.GA76988@nagual.pp.ru> Date: Thu, 17 Jan 2002 17:37:22 +0200 Message-ID: <25934.1011281842@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Jan 2002 18:26:25 +0300, "Andrey A. Chernov" wrote: > Yes. Why not once a months? Once a year? Once a minute? or second? Suggestion withdrawn. Your objections and Brians highlight what an icky area this is. At best, the curses code should whine the way sendmail does about /etc/aliases and then use the flat text file. Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 8:19:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (esplanaden.cybercity.dk [212.242.40.114]) by hub.freebsd.org (Postfix) with ESMTP id 535D837B405 for ; Thu, 17 Jan 2002 08:19:39 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0HGGtl06168; Thu, 17 Jan 2002 17:16:55 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: "Andrey A. Chernov" Cc: Sheldon Hearn , Ted Mittelstaedt , arch@FreeBSD.ORG, jasone@canonware.com Subject: Re: termcap versus terminfo In-Reply-To: Your message of "Thu, 17 Jan 2002 18:26:25 +0300." <20020117152624.GA76988@nagual.pp.ru> Date: Thu, 17 Jan 2002 17:16:55 +0100 Message-ID: <6166.1011284215@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020117152624.GA76988@nagual.pp.ru>, "Andrey A. Chernov" writes: >On Thu, Jan 17, 2002 at 11:56:44 +0200, Sheldon Hearn wrote: >> >> Are there any objections to rebuilding termcap.db once a day from the >> periodic scripts? Put a Makefile on it to do it if needed ? >Yes. Why not once a months? Once a year? Once a minute? or second? You mean like the Daylight-savings-change daemon :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 8:45: 0 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 4286637B416 for ; Thu, 17 Jan 2002 08:44:50 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0HGim4a061844 for ; Thu, 17 Jan 2002 17:44:48 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0HGimLd061841 for ; Thu, 17 Jan 2002 17:44:48 +0100 (CET) Date: Thu, 17 Jan 2002 17:44:48 +0100 (CET) From: Michal Mertl To: arch@freebsd.org Subject: Re: 64 bit counters again Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I promise that if there isn't anything new from you this is the last time I'm talking about it. I wan't to inform you that I tried to look at some system pushing data with different size/implementation network counters. I did my last test on dual PIII 750. I don't know, of any good way to measure the load, so I just run vmstat -w1 (and calculated average idle) while pushing the data and also looked at the throughput at 100Mbit Full-Duplex. System was performing about 10000 interrupts and 15000 packets per second. I didn't notice any difference between using 32 bits non atomic operations (3 clocks per op) or 64 bit atomic (lock;cmpxchg8b - 50 clocks). I did also measure it on single Duron 800 with the same result. It was TCP traffic so there were at least three classes of counters updated - interface, ip and tcp. Interface counters were rather cheap because fxp does updating only once per second but protocol were I think worse. Just a dumb guess is that there were 5 adds per packet so it means losing 15000*47*5=3525000 clocks. Judge by yourself. -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 10: 9:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id 5C0D937B402; Thu, 17 Jan 2002 10:09:22 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.11.3/8.11.3) with ESMTP id g0HI9Je125422; Thu, 17 Jan 2002 13:09:19 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <20020117100308.B27310@sunbay.com> References: <20020117100308.B27310@sunbay.com> Date: Thu, 17 Jan 2002 13:09:18 -0500 To: Ruslan Ermilov From: Garance A Drosihn Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Cc: Robert Watson , Joerg Wunsch , arch@FreeBSD.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.1 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 10:03 AM +0200 1/17/02, Ruslan Ermilov wrote: >On Wed, Jan 16, 2002 at 04:09:37PM -0500, Garance A Drosihn wrote: > > In my case, I have about a dozen man pages that I reference a > > lot, and a lot of man pages that I never reference. If something > > could keep track of which pages were actually referenced a lot, > > then some system daemon could generate cat-versions of just those > > man pages. > >catpages take less space than manpages, so I don't see a reason to >fine grain this. For tonight's -CURRENT: > >du -skc /usr/share/man/man* | tail -1 >7800 total >du -skc /usr/share/man/cat* | tail -1 >6599 total Note that you are comparing "all cat pages and no man pages" to "all man pages and no cat pages". In that sense, yes, cat pages may take up less space than man pages. The reality is that people will always have all the man pages, so generating all the cat pages is always more disk space than *not* generating all the cat pages. In my case, the issue is really more of build times. I rebuild my systems pretty often, and I would just as soon keep the buildworld times as short as possible. But let me repeat that I do think the change to 'man' is the right idea, due to the security concerns. I do not wish to argue against that change. I'm just musing about alternate ways to get cat pages. >periodic.conf(5)'s weekly_catman_enable=YES seens to be a good >option here, no? Yes, this will be a useful option. I was just trying to come up with some more bells and whistles for it... :-) -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 12:30: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by hub.freebsd.org (Postfix) with ESMTP id 6251E37B404 for ; Thu, 17 Jan 2002 12:29:57 -0800 (PST) Received: from pool0559.cvx40-bradley.dialup.earthlink.net ([216.244.44.49] helo=mindspring.com) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 16RJAP-0000gm-00; Thu, 17 Jan 2002 12:29:45 -0800 Message-ID: <3C473436.BBBABF6B@mindspring.com> Date: Thu, 17 Jan 2002 12:29:42 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Michal Mertl Cc: arch@freebsd.org Subject: Re: 64 bit counters again References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Michal Mertl wrote: > I wan't to inform you that I tried to look at some system pushing data > with different size/implementation network counters. I did my last test on > dual PIII 750. I don't know, of any good way to measure the load, so I > just run vmstat -w1 (and calculated average idle) while pushing the data > and also looked at the throughput at 100Mbit Full-Duplex. System was > performing about 10000 interrupts and 15000 packets per second. I didn't > notice any difference between using 32 bits non atomic operations (3 > clocks per op) or 64 bit atomic (lock;cmpxchg8b - 50 clocks). I did also > measure it on single Duron 800 with the same result. 1) Use a gigabit interface, not an FXP. 2) Use 1K payload data. 3) Tune your system so that it can push data at wire speed on the gigabit; this isn't hard, but it is a test of whether you can recognize overhead when you see it. 4) Measure CPU overhead as well as I/O overhead. 5) Measure total number of mbufs and clusters in use, 6) Use an SMP system, make sure that you have a sender on both CPUs, and measure TLB shootdown and page mapping turnover to ensure you get that overhead in there, too (plus the lock overhead). 7) Make sure you are sending data already in the kernel, so you aren't including copy overhead in the CPU cost, since practically no one implements servers with copy overhead these days. If you push data at 100Mbit, and not even at full throttle at that, you can't reasonably expect to see a slowdown when you have other bottlenecks between you and the changes. In particular, you're not going to see things like the pool size go up because of increased pool retention time, etc., due to the overhead of doing the calculations. Also, realize that even though simply pushing data doesn't use up a lot of CPU on FreeBSD if you do it right, even 2% or 4% increase in CPU overhead overall is enough to cause problems for already CPU-bound applications (i.e. that's ~40 less SSL connections per server). -- Personally, I don't consider 3 and a half million clocks to be peanuts, and if you ran at wire speed on the 100Mbit, I would expect that to become 35 million clocks on a gigabit. But we can wait for your effects on the mbuf count high watermark and CPU utilization values before jumping to any conclusions... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 13: 4:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id A0B9C37B400; Thu, 17 Jan 2002 13:04:40 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0HJQRS00609; Thu, 17 Jan 2002 20:26:27 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Bruce Evans Cc: "Brian F. Feldman" , Max Khon , arch@FreeBSD.ORG Subject: Re: request for review In-Reply-To: Your message of "Wed, 16 Jan 2002 15:58:19 +1100." <20020116155420.N487-100000@gamplex.bde.org> Date: Thu, 17 Jan 2002 20:26:27 +0100 Message-ID: <607.1011295587@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020116155420.N487-100000@gamplex.bde.org>, Bruce Evans writes: >On Tue, 15 Jan 2002, Brian F. Feldman wrote: > >> Bruce Evans wrote: >> > POSIX.1-2001 specifies st_blksize. >> >> Great, right after I get POSIX.1-1996 ;) So how does it specify st_blksize, >> then? > >The same as the quote from www.opengroup.org in the comment in vn_stat(). >It says nothing important that isn't in that quote. Well, the one thing missing in this thread is the concensus on what value to return when no value can be said to make any amount of sense for devices in general and zero is considered illegal. What's the verdict ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 13:40:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id 3AD8837B42B; Thu, 17 Jan 2002 13:40:10 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020117214008.ZLSK10199.rwcrmhc53.attbi.com@InterJet.elischer.org>; Thu, 17 Jan 2002 21:40:08 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA93089; Thu, 17 Jan 2002 13:35:43 -0800 (PST) Date: Thu, 17 Jan 2002 13:35:42 -0800 (PST) From: Julian Elischer To: Poul-Henning Kamp Cc: Bruce Evans , "Brian F. Feldman" , Max Khon , arch@FreeBSD.ORG Subject: Re: request for review In-Reply-To: <607.1011295587@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This number is occasionally used to indicate the minimum size that can be written to the device. (or multiples of that). for a lot of devices that number would be 1, no? On Thu, 17 Jan 2002, Poul-Henning Kamp wrote: > In message <20020116155420.N487-100000@gamplex.bde.org>, Bruce Evans writes: > >On Tue, 15 Jan 2002, Brian F. Feldman wrote: > > > >> Bruce Evans wrote: > >> > POSIX.1-2001 specifies st_blksize. > >> > >> Great, right after I get POSIX.1-1996 ;) So how does it specify st_blksize, > >> then? > > > >The same as the quote from www.opengroup.org in the comment in vn_stat(). > >It says nothing important that isn't in that quote. > > Well, the one thing missing in this thread is the concensus on > what value to return when no value can be said to make any amount > of sense for devices in general and zero is considered illegal. > > What's the verdict ? > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 15:35:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sneakerz.org (sneakerz.org [216.33.66.254]) by hub.freebsd.org (Postfix) with ESMTP id 1443137B416 for ; Thu, 17 Jan 2002 15:35:38 -0800 (PST) Received: by sneakerz.org (Postfix, from userid 1023) id 77CD25D006; Thu, 17 Jan 2002 17:35:32 -0600 (CST) Date: Thu, 17 Jan 2002 17:35:32 -0600 From: mux@sneakerz.org To: freebsd-arch@FreeBSD.org Subject: Proposal for a new mount API Message-ID: <20020117173532.A48367@sneakerz.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, As you already know if you read the FreeBSD projects development status report, I've been working with Poul Henning-Kamp to design a new mount API. The text below explains the problems we are having with the existing API and how we intend to solve them with the new one. The new mount API intends to overcome some limitations of the existing API. The problem is how we pass data to the filesystems when mounting them. Right now we can have only 32 mount options because we use a bitmap to communicate them, along with a void * for some of the special cases. Some things which are really mount options, like the device name, are treated specially, while other mount options like quota information is not dealt with at all. With nmount, we have a generic way to pass an indefinite number of options to the filesystems, which are transferred in a struct uio. These options are uiomove()'d by the vfs_nmount() function, so we are able to pass these options from the kernel as well. We are not limited by the 32bits flags anymore, and it will help fixing problems with the export stuff and should greatly simplify the handling of quotas. From the filesystem point of view, we have two different functions to get the options : vfs_getopt() and vfs_copyopt(). The first one returns a pointer to the value of the option. This is probably only convenient for strings and fixed size types like integers and structs. The second one will copy the value of the option to a specified buffer, which fits more for binary data like the Unicode tables of msdosfs or, as we could imagine, cryptographic keys for some crypto fs. Here are the prototypes of these two functions : int vfs_getopt(struct vfsoptlist *opts, const char *name, void **buf, int *len); int vfs_copyopt(struct vfsoptlist *opts, const char *name, void *dest, int len, int *done); The exact behaviour of these functions is explained in the comments above their definition. The one special case to consider is MNT_UPDATE. The struct mount holds a pointer to another option list, only set in the mount update case. It allows the underlying filesystem to compare between the two lists of options and to behave accordingly. The vfs_nmount() function is not intended to be called directly from the kernel as the vfs_mount() function was. If we need to mount a filesystem inside the kernel, we have two wrappers for this purpose. Their prototypes are : int kernel_mount(struct iovec *iovp, unsigned int iovcnt, int flags); int kernel_vmount(int flags, ...); The iovec's of kernel_mount() are intended to be passed by pairs, the first one pointing to the name of the option and the second one to its value. The old const char *type and const char *path parameters are now passed as options as well. kernel_vmount() is a more convenient form of kernel_mount(), since it takes a NULL terminated list of strings of the form "name=value". Obviously, this can only be used with filesystems that don't require binary data. These two functions then both create the struct uio and call vfs_nmount(). From userland point of view, the prototype of the new mount syscall is identical to kernel_mount()'s one and we use it exactly the same way. int nmount(struct iovec *iovp, unsigned int iovcnt, int flags); Of course, we got rid of the void * parameter. It may be interesting to write a libc wrapper in the future if we want to have a kernel_vmount()-like call in userland. The code which implements this API is available at : http://www.sneakerz.org/~mux/mount.diff For the moment, filesystems that want to use the new mount API have to set their VFS_MOUNT function to NULL in the struct vfsops, and provide their new VFS_NMOUNT function which has been added at the end of struct vfsops. This allows the patch to be applied without hopefully breaking anything :-) Opinions, comments and advices are of course greatly appreciated. Maxime Henrion To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 16: 6:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 47A7F37B404 for ; Thu, 17 Jan 2002 16:06:21 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0I06J4a083862; Fri, 18 Jan 2002 01:06:20 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0I06Js1083859; Fri, 18 Jan 2002 01:06:19 +0100 (CET) Date: Fri, 18 Jan 2002 01:06:19 +0100 (CET) From: Michal Mertl To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Re: 64 bit counters again In-Reply-To: <3C473436.BBBABF6B@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Jan 2002, Terry Lambert wrote: > Michal Mertl wrote: > > I wan't to inform you that I tried to look at some system pushing data > > with different size/implementation network counters. I did my last test on > > dual PIII 750. I don't know, of any good way to measure the load, so I > > just run vmstat -w1 (and calculated average idle) while pushing the data > > and also looked at the throughput at 100Mbit Full-Duplex. System was > > performing about 10000 interrupts and 15000 packets per second. I didn't > > notice any difference between using 32 bits non atomic operations (3 > > clocks per op) or 64 bit atomic (lock;cmpxchg8b - 50 clocks). I did also > > measure it on single Duron 800 with the same result. > > 1) Use a gigabit interface, not an FXP. > Don't have any :-(. It probably doesn't change a situation that I use realtek which changes the counter inside interrupt on the Duron. > 2) Use 1K payload data. > > 3) Tune your system so that it can push data at wire speed > on the gigabit; this isn't hard, but it is a test of > whether you can recognize overhead when you see it. > > 4) Measure CPU overhead as well as I/O overhead. > I don't know what do you mean by I/O overhead here. > 5) Measure total number of mbufs and clusters in use, > > 6) Use an SMP system, make sure that you have a sender > on both CPUs, and measure TLB shootdown and page > mapping turnover to ensure you get that overhead in > there, too (plus the lock overhead). > I'm afraid I don't understand. I don't see that deep into kernel unfortunately. If you tell me what to look at and how... > 7) Make sure you are sending data already in the kernel, > so you aren't including copy overhead in the CPU cost, > since practically no one implements servers with copy > overhead these days. > What do you mean by that? Zero-copy operation? Like sendfile? Is Apache 1.x zero-copy? > If you push data at 100Mbit, and not even at full throttle at > that, you can't reasonably expect to see a slowdown when you > have other bottlenecks between you and the changes. > > In particular, you're not going to see things like the pool > size go up because of increased pool retention time, etc., > due to the overhead of doing the calculations. > That's probably correct eventhough I again don't fully understand what you're talking about :-). > Also, realize that even though simply pushing data doesn't > use up a lot of CPU on FreeBSD if you do it right, even 2% > or 4% increase in CPU overhead overall is enough to cause > problems for already CPU-bound applications (i.e. that's > ~40 less SSL connections per server). > You're right with that too. Of course I know that at full CPU load the clocks will be missing and maybe other things (memory bandwidth with locked operations?) will suffer. > -- > > Personally, I don't consider 3 and a half million clocks to > be peanuts, and if you ran at wire speed on the 100Mbit, I > would expect that to become 35 million clocks on a gigabit. > Yes. It seems so. > But we can wait for your effects on the mbuf count high > watermark and CPU utilization values before jumping to any > conclusions... > I'm afraid I can't provide any measurement with faster interfaces. I can try to use real server to sned me some data so it's executing on both processors, but I would probably become limited with 100Mbit sooner than I'll notice processors have less time to do their job :(. I forgot to repeat that 3 clocks operation (unlocked 32 bit add) is potentially unsafe on SMP. So I was comparing apples to oranges, just to show that even under such unfair conditions it's not such a big problem. THE MOST IMPORTANT QUESTION, to which lots of you probably know answer is, DO WE NEED ATOMIC OPERATIONS FOR ACCESSING DIFFERENT COUNTERS (e.g. network-device (modified in ISR? - YES/NO) or network-protocol or filesystem ...)? NO MATTER WHAT THE SIZE OF THE COUNTER IS. If we need atomic, we need atomic 32 bit as much as 64 bit. If we don't, we can have cheaper 64 bits counters. My API allows for different treatment of different classes of counters (if simple answer to my question exists) or places in kernel (you know you're calling when interrupt can occur, other CPU may modify the same counter...). I run the SMP kernel with the same test with "simple 64 bit add (addl,adcl)" without noticing anything went wrong and that sure isn't anywhere near as expensive as lock;cmpxchg8b. -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 17:17:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from lists.unixathome.org (lists.unixathome.org [210.48.103.158]) by hub.freebsd.org (Postfix) with ESMTP id 16D3E37B405; Thu, 17 Jan 2002 17:17:23 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by lists.unixathome.org (8.11.6/8.11.6) with ESMTP id g0GGvND18457; Thu, 17 Jan 2002 05:57:23 +1300 (NZDT) (envelope-from dan@langille.org) Date: Thu, 17 Jan 2002 05:57:23 +1300 (NZDT) From: Dan Langille X-X-Sender: dan@lists.unixathome.org To: Ruslan Ermilov Cc: Joerg Wunsch , , , Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist In-Reply-To: <20020116183712.G13904@sunbay.com> Message-ID: <20020117053925.A18072-100000@lists.unixathome.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 16 Jan 2002, Ruslan Ermilov wrote: > On Wed, Jan 16, 2002 at 05:11:44PM +0100, Joerg Wunsch wrote: > > Get me right, it's OK by me to put a big warning above that knob > > into /etc/defaults/make.conf. But for sites less concerned about > > security (like my home computer), compromising the "man" account > > by the only other user on my machine (that's my wife :) isn't > > anything i would care much about. Even though the machine is > > pretty fast, i prefer the cached catpages for efficiency. (setuid > > root binaries are a totally different matter, since they might > > always serve as a target by a potential intruder. But i don't > > think such an intruder would gain much by breaking a setuid man > > man(1), and then replace all my catpages. ;-) > > > OK, what do others think about this? We've had users coming into IRC channels asking about the man page changes. I don't fully understand the implications yet. From the comments above, it sounds like each time I issue 'man ls', it's going to decompress the man page unless I do some tweaking. Is that correct? If so, that's not good for me, and I suspect for a large number of other people. Else, please ignore the next paragraph. Please don't force the majority of people to tweak their systems in order to avoid a theoretical problem for a small minority. But if you insist upon doing so, then I feel you must provide a knob in /etc/defaults/make.conf. I have read the commit message, but I don't have the knowledge to understand the implications. Perhaps someone could explain it so I and others can understand what this change really means. thanks. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 17:42:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailsrv.otenet.gr (mailsrv.otenet.gr [195.170.0.5]) by hub.freebsd.org (Postfix) with ESMTP id 7922E37B42A; Thu, 17 Jan 2002 17:42:21 -0800 (PST) Received: from hades.hell.gr (patr530-a038.otenet.gr [212.205.215.38]) by mailsrv.otenet.gr (8.11.5/8.11.5) with ESMTP id g0I1gCw18496; Fri, 18 Jan 2002 03:42:13 +0200 (EET) Received: (from charon@localhost) by hades.hell.gr (8.11.6/8.11.6) id g0HLUiD41375; Thu, 17 Jan 2002 23:30:44 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Date: Thu, 17 Jan 2002 23:30:43 +0200 From: Giorgos Keramidas To: Ruslan Ermilov Cc: Ted Mittelstaedt , arch@freebsd.org, jasone@canonware.com Subject: Re: termcap versus terminfo Message-ID: <20020117213043.GB41262@hades.hell.gr> References: <001501c19f3b$94c35280$1401a8c0@tedm.placo.com> <20020117120319.L27310@sunbay.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020117120319.L27310@sunbay.com> User-Agent: Mutt/1.3.25i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-01-17 12:03:19, Ruslan Ermilov wrote: > On Thu, Jan 17, 2002 at 01:44:39AM -0800, Ted Mittelstaedt wrote: > > > > I don't see as how any admin is going to figure out how to add a terminal > > description other than trial and error so what "user friendliness" gained by > > holding to the human-readable /etc/termcap format is lost in the current > > scheme and really shouldn't be an issue to use in deciding between > > termcap and terminfo. > > > What's wrong with reading termcap(5)? :-) Uhm, nothing. And FILES already mentions cap_mkdb(1). This means my earlier message about describing how changes can be made to termcap, is probably meaningless. Ah well :/ The manpage is great, now that I've read all of it. If the need arises and people do indeed ask about making their own terminal entries, I'll go write a FAQ entry about the source of it all in src/share/termcap/termcap.src -- Giorgos Keramidas . . . . . . . . . keramida@{ceid.upatras.gr,freebsd.org} FreeBSD Documentation Project . . . http://www.freebsd.org/docproj/ FreeBSD: The power to serve . . . . http://www.freebsd.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 17:42:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailsrv.otenet.gr (mailsrv.otenet.gr [195.170.0.5]) by hub.freebsd.org (Postfix) with ESMTP id A166C37B405 for ; Thu, 17 Jan 2002 17:42:23 -0800 (PST) Received: from hades.hell.gr (patr530-a038.otenet.gr [212.205.215.38]) by mailsrv.otenet.gr (8.11.5/8.11.5) with ESMTP id g0I1gHw18583; Fri, 18 Jan 2002 03:42:17 +0200 (EET) Received: (from charon@localhost) by hades.hell.gr (8.11.6/8.11.6) id g0HLNSh41312; Thu, 17 Jan 2002 23:23:28 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Date: Thu, 17 Jan 2002 23:23:28 +0200 From: Giorgos Keramidas To: Ted Mittelstaedt Cc: arch@freebsd.org, jasone@canonware.com Subject: Re: termcap versus terminfo Message-ID: <20020117212327.GA41262@hades.hell.gr> References: <001501c19f3b$94c35280$1401a8c0@tedm.placo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <001501c19f3b$94c35280$1401a8c0@tedm.placo.com> User-Agent: Mutt/1.3.25i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-01-17 01:44:39, Ted Mittelstaedt wrote: > > The current FreeBSD scheme with the compiled termcap.db > has terrible documentation. In fact the only mention of the > need to use cap_mkdb to build termcap.db is in the cap_mkdb > man page, and it's not even a mention, it's just a link in > SEE ALSO. It's not mentioned in the man page for termcap. This is a bug in the documentation. It's not a termcap problem per se, but the termcap manpage needs to be changed to include a section about 'making local changes to the system termcap'. If nobody has a better suggestion, I'll try to come up with a small addition to termcap's manpage in a few days. > I don't see as how any admin is going to figure out how to add a terminal > description other than trial and error so what "user friendliness" gained by > holding to the human-readable /etc/termcap format is lost in the current > scheme and really shouldn't be an issue to use in deciding between > termcap and terminfo. > > With regards to Terry's comments about corporate entities wanting > to reduce support overhead, I think this is a bit pedantic. Sun > USED to keep infocmp separate in the Toolbox, but in Solaris 8, > infocmp is included and can be used to extract terminfo source, > and tic which is also included can be used to compile it back in. > How many other commercial UNIX vendors are still unbundling infocmp > I would ask - my guess is very few. I don't see that the infocmp-edit-tic > way of modifing the termcap entry is superior or inferior to the > edit-termcap-and-run-cap_mkdb, from an admin's point of view. > > The principle need admins have to tamper with terminfo or termcap entries is > to work around deviations from whatever standard emulation their > display devices are supposed to be following. In the old days when > terminal manufacturers would fix bugs and release new ROMS, the cost > to an enterprise of running around and replacing them in their terminals > pretty much guarenteed that once a terminal was installed, it would > never be touched. This meshed well with a central termcap authority > controlling termcap entries in all UNIX everywhere because things > didn't change much in the displays people were using, so it made > sense to leverage effort to figure out the changes in new ROMS. > > Today though, most people use terminal emulation on PC's or hardware > devices with upgradable hardware, so there's much wider deviation from > terminal emulation "standards" like the vt102, ANSI, Wyse60 and so on. > And the deviations occur not just from emulation program to emulation > program but from version to version. So, having a single unified set of > terminal entries that's kept maintained centrally isn't as important as it > once was, because everyone has to change everything for their own stuff. > What's more important is the default termcap supplied with the > system have a set of common emulations, that people can use as starting > points for their local mods. > > > Ted Mittelstaedt > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- Giorgos Keramidas . . . . . . . . . keramida@{ceid.upatras.gr,freebsd.org} FreeBSD Documentation Project . . . http://www.freebsd.org/docproj/ FreeBSD: The power to serve . . . . http://www.freebsd.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 18:14:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 3EE0B37B400; Thu, 17 Jan 2002 18:14:47 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id NAA16012; Fri, 18 Jan 2002 13:14:38 +1100 Date: Fri, 18 Jan 2002 13:15:55 +1100 (EST) From: Bruce Evans X-X-Sender: To: Poul-Henning Kamp Cc: "Brian F. Feldman" , Max Khon , Subject: Re: request for review In-Reply-To: <607.1011295587@critter.freebsd.dk> Message-ID: <20020118131144.T4377-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Jan 2002, Poul-Henning Kamp wrote: > Well, the one thing missing in this thread is the concensus on > what value to return when no value can be said to make any amount > of sense for devices in general and zero is considered illegal. I like PAGE_SIZE for a general default, but I think most (classes of) devices should set their own size. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 17 21:31:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.freebsd-corp-net-guide.com (mail.freebsd-corp-net-guide.com [206.29.169.15]) by hub.freebsd.org (Postfix) with ESMTP id 2AA1837B404; Thu, 17 Jan 2002 21:31:06 -0800 (PST) Received: from tedm.placo.com (nat-rtr.freebsd-corp-net-guide.com [206.29.168.154]) by mail.freebsd-corp-net-guide.com (8.11.1/8.11.1) with SMTP id g0I5UmR80661; Thu, 17 Jan 2002 21:30:49 -0800 (PST) (envelope-from tedm@toybox.placo.com) From: "Ted Mittelstaedt" To: "Giorgos Keramidas" , "Ruslan Ermilov" Cc: , Subject: RE: termcap versus terminfo Date: Thu, 17 Jan 2002 21:30:48 -0800 Message-ID: <000701c19fe1$492e17a0$1401a8c0@tedm.placo.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 In-Reply-To: <20020117213043.GB41262@hades.hell.gr> X-Mimeole: Produced By Microsoft MimeOLE V4.72.3155.0 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >-----Original Message----- >From: Giorgos Keramidas [mailto:keramida@ceid.upatras.gr] >Sent: Thursday, January 17, 2002 1:31 PM >To: Ruslan Ermilov >Cc: Ted Mittelstaedt; arch@freebsd.org; jasone@canonware.com >Subject: Re: termcap versus terminfo > > >Uhm, nothing. And FILES already mentions cap_mkdb(1). >This means my earlier message about describing how changes can be made >to termcap, is probably meaningless. Ah well :/ > No, rather it highlights the problem in the documentation. You also missed the FILES reference in termcap(5) as well until it was mentioned. If the documentation is so sparse that people have to get on mailing lists and waste bandwidth asking about something then the documentation is inadequate. And, a single reference still doesen't explain the procedure. Documentation is like bandwidth, people may bitch about it but nobody willingly gives up any of what they already have. Anyway, in this case further modifications of the documentation just makes the hacked up ncurses in FreeBSD deviate even more from the ncurses distribution, creating an even further support burden. >If the need arises and people do indeed ask about making their own >terminal entries, I'll go write a FAQ entry about the source of it all >in src/share/termcap/termcap.src > This is also a good idea, we are talking maybe 11 lines of text. How's the following sound: (in replacement of line # 39 about mailing changes to berkeley, which isn't maintaining termcap anymore anyway) -------------cut here------------ # # To add a termcap entry under FreeBSD for a new terminal type, insert # the entry in the appropriate location in /etc/termcap then issue this # command: # # cap_mkdb -f /usr/share/misc/termcap /etc/termcap # # Terminfo source entries can be converted to termcap entries with the # tic program that is part of the ncurses distribution, see the ports # section. # ------------cut here------------- Ted To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 0:22:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mta5.snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by hub.freebsd.org (Postfix) with ESMTP id C6CE637B421 for ; Fri, 18 Jan 2002 00:22:22 -0800 (PST) Received: from elischer.org ([64.164.9.2]) by mta5.snfc21.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GQ40010VKL94T@mta5.snfc21.pbi.net> for freebsd-arch@FreeBSD.org; Fri, 18 Jan 2002 00:22:21 -0800 (PST) Date: Fri, 18 Jan 2002 00:22:21 -0800 From: Julian Elischer Subject: Re: Proposal for a new mount API To: mux@sneakerz.org Cc: freebsd-arch@FreeBSD.org Message-id: <3C47DB3D.22DD975A@elischer.org> MIME-version: 1.0 X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) Content-type: text/plain; charset=iso-8859-2 Content-transfer-encoding: 7BIT X-Accept-Language: en, hu References: <20020117173532.A48367@sneakerz.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG mux@sneakerz.org wrote: [interesting stuff] > I haven't looked at the diff itself but you seem to be heading in the right direction from what you say.. I just wanted to let you know that we are out here reading it and that you shouldn't get discouraged.. I'll try look at the diffs themselves later as well, though any spare moments I get are being spent on the KSE stuff.  -- +------------------------------------+ ______ _ __ | __--_|\ Julian Elischer | \ U \/ / hard at work in | / \ julian@elischer.org +------>x USA \ a very strange | ( OZ ) \___ ___ | country ! +- X_.---._/ presently in San Francisco \_/ \\ v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 0:50:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 4833737B419 for ; Fri, 18 Jan 2002 00:50:07 -0800 (PST) Received: from pool0026.cvx22-bradley.dialup.earthlink.net ([209.179.198.26] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16RUik-00073r-00; Fri, 18 Jan 2002 00:49:58 -0800 Message-ID: <3C47E1B2.6938136@mindspring.com> Date: Fri, 18 Jan 2002 00:49:55 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Michal Mertl Cc: arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Michal Mertl wrote: > > 4) Measure CPU overhead as well as I/O overhead. > > I don't know what do you mean by I/O overhead here. Say you could flood a gigabit interface, and it was 6% of the CPU on average. Now after you patches, suppose that it's 10% of the CPU. The limiting factor is the interface... but that's only for your application, which is not doing CPU intensive processing. Something that did a lot of CPU work (like SSL), would have a different profile, and you would be limiting the application by causing it to become CPU bound earlier. > > 6) Use an SMP system, make sure that you have a sender > > on both CPUs, and measure TLB shootdown and page > > mapping turnover to ensure you get that overhead in > > there, too (plus the lock overhead). > > I'm afraid I don't understand. I don't see that deep into kernel > unfortunately. If you tell me what to look at and how... The additional locks required for i386 64 bit atomicity will, if the counter is accessed by more than one CPU, result in bus contention for inter-CPU coherency. > > 7) Make sure you are sending data already in the kernel, > > so you aren't including copy overhead in the CPU cost, > > since practically no one implements servers with copy > > overhead these days. > > What do you mean by that? Zero-copy operation? Like sendfile? Is Apache > 1.x zero-copy? Yes, zero copy. Sendfile isn't ideal, but works. Apache is not zero copy. The idea is to not include a lot of CPU work on copies between the user space and the kernel, which aren't going to happen in an extremely optimized application. > > If you push data at 100Mbit, and not even at full throttle at > > that, you can't reasonably expect to see a slowdown when you > > have other bottlenecks between you and the changes. > > > > In particular, you're not going to see things like the pool > > size go up because of increased pool retention time, etc., > > due to the overhead of doing the calculations. > > That's probably correct eventhough I again don't fully understand what > you're talking about :-). Look at the max number of mbufs allocated. They form a pool of type stable memory from which mbufs are allocated (things that get allocated get freed to the pool instead of freed to the system). You can see this in the zone counts by dumping the zone information with vmstat, and in the mbuf counts in the netstat -m case. Basically, if you run without the 64 bit stuff, and get one number, and then run with it, and get a larger number, then this means that the time you are spending doing the stats is increasing the amount of time it takes in the code path, and so the mbufs don't get processed out as quickly. The implication, IFF this is the case, is that the additional processing overhead has increased the amount of time a buffer remains in transit -- the pool retention time -- and thus it increases the overall total pool size for a given throughput. The upshot of this happening is that you now require more memory for the same amount of work, or, if your machine is "maxed out", then the high end amount of work you can do is reduced by the changes. > > Also, realize that even though simply pushing data doesn't > > use up a lot of CPU on FreeBSD if you do it right, even 2% > > or 4% increase in CPU overhead overall is enough to cause > > problems for already CPU-bound applications (i.e. that's > > ~40 less SSL connections per server). > > You're right with that too. Of course I know that at full CPU load the > clocks will be missing and maybe other things (memory bandwidth with > locked operations?) will suffer. Yes. It's important to know whether it is significant for the bottleneck figure of merit for a particular application. For SSL, this is CPU cycles. For an NFS server, this is how much data it can push in a given period of time (overall throughput). For some other application, it's some other number. For example, the thing that limits the top end speed of SQUID is how fast it can log, and the number one factor there is actually the rate at which gettimeofday() can be called, and still maintain the exhaustive log records that users have come to expect (these are basically UI "eye candy", except when the logs are digested and used for billing purposes, at which point they are really absolutely critical). Because network processing for almost all packets in or out in the current FreeBSD occurs at NETISR, this basically means that the closer it takes to a quantum, the closer you are to a condition called "receiver livelock". This actually drops your top end by up to 15%, and can actually stop your server in its tracks if you aren't very careful (RED queueing, weighted fair share queue scheduling, to ensure you don't spend all your time in the kernel, and none in user space processing request, etc.). > > But we can wait for your effects on the mbuf count high > > watermark and CPU utilization values before jumping to any > > conclusions... > > I'm afraid I can't provide any measurement with faster interfaces. I can > try to use real server to sned me some data so it's executing on both > processors, but I would probably become limited with 100Mbit sooner than > I'll notice processors have less time to do their job :(. Well, you probably should collect *all* statistics you can, in the most "this is the only thing I'm doing with the box" way you can, before and after the code change, and then plot the ones that get worse (or better) as a result of the change. [ ... ] > THE MOST IMPORTANT QUESTION, to which lots of you probably know answer > is, DO WE NEED ATOMIC OPERATIONS FOR ACCESSING DIFFERENT COUNTERS (e.g. > network-device (modified in ISR? - YES/NO) or network-protocol or > filesystem ...)? NO MATTER WHAT THE SIZE OF THE COUNTER IS. > > If we need atomic, we need atomic 32 bit as much as 64 bit. If we don't, > we can have cheaper 64 bits counters. My API allows for different > treatment of different classes of counters (if simple answer to my > question exists) or places in kernel (you know you're calling when > interrupt can occur, other CPU may modify the same counter...). I run the > SMP kernel with the same test with "simple 64 bit add (addl,adcl)" without > noticing anything went wrong and that sure isn't anywhere near as > expensive as lock;cmpxchg8b. I think the answer is "yes, we need atomic counters". Whether they need to be 64 bit or just 32 bit is really application dependent (we have all agreed to that, I think). See Bruce's posting about atomicity; I think it speaks very eleoquently on the issue (much more brief than what I'd write to say the same thing ;^)). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 1: 8: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by hub.freebsd.org (Postfix) with ESMTP id 2DFAF37B402; Fri, 18 Jan 2002 01:08:00 -0800 (PST) Received: from dialup-209.244.107.191.dial1.sanjose1.level3.net ([209.244.107.191] helo=blossom.cjclark.org) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 16RV02-0003Pm-00; Fri, 18 Jan 2002 01:07:55 -0800 Received: (from cjc@localhost) by blossom.cjclark.org (8.11.6/8.11.3) id g0I971642682; Fri, 18 Jan 2002 01:07:01 -0800 (PST) (envelope-from cjc) Date: Fri, 18 Jan 2002 01:07:01 -0800 From: "Crist J . Clark" To: Garance A Drosihn Cc: Ruslan Ermilov , Robert Watson , Joerg Wunsch , arch@FreeBSD.ORG Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020118010701.F40472@blossom.cjclark.org> References: <20020117100308.B27310@sunbay.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from drosih@rpi.edu on Thu, Jan 17, 2002 at 01:09:18PM -0500 X-URL: http://people.freebsd.org/~cjc/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Jan 17, 2002 at 01:09:18PM -0500, Garance A Drosihn wrote: > At 10:03 AM +0200 1/17/02, Ruslan Ermilov wrote: > >On Wed, Jan 16, 2002 at 04:09:37PM -0500, Garance A Drosihn wrote: > > > In my case, I have about a dozen man pages that I reference a > > > lot, and a lot of man pages that I never reference. If something > > > could keep track of which pages were actually referenced a lot, > > > then some system daemon could generate cat-versions of just those > > > man pages. > > > >catpages take less space than manpages, so I don't see a reason to > >fine grain this. For tonight's -CURRENT: > > > >du -skc /usr/share/man/man* | tail -1 > >7800 total > >du -skc /usr/share/man/cat* | tail -1 > >6599 total > > Note that you are comparing "all cat pages and no man pages" to > "all man pages and no cat pages". In that sense, yes, cat pages may > take up less space than man pages. The reality is that people will > always have all the man pages, Why's that? "Other OSes" do just distribute catpages. Most people don't need the markup pages. If you only ever use man(1), you don't really need markup pages. (I'm not advocating installing catpages without manpages as the default.) > so generating all the cat pages is > always more disk space than *not* generating all the cat pages. > > In my case, the issue is really more of build times. I rebuild my > systems pretty often, and I would just as soon keep the buildworld > times as short as possible. Making buildworld "fast" is not really a major design concern. -- Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 1:40:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by hub.freebsd.org (Postfix) with ESMTP id 7C91737B402; Fri, 18 Jan 2002 01:39:56 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.9.3/8.9.3) with UUCP id KAA07220; Fri, 18 Jan 2002 10:39:49 +0100 (CET) Received: (from j@localhost) by uriah.heep.sax.de (8.11.6/8.11.6) id g0I9XYe55087; Fri, 18 Jan 2002 10:33:34 +0100 (MET) (envelope-from j) Date: Fri, 18 Jan 2002 10:33:34 +0100 From: Joerg Wunsch To: "Crist J . Clark" Cc: Garance A Drosihn , arch@FreeBSD.ORG Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020118103334.A54530@uriah.heep.sax.de> Reply-To: Joerg Wunsch References: <20020117100308.B27310@sunbay.com> <20020118010701.F40472@blossom.cjclark.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020118010701.F40472@blossom.cjclark.org>; from cjc@FreeBSD.ORG on Fri, Jan 18, 2002 at 01:07:01AM -0800 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG As Crist J . Clark wrote: > Why's that? "Other OSes" do just distribute catpages. That used to be the case at the beginning of the 90's of last century. AT&T attempted to sell a polished-up troff as a `documenter's workbench', so they had to rip it off the base system. groff has more than filled the gap since. These days, Systems like Solaris and HP-UX ship with source man pages as well, where in Solaris, they recently switched to SGML-based man pages by default (but can handle custom-installed troff pages still). > (I'm not advocating installing catpages without manpages as the > default.) As a non-default, you've been getting this option with FreeBSD for many years already: just select the catpages distribution only when installing. (I originally created the catpages distribution when the junk computer at home i was using for testing was still a 386/sx16. It simply took too long on that machine to format man pages, so i thought the idea of a catpages distribution would be something neat for people with slow machines. ;-) -- cheers, J"org .-.-. --... ...-- -.. . DL8DTL http://www.sax.de/~joerg/ NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 6:46: 2 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 40D9B37B421 for ; Fri, 18 Jan 2002 06:44:40 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0IEhm4a069689; Fri, 18 Jan 2002 15:43:52 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0IEhl8B069683; Fri, 18 Jan 2002 15:43:48 +0100 (CET) Date: Fri, 18 Jan 2002 15:43:47 +0100 (CET) From: Michal Mertl To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Re: 64 bit counters again In-Reply-To: <3C47E1B2.6938136@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 18 Jan 2002, Terry Lambert wrote: > Michal Mertl wrote: > > > 4) Measure CPU overhead as well as I/O overhead. > > > > I don't know what do you mean by I/O overhead here. > > Say you could flood a gigabit interface, and it was 6% of the > CPU on average. Now after you patches, suppose that it's 10% > of the CPU. The limiting factor is the interface... but that's > only for your application, which is not doing CPU intensive > processing. Something that did a lot of CPU work (like SSL), > would have a different profile, and you would be limiting the > application by causing it to become CPU bound earlier. > That's explaining only CPU overhead which I knew there is some. > > > 6) Use an SMP system, make sure that you have a sender > > > on both CPUs, and measure TLB shootdown and page > > > mapping turnover to ensure you get that overhead in > > > there, too (plus the lock overhead). > > > > I'm afraid I don't understand. I don't see that deep into kernel > > unfortunately. If you tell me what to look at and how... > > The additional locks required for i386 64 bit atomicity will, > if the counter is accessed by more than one CPU, result in > bus contention for inter-CPU coherency. > What additional locks? The lock prefix for cmpxchg8b? It's required for 32 bit too and it increases time spent on operation from 3 to 21 clocks making the difference between 32 and 64 bit "only" 29 clocks instead on 47. > > > 7) Make sure you are sending data already in the kernel, > > > so you aren't including copy overhead in the CPU cost, > > > since practically no one implements servers with copy > > > overhead these days. > > > > What do you mean by that? Zero-copy operation? Like sendfile? Is Apache > > 1.x zero-copy? > > Yes, zero copy. Sendfile isn't ideal, but works. Apache is > not zero copy. The idea is to not include a lot of CPU work > on copies between the user space and the kernel, which aren't > going to happen in an extremely optimized application. > An "extremely optimized" application is a thing which would have an administrator who doesn't enable costly counters. > > > If you push data at 100Mbit, and not even at full throttle at > > > that, you can't reasonably expect to see a slowdown when you > > > have other bottlenecks between you and the changes. > > > > > > In particular, you're not going to see things like the pool > > > size go up because of increased pool retention time, etc., > > > due to the overhead of doing the calculations. > > > > That's probably correct eventhough I again don't fully understand what > > you're talking about :-). > > Look at the max number of mbufs allocated. They form a pool > of type stable memory from which mbufs are allocated (things Thanks. > > > Also, realize that even though simply pushing data doesn't > > > use up a lot of CPU on FreeBSD if you do it right, even 2% > > > or 4% increase in CPU overhead overall is enough to cause > > > problems for already CPU-bound applications (i.e. that's > > > ~40 less SSL connections per server). > > > > You're right with that too. Of course I know that at full CPU load the > > clocks will be missing and maybe other things (memory bandwidth with > > locked operations?) will suffer. > > Yes. It's important to know whether it is significant for > the bottleneck figure of merit for a particular application. > > For SSL, this is CPU cycles. For an NFS server, this is how > much data it can push in a given period of time (overall > throughput). For some other application, it's some other > number. Agreed. > > > But we can wait for your effects on the mbuf count high > > > watermark and CPU utilization values before jumping to any > > > conclusions... > > > > I'm afraid I can't provide any measurement with faster interfaces. I can > > try to use real server to sned me some data so it's executing on both > > processors, but I would probably become limited with 100Mbit sooner than > > I'll notice processors have less time to do their job :(. > > Well, you probably should collect *all* statistics you can, > in the most "this is the only thing I'm doing with the box" > way you can, before and after the code change, and then plot > the ones that get worse (or better) as a result of the change. Will do eventually, but unfotunately don't have the time to devote to it at the moment. > > THE MOST IMPORTANT QUESTION, to which lots of you probably know answer > > is, DO WE NEED ATOMIC OPERATIONS FOR ACCESSING DIFFERENT COUNTERS (e.g. > > network-device (modified in ISR? - YES/NO) or network-protocol or > > filesystem ...)? NO MATTER WHAT THE SIZE OF THE COUNTER IS. > > > I think the answer is "yes, we need atomic counters". Whether they > need to be 64 bit or just 32 bit is really application dependent > (we have all agreed to that, I think). Thanks. Do you think it's always true (STABLE/CURRENT,network device ISRs, /sys/netinet routines) ? > See Bruce's posting about atomicity; I think it speaks very > eleoquently on the issue (much more brief than what I'd write > to say the same thing ;^)). If you mean the email where he talks about atomic_t ("atomic_t would be "int" if anything") it doesn't fully apply. I am not inventing atomic_t anymore anyway :-). Isn't there a platform, which better works with 64 bit ints than with 32 bits (a-la 32/16 bits on modern i386)? -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 7:42:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from aurora.nsu.ru (aurora.nsu.ru [193.124.215.195]) by hub.freebsd.org (Postfix) with ESMTP id 43A1A37B404 for ; Fri, 18 Jan 2002 07:42:11 -0800 (PST) Received: from inet.ssc.nsu.ru ([62.76.110.12]) by aurora.nsu.ru with esmtp (Exim 3.22 #1) id 16Rb9X-0004RW-00 for arch@freebsd.org; Fri, 18 Jan 2002 21:42:03 +0600 Received: from localhost (danfe@localhost) by inet.ssc.nsu.ru (8.9.3/8.9.3) with ESMTP id VAA13879 for ; Fri, 18 Jan 2002 21:41:29 +0600 Date: Fri, 18 Jan 2002 21:41:29 +0600 (NOVT) From: Alexey Dokuchaev To: arch@freebsd.org Subject: Re: request for review Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello! The following program was compiled and run on different systems in order to examine their behaviour with respect to st_blksize returned for tty: #include #include void f(int fd) { struct stat s; fstat(fd, &s); printf("fd %d: st_blksize: %d, st_mode: (0%08o)\n", fd, s.st_blksize, (unsigned) s.st_mode); } main() { f(0); f(1); return 0; } The results obtained are: OpenBSD 2.9 (i386): fd 0: st_blksize: 65536, st_mode: (000020620) fd 1: st_blksize: 65536, st_mode: (000020620) --- SunOS 5.6 (sun4u sparc SUNW,Ultra-Enterprise): fd 0: st_blksize: 8192, st_mode: (000020620) fd 1: st_blksize: 8192, st_mode: (000020620) --- Linux 2.2.19 and 2.4.16 (i686): fd 0: st_blksize: 1024, st_mode: (000020620) fd 1: st_blksize: 1024, st_mode: (000020620) --- FreeBSD 4.4-STABLE (December 6, 2001, August ?, 2001): fd 0: st_blksize: 0, st_mode: (000020620) fd 1: st_blksize: 0, st_mode: (000020620) Note that I stated the values for two -STABLE systems of mine, the reason for doing so arised from the same tests I conducted on both systems *after* I've applied the patch suggested by Max: FreeBSD 4.4-STABLE (December 6, 2001), with patched vfs_vnops.c: fd 0: st_blksize: 16384, st_mode: (000020620) fd 1: st_blksize: 16384, st_mode: (000020620) --- FreeBSD 4.4-STABLE (August ?, 2001), with patched vfs_vnops.c fd 0: st_blksize: 8192, st_mode: (000020620) fd 1: st_blksize: 8192, st_mode: (000020620) Note the different values returned for st_blksize in both cases. > Date: Tue, 15 Jan 2002 06:34:44 -0800 > From: Cy Schubert - ITSD Open Systems Group > To: Poul-Henning Kamp > Cc: Max Khon , arch@freebsd.org > Subject: Re: request for review > > In addition to making FreeBSD consistent with OpenBSD and Linux, as > stated by the originator of this thread, the patch also makes FreeBSD > consistent with Solaris and Tru64-UNIX. I'm for it. That's exactly what bugs me! According to my tests, things are not quite consistent even within *BSD. Right now I do not have time to dig into the source code to get an explanation of why OpenBSD returned 64K, -CURRENT (according to Max) returned 512 bytes (!), and two of my -STABLE systems yielded different numbers as well. Maybe someone has the explanation off-hand, in this case, I'd appreciate the one sharing it with us :-) > > >? current-diffs > > >Index: vfs_vnops.c > > >=================================================================== > > >RCS file: /home/ncvs/src/sys/kern/vfs_vnops.c,v > > >retrieving revision 1.125 > > >diff -u -p -r1.125 vfs_vnops.c > > >--- vfs_vnops.c 18 Dec 2001 20:48:54 -0000 1.125 > > >+++ vfs_vnops.c 14 Jan 2002 18:04:45 -0000 > > >@@ -579,6 +579,8 @@ vn_stat(vp, sb, td) > > > sb->st_blksize = vp->v_rdev->si_bsize_phys; > > > if (sb->st_blksize < BLKDEV_IOSIZE) > > > sb->st_blksize = BLKDEV_IOSIZE; > > >+ } else if (vap->va_type == VCHR) { > > >+ sb->st_blksize = vap->va_blocksize; > > > } else { > > > sb->st_blksize = 0; > > > } And the last thing, I've been looking at the patch, and thought, maybe it would be more clear to patch vfs_vnops.c the following way instead: --- /sys/kern/vfs_vnops.c.orig Fri Jan 18 21:16:33 2002 +++ /sys/kern/vfs_vnops.c Thu Jan 17 20:26:11 2002 @@ -530,7 +530,7 @@ * Default to zero to catch bogus uses of this field. */ - if (vap->va_type == VREG) { + if (vap->va_type == VREG || vap->va_type == VCHR) { sb->st_blksize = vap->va_blocksize; } else if (vn_isdisk(vp, NULL)) { sb->st_blksize = vp->v_rdev->si_bsize_best; What do you think? Sincerely, DAN Fe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 8:12:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-179-27.gte.com (h132-197-179-27.gte.com [132.197.179.27]) by hub.freebsd.org (Postfix) with ESMTP id 09CC737B400 for ; Fri, 18 Jan 2002 08:12:48 -0800 (PST) Received: from kanpc.gte.com (localhost [127.0.0.1]) by h132-197-179-27.gte.com (8.11.6/8.11.4) with SMTP id g0IGCkZ94386; Fri, 18 Jan 2002 11:12:46 -0500 (EST) (envelope-from ak03@gte.com) Date: Fri, 18 Jan 2002 11:12:46 -0500 From: Alexander Kabaev To: mux@sneakerz.org Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Proposal for a new mount API Message-Id: <20020118111246.55725cdd.ak03@gte.com> In-Reply-To: <20020117173532.A48367@sneakerz.org> References: <20020117173532.A48367@sneakerz.org> Organization: Verizon Data Services X-Mailer: Sylpheed version 0.7.0claws8 (GTK+ 1.2.10; i386--freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG int nmount(struct iovec *iovp, unsigned int iovcnt, int flags); According to the folowing prototype, you are planning to pass standard mount options through flags. This will limit you to max 32 standard mount options on 32-bit platforms. What people think about getting rid of the flags parameter and passing standard mount options in the same way as other parameters using their string representations such as ro/rw? We could implement analog of getmntopts in kernel space to reconstruct binary MNT_XXX flags for internal use and in to save some space in memory. As far as I understand, you allow parameters to be passed as binary blobs. I think adding verbose output to the mount(8) command to list all the options active on given mount point is a desirable feature and the new API should make provisions for that. Hopefully, mount(8) should use single generic API to fetch all the options currently in effect from the mounted FS, no matter what the FS type is. Binary blobs, which only particular FS is able to understand, are not making adding this feature any easier. May be we should allow FSes to register description of the parameters and types they understand with the new mount subsystem and change proposed API to pass name, type and value instead of just name/value pair for each parameter to the syscall. This way, filesystems writes will be able to use function calls like int vfs_getopt_int(opts, "mask", &mask) to fetch the integer parameter. New mount code wil match the requested type with the type in syscall parameter list and do necessary conversion, if possible. The existing netgraph code could be used for a conversion between binary types and strings. This will also allow programs such as amd to use generic code to mount any filesystem type. It will just pass all parameters defined in the map entry as strings and the new mount syscall will present these parameters to FS in the form it expects. Just my $0.2 -- Alexander Kabaev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 8:16:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 2843137B490 for ; Fri, 18 Jan 2002 08:16:21 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0IGDcA11627; Fri, 18 Jan 2002 17:13:39 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alexander Kabaev Cc: mux@sneakerz.org, freebsd-arch@FreeBSD.ORG Subject: Re: Proposal for a new mount API In-Reply-To: Your message of "Fri, 18 Jan 2002 11:12:46 EST." <20020118111246.55725cdd.ak03@gte.com> Date: Fri, 18 Jan 2002 17:13:38 +0100 Message-ID: <11625.1011370418@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020118111246.55725cdd.ak03@gte.com>, Alexander Kabaev writes: > int nmount(struct iovec *iovp, unsigned int iovcnt, int flags); > > According to the folowing prototype, you are planning to pass standard >mount options through flags. No, all mount options are passed in elements of the iovec. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 8:22:37 2002 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-179-27.gte.com (h132-197-179-27.gte.com [132.197.179.27]) by hub.freebsd.org (Postfix) with ESMTP id 3220C37B41D for ; Fri, 18 Jan 2002 08:22:32 -0800 (PST) Received: from kanpc.gte.com (localhost [127.0.0.1]) by h132-197-179-27.gte.com (8.11.6/8.11.4) with SMTP id g0IGMOZ94456; Fri, 18 Jan 2002 11:22:24 -0500 (EST) (envelope-from ak03@gte.com) Date: Fri, 18 Jan 2002 11:22:24 -0500 From: Alexander Kabaev To: Poul-Henning Kamp Cc: mux@sneakerz.org, freebsd-arch@FreeBSD.ORG Subject: Re: Proposal for a new mount API Message-Id: <20020118112224.236b4754.ak03@gte.com> In-Reply-To: <11625.1011370418@critter.freebsd.dk> References: <20020118111246.55725cdd.ak03@gte.com> <11625.1011370418@critter.freebsd.dk> Organization: Verizon Data Services X-Mailer: Sylpheed version 0.7.0claws8 (GTK+ 1.2.10; i386--freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > No, all mount options are passed in elements of the iovec. Then what is flags doing in the nmount prototype? And the patch posted certainly does not work like you described. What about parameter conversion to/from strings? I think this could be useful for the reasons I tried to describe in previous message. -- Alexander Kabaev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 8:28:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id BA4B837B416 for ; Fri, 18 Jan 2002 08:28:12 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0IGPbA12161; Fri, 18 Jan 2002 17:25:37 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alexander Kabaev Cc: mux@sneakerz.org, freebsd-arch@FreeBSD.ORG Subject: Re: Proposal for a new mount API In-Reply-To: Your message of "Fri, 18 Jan 2002 11:22:24 EST." <20020118112224.236b4754.ak03@gte.com> Date: Fri, 18 Jan 2002 17:25:37 +0100 Message-ID: <12159.1011371137@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020118112224.236b4754.ak03@gte.com>, Alexander Kabaev writes: >> No, all mount options are passed in elements of the iovec. >Then what is flags doing in the nmount prototype? And the patch posted >certainly does not work like you described. I belive what you see is the backwards compatibility stuff. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 8:41:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-179-27.gte.com (h132-197-179-27.gte.com [132.197.179.27]) by hub.freebsd.org (Postfix) with ESMTP id F20BA37B400 for ; Fri, 18 Jan 2002 08:41:32 -0800 (PST) Received: from kanpc.gte.com (localhost [127.0.0.1]) by h132-197-179-27.gte.com (8.11.6/8.11.4) with SMTP id g0IGfOZ94590; Fri, 18 Jan 2002 11:41:24 -0500 (EST) (envelope-from ak03@gte.com) Date: Fri, 18 Jan 2002 11:41:24 -0500 From: Alexander Kabaev To: Poul-Henning Kamp Cc: mux@sneakerz.org, freebsd-arch@FreeBSD.ORG Subject: Re: Proposal for a new mount API Message-Id: <20020118114124.61c39faf.ak03@gte.com> In-Reply-To: <12159.1011371137@critter.freebsd.dk> References: <20020118112224.236b4754.ak03@gte.com> <12159.1011371137@critter.freebsd.dk> Organization: Verizon Data Services X-Mailer: Sylpheed version 0.7.0claws8 (GTK+ 1.2.10; i386--freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > I belive what you see is the backwards compatibility stuff. How is that? The vfs_nmount sycall accepts flags as its first parameter and then proceeds testing it for flags various MNT_??? bits almost exactly as the existing mount sysccall is doing today. What is the point in backwards compatibility for these functions anyway? The new mount was supposed to become a new syscall, if I am not mistaken. -- Alexander Kabaev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 8:53:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sneakerz.org (sneakerz.org [216.33.66.254]) by hub.freebsd.org (Postfix) with ESMTP id 8569337B405 for ; Fri, 18 Jan 2002 08:53:41 -0800 (PST) Received: by sneakerz.org (Postfix, from userid 1023) id EAE405D006; Fri, 18 Jan 2002 10:53:35 -0600 (CST) Date: Fri, 18 Jan 2002 10:53:35 -0600 From: Maxime Henrion To: freebsd-arch@freebsd.org Cc: Alexander Kabaev Subject: Re: Proposal for a new mount API Message-ID: <20020118105335.A50775@sneakerz.org> References: <20020118112224.236b4754.ak03@gte.com> <12159.1011371137@critter.freebsd.dk> <20020118114124.61c39faf.ak03@gte.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20020118114124.61c39faf.ak03@gte.com>; from ak03@gte.com on Fri, Jan 18, 2002 at 11:41:24AM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Alexander Kabaev wrote: > > I belive what you see is the backwards compatibility stuff. > How is that? The vfs_nmount sycall accepts flags as its first parameter > and then proceeds testing it for flags various MNT_??? bits almost > exactly as the existing mount sysccall is doing today. What is the point > in backwards compatibility for these functions anyway? The new mount was > supposed to become a new syscall, if I am not mistaken. I didn't feel like it was worth getting rid of this flags parameter, mainly for things like MNT_RDONLY or MNT_NOEXEC which apply to all filesystems. Converting all these flags to mount options will be a pain, and an unnecessary one, IMO. However, it's true that there are some MNT_* flags that don't belong here at all (like the MNT_EX* stuff) and those will be converted to mount options once we came to an agreement concerning the API. Maxime To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 8:55:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sneakerz.org (sneakerz.org [216.33.66.254]) by hub.freebsd.org (Postfix) with ESMTP id 0E97537B417 for ; Fri, 18 Jan 2002 08:55:25 -0800 (PST) Received: by sneakerz.org (Postfix, from userid 1023) id 971935D006; Fri, 18 Jan 2002 10:55:24 -0600 (CST) Date: Fri, 18 Jan 2002 10:55:24 -0600 From: Maxime Henrion To: freebsd-arch@freebsd.org Cc: Julian Elischer Subject: Re: Proposal for a new mount API Message-ID: <20020118105524.B50775@sneakerz.org> References: <20020117173532.A48367@sneakerz.org> <3C47DB3D.22DD975A@elischer.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Mutt/1.2i In-Reply-To: <3C47DB3D.22DD975A@elischer.org>; from julian@elischer.org on Fri, Jan 18, 2002 at 12:22:21AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer wrote: > mux@sneakerz.org wrote: > [interesting stuff] > >=20 >=20 > I haven't looked at the diff itself but you seem to be heading in the rig= ht=20 > direction from what you say.. > I just wanted to let you know that we are out here reading it and that yo= u=20 > shouldn't get discouraged.. Thanks. :-) >=20 > I'll try look at the diffs themselves later as well, though any spare mom= ents I > get are being > spent on the KSE stuff. > =18 I look forward seeing your comments on this stuff if we ever get the chance to look at it. Maxime To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 9:20:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from camelot.spsl.nsc.ru (camelot.spsl.nsc.ru [194.226.174.80]) by hub.freebsd.org (Postfix) with ESMTP id B3D5537B419 for ; Fri, 18 Jan 2002 09:20:01 -0800 (PST) Received: (from root@localhost) by camelot.spsl.nsc.ru (8.11.6/8.11.6) id g0IHJs842224 for arch@freebsd.org.AVP; Fri, 18 Jan 2002 23:19:54 +0600 (NOVT) (envelope-from fjoe@iclub.nsu.ru) Received: from iclub.nsu.ru (root@iclub.nsu.ru [193.124.222.66]) by camelot.spsl.nsc.ru (8.11.6/8.11.6) with ESMTP id g0IHJrX42215; Fri, 18 Jan 2002 23:19:53 +0600 (NOVT) (envelope-from fjoe@iclub.nsu.ru) Received: (from fjoe@localhost) by iclub.nsu.ru (8.11.6/8.11.6) id g0IHJuH53624; Fri, 18 Jan 2002 23:19:56 +0600 (NS) (envelope-from fjoe) Date: Fri, 18 Jan 2002 23:19:56 +0600 From: Max Khon To: Alexey Dokuchaev Cc: arch@freebsd.org Subject: Re: request for review Message-ID: <20020118231955.A53559@iclub.nsu.ru> References: <20020115205053.A9241@iclub.nsu.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from danfe@inet.ssc.nsu.ru on Fri, Jan 18, 2002 at 09:20:31PM +0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG hi, there! On Fri, Jan 18, 2002 at 09:20:31PM +0600, Alexey Dokuchaev wrote: > And the last thing, I've been looking at the patch, and thought, maybe it > would be more clear to patch vfs_vnops.c the following way instead: > > --- /sys/kern/vfs_vnops.c.orig Fri Jan 18 21:16:33 2002 > +++ /sys/kern/vfs_vnops.c Thu Jan 17 20:26:11 2002 > @@ -530,7 +530,7 @@ > * Default to zero to catch bogus uses of this field. > */ > > - if (vap->va_type == VREG) { > + if (vap->va_type == VREG || vap->va_type == VCHR) { > sb->st_blksize = vap->va_blocksize; > } else if (vn_isdisk(vp, NULL)) { > sb->st_blksize = vp->v_rdev->si_bsize_best; > > What do you think? this will break `vn_isdisk(vp, NULL)' check (it will always return 0) /fjoe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 10:59:23 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 6C84537B405; Fri, 18 Jan 2002 10:59:13 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g0IIwuD26188; Fri, 18 Jan 2002 13:58:56 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Fri, 18 Jan 2002 13:58:56 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Dan Langille Cc: Ruslan Ermilov , Joerg Wunsch , cvs-committers@FreeBSD.ORG, cvs-all@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist In-Reply-To: <20020117053925.A18072-100000@lists.unixathome.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 17 Jan 2002, Dan Langille wrote: > We've had users coming into IRC channels asking about the man page > changes. I don't fully understand the implications yet. From the > comments above, it sounds like each time I issue 'man ls', it's going to > decompress the man page unless I do some tweaking. Is that correct? > > If so, that's not good for me, and I suspect for a large number of other > people. Else, please ignore the next paragraph. I don't understand the criticism here. Here are the two cases in the current scenario: (1) catman doesn't exist, so man decompresses the nroff source, processes the nroff, spits it into a compressed catman file, and displays it to the user (I'm not sure if there's a double decompress here or not). (2) catman does exist, so man decompresses the processed nroff and displays it for the user. In the new scenario, (1) is eschewed under almost all situations so as to avoid using a setuid application. In no situation, new or old, was compression ever not performed. > Please don't force the majority of people to tweak their systems in > order to avoid a theoretical problem for a small minority. But if you > insist upon doing so, then I feel you must provide a knob in > /etc/defaults/make.conf. Even the people who are objecting to providing no tweak here admit they are not in the majority. > I have read the commit message, but I don't have the knowledge to > understand the implications. Perhaps someone could explain it so I and > others can understand what this change really means. Then you should read the other messages in the thread, and the posts to -arch. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 11:47:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-179-27.gte.com (h132-197-179-27.gte.com [132.197.179.27]) by hub.freebsd.org (Postfix) with ESMTP id 1D7C437B404 for ; Fri, 18 Jan 2002 11:47:17 -0800 (PST) Received: from kanpc.gte.com (localhost [127.0.0.1]) by h132-197-179-27.gte.com (8.11.6/8.11.4) with SMTP id g0IJlFZ95883; Fri, 18 Jan 2002 14:47:15 -0500 (EST) (envelope-from ak03@gte.com) Date: Fri, 18 Jan 2002 14:47:15 -0500 From: Alexander Kabaev To: Maxime Henrion Cc: freebsd-arch@freebsd.org Subject: Re: Proposal for a new mount API Message-Id: <20020118144715.587a577c.ak03@gte.com> In-Reply-To: <20020118105335.A50775@sneakerz.org> References: <20020118112224.236b4754.ak03@gte.com> <12159.1011371137@critter.freebsd.dk> <20020118114124.61c39faf.ak03@gte.com> <20020118105335.A50775@sneakerz.org> Organization: Verizon Data Services X-Mailer: Sylpheed version 0.7.0claws8 (GTK+ 1.2.10; i386--freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Maxime, are parameters in iovec supposed to be strings only, or you plan to allow arbitrary binary data? I apologize for being skeptical, but I fail to see what the new mount API in its current form improves upon existing implementation if arbitrary binary data will be allowed in parameters. Each and every FS in the tree today uses void *args to pass FS-specific structure as an argument to mount(2) syscall and is effectively able to work around the 32-bit flags restriction, so iovec with binary data within is just another way to repackage the same FS-specific structure. User-land programs will still be required to have intimate knowledge about what parameters and in what exactly format (binary? struct? string? int?) given FS requires on mount. In Linux, in contrast, a single 'mount' binary is able to mount most available FS-es in uniform way, because it simply passes parameters as strings to the kernel and allows each individual FS to process them. The proposed API does not allow that unless you require all the parameters in iovec to be strings. Even then each FS still has to perform its own parameter parsing even for simplest parameters. Common library of convenience functions to fetch parameter value possibly converting it to desired type from string format could take care of that. Do we want such functionality in the kernel? I asked this question before but nobody has responded so far :(-- Alexander Kabaev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 11:50:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 14DEC37B43D for ; Fri, 18 Jan 2002 11:50:29 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0IJlrA15523; Fri, 18 Jan 2002 20:47:53 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alexander Kabaev Cc: Maxime Henrion , freebsd-arch@FreeBSD.ORG Subject: Re: Proposal for a new mount API In-Reply-To: Your message of "Fri, 18 Jan 2002 14:47:15 EST." <20020118144715.587a577c.ak03@gte.com> Date: Fri, 18 Jan 2002 20:47:53 +0100 Message-ID: <15521.1011383273@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020118144715.587a577c.ak03@gte.com>, Alexander Kabaev writes: >Maxime, > >are parameters in iovec supposed to be strings only, or you plan to >allow arbitrary binary data? The idea is to use strings wherever possible, be able to deal with binary blobs of data, but discourage its use. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 11:57: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailsrv.otenet.gr (mailsrv.otenet.gr [195.170.0.5]) by hub.freebsd.org (Postfix) with ESMTP id ED5A837B416 for ; Fri, 18 Jan 2002 11:57:01 -0800 (PST) Received: from hades.hell.gr (patr530-a089.otenet.gr [212.205.215.89]) by mailsrv.otenet.gr (8.11.5/8.11.5) with ESMTP id g0IJuwS11671; Fri, 18 Jan 2002 21:56:58 +0200 (EET) Received: (from charon@localhost) by hades.hell.gr (8.11.6/8.11.6) id g0IJutW75262; Fri, 18 Jan 2002 21:56:55 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Date: Fri, 18 Jan 2002 21:56:55 +0200 From: Giorgos Keramidas To: Ted Mittelstaedt Cc: arch@freebsd.org Subject: Re: termcap versus terminfo Message-ID: <20020118195654.GA73428@hades.hell.gr> References: <20020117213043.GB41262@hades.hell.gr> <000701c19fe1$492e17a0$1401a8c0@tedm.placo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <000701c19fe1$492e17a0$1401a8c0@tedm.placo.com> User-Agent: Mutt/1.3.25i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2002-01-17 21:30:48, Ted Mittelstaedt wrote: > This is also a good idea, we are talking maybe 11 lines of text. How's > the following sound: (in replacement of line # 39 about mailing changes > to berkeley, which isn't maintaining termcap anymore anyway) > > -------------cut here------------ > # > # To add a termcap entry under FreeBSD for a new terminal type, insert > # the entry in the appropriate location in /etc/termcap then issue this > # command: > # > # cap_mkdb -f /usr/share/misc/termcap /etc/termcap > # > # Terminfo source entries can be converted to termcap entries with the > # tic program that is part of the ncurses distribution, see the ports > # section. > # > ------------cut here------------- This looks nice (well, apart from double spaces after '#' in comments). Any src/ committers around to help us getting this committed to -CURRENT? -- Giorgos Keramidas . . . . . . . . . keramida@{ceid.upatras.gr,freebsd.org} FreeBSD Documentation Project . . . http://www.freebsd.org/docproj/ FreeBSD: The power to serve . . . . http://www.freebsd.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 13:29:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sneakerz.org (sneakerz.org [216.33.66.254]) by hub.freebsd.org (Postfix) with ESMTP id 6C7C237B400 for ; Fri, 18 Jan 2002 13:29:21 -0800 (PST) Received: by sneakerz.org (Postfix, from userid 1023) id 03DF15D006; Fri, 18 Jan 2002 15:29:20 -0600 (CST) Date: Fri, 18 Jan 2002 15:29:20 -0600 From: Maxime Henrion To: freebsd-arch@freebsd.org Cc: Alexander Kabaev Subject: Re: Proposal for a new mount API Message-ID: <20020118152920.A52542@sneakerz.org> References: <20020118112224.236b4754.ak03@gte.com> <12159.1011371137@critter.freebsd.dk> <20020118114124.61c39faf.ak03@gte.com> <20020118105335.A50775@sneakerz.org> <20020118144715.587a577c.ak03@gte.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20020118144715.587a577c.ak03@gte.com>; from ak03@gte.com on Fri, Jan 18, 2002 at 02:47:15PM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Alexander Kabaev wrote: > Maxime, > > are parameters in iovec supposed to be strings only, or you plan to > allow arbitrary binary data? As Poul already stated, binary data is allowed, but discouraged. > > I apologize for being skeptical, but I fail to see what the new mount > API in its current form improves upon existing implementation if > arbitrary binary data will be allowed in parameters. Each and every FS > in the tree today uses void *args to pass FS-specific structure as an > argument to mount(2) syscall and is effectively able to work around the > 32-bit flags restriction, so iovec with binary data within is just > another way to repackage the same FS-specific structure. No, it's not another way to repackage the structure. The iovec's are then put in a struct uio and uiomove()'d, so now we can access this FS-specific structure from the kernel as well using the kernel_mount() wrapper. This wasn't possible before, that's why this 32bits flags restriction was annoying. Moreover, mount options are now retained in the mountpoint, and it will be possible to retrieve them any time. This wasn't the case before as well. > User-land > programs will still be required to have intimate knowledge about what > parameters and in what exactly format (binary? struct? string? int?) > given FS requires on mount. In Linux, in contrast, a single 'mount' > binary is able to mount most available FS-es in uniform way, because it > simply passes parameters as strings to the kernel and allows each > individual FS to process them. Which version of Linux are you talking about ? Linux 2.2 has a void * and I thought this didn't change in Linux 2.4. > The proposed API does not allow that > unless you require all the parameters in iovec to be strings. Which explains why we discourage the use of these binary parameters. However, there are still cases where they are needed, so we can't go without them. > Even then > each FS still has to perform its own parameter parsing even for simplest > parameters. Common library of convenience functions to fetch parameter > value possibly converting it to desired type from string format could > take care of that. Do we want such functionality in the kernel? I asked > this question before but nobody has responded so far :(-- This may be reasonable. Cheers, Maxime To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 14:25:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from avocet.prod.itd.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by hub.freebsd.org (Postfix) with ESMTP id 85E1637B402 for ; Fri, 18 Jan 2002 14:25:50 -0800 (PST) Received: from pool0424.cvx22-bradley.dialup.earthlink.net ([209.179.199.169] helo=mindspring.com) by avocet.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16RhSF-0004Ed-00; Fri, 18 Jan 2002 14:25:47 -0800 Message-ID: <3C48A0E7.F97BC01@mindspring.com> Date: Fri, 18 Jan 2002 14:25:43 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Michal Mertl Cc: arch@FreeBSD.ORG Subject: Re: 64 bit counters again References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Michal Mertl wrote: > That's explaining only CPU overhead which I knew there is some. Yes, the question is whether or not it will impact anything, and to know that for an arbitrary application, you need to quantify "some". > > The additional locks required for i386 64 bit atomicity will, > > if the counter is accessed by more than one CPU, result in > > bus contention for inter-CPU coherency. > > What additional locks? The lock prefix for cmpxchg8b? It's required for 32 > bit too and it increases time spent on operation from 3 to 21 clocks > making the difference between 32 and 64 bit "only" 29 clocks instead on > 47. The additional locks on PPC, SPARC, and Alpha. THe lock also is a barrier instruction. You need to read the Intel programming guid on barrier instructions. On a P4, it will effectively stall two of the three pipelines. On standard SMP systems, it will cause a coherency cycle, if the data being changed is in a cache line on another CPU. With ithreads, this is very likely the case, since the interrupt is not necessarily likely to maintain affinity for a single CPU (at least until the scheduler gets fixed). Invalidation cycles happen at the memory bus speed. Most fast memory busses these days are 133MHz (though I've seen 233MHz, parts are hard to find), and for a 2GHz clock speed, that's a factor of 15 for the 133MHz parts and a factor of 8 for the 233MHz parts. > > > What do you mean by that? Zero-copy operation? Like sendfile? Is Apache > > > 1.x zero-copy? > > > > Yes, zero copy. Sendfile isn't ideal, but works. Apache is > > not zero copy. The idea is to not include a lot of CPU work > > on copies between the user space and the kernel, which aren't > > going to happen in an extremely optimized application. > > An "extremely optimized" application is a thing which would have > an administrator who doesn't enable costly counters. No. If we are talking a BSD-based embedded system, then it's just one written by someone who was not playing at being an engineer (assuming the performance requirements were there; otherwise, their just an engineer who went after the low hanging fruit, and it's a legitimate design decision). It's incredibly easy to get zero copy, if you are willing to make minor kernel changes to get it. It's a little harder otherwise. Note that the NFS code is zero copy (I don't know if the patches have been rolled into -current yet; I haven't been watching), and the sendfile is technically zero copy, although it had bad things about it that you have to worry about (there are several DOS attacks you can use, if you know the server is written to use sendfile, and sendfile practically requires that you store files to be sent with termination, so you might as well be running MS-DOS, and conversion of programs to expect this to be a line terminator is annoying, to say the least, and it has high relative system call overhead, compared to other approaches). > > Well, you probably should collect *all* statistics you can, > > in the most "this is the only thing I'm doing with the box" > > way you can, before and after the code change, and then plot > > the ones that get worse (or better) as a result of the change. > > Will do eventually, but unfotunately don't have the time to devote to it > at the moment. I think it's a requirement to advocate this change. [ ... ] > > I think the answer is "yes, we need atomic counters". Whether they > > need to be 64 bit or just 32 bit is really application dependent > > (we have all agreed to that, I think). > > Thanks. Do you think it's always true (STABLE/CURRENT,network device > ISRs, /sys/netinet routines) ? I think it's true of all open-ended counters, where there is a risk of overflow if they are 32 bit, and some application could be bitten by the overflow, and still be consideted to be "well written"... in other words, anywhere overflow is *expected*. > > See Bruce's posting about atomicity; I think it speaks very > > eleoquently on the issue (much more brief than what I'd write > > to say the same thing ;^)). > > If you mean the email where he talks about atomic_t ("atomic_t would be > "int" if anything") it doesn't fully apply. I am not inventing atomic_t > anymore anyway :-). Isn't there a platform, which better works with 64 bit > ints than with 32 bits (a-la 32/16 bits on modern i386)? Yes. IA64. SPARC 9b (SPARC64) and Alpha, which are 64 bits, require locks, since they don't have the ability to do an atomic "lock; cmpxchg8b". An atomic_t is a moderately good idea in this regard; for it to be other than simple "int", though, I think would require that the valus be read/written using accessor/mutators in macro form, so that the overhead on platforms that need it (everything but IA64, effectively -- I don't think this point has been commmunicated strongly enough!) could be masked on the platforms that don't have nearly as high an overhead for the operations. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 14:28:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-179-27.gte.com (h132-197-179-27.gte.com [132.197.179.27]) by hub.freebsd.org (Postfix) with ESMTP id 09DCE37B402 for ; Fri, 18 Jan 2002 14:28:22 -0800 (PST) Received: from kanpc.gte.com (localhost [127.0.0.1]) by h132-197-179-27.gte.com (8.11.6/8.11.4) with SMTP id g0IMSKZ97550; Fri, 18 Jan 2002 17:28:20 -0500 (EST) (envelope-from ak03@gte.com) Date: Fri, 18 Jan 2002 17:28:20 -0500 From: Alexander Kabaev To: Maxime Henrion Cc: freebsd-arch@freebsd.org Subject: Re: Proposal for a new mount API Message-Id: <20020118172820.21f4e2b5.ak03@gte.com> In-Reply-To: <20020118152920.A52542@sneakerz.org> References: <20020118112224.236b4754.ak03@gte.com> <12159.1011371137@critter.freebsd.dk> <20020118114124.61c39faf.ak03@gte.com> <20020118105335.A50775@sneakerz.org> <20020118144715.587a577c.ak03@gte.com> <20020118152920.A52542@sneakerz.org> Organization: Verizon Data Services X-Mailer: Sylpheed version 0.7.0claws8 (GTK+ 1.2.10; i386--freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I do not see what 32bit restriction and inability to call vfs_mount from kernel space have in common. The new API clearly does not solve 32-bit flags problem, as no such problem really existed. It just provides more elegant way of doing what practically each filesystem has done on its own before. It was possible to use ugly stackgap hack to call vfs_mount from kernel before. The new method is obviously infinitely more elegant. That is the only advantage of the new API I see so far. > Moreover, mount options are now retained in the mountpoint, and it > will be possible to retrieve them any time. This wasn't the case > before as well. Almost every filesystem in the tree has arg_ struct as the first member of its private mount structure. If parameters can only be decoded by the filesystem itself, what is advantage of keeping it in FS-independent mount struct? What good is in discouraging binary parameters if you cannot tell beforehand which parameter is safe to access as a string and which is not? > Which version of Linux are you talking about ? Linux 2.2 has a void * > and I thought this didn't change in Linux 2.4. It did not, but all filesytems in Linux kernel simply treat it as a string, containing comma-separated options from the mount -o command. > Which explains why we discourage the use of these binary parameters. > However, there are still cases where they are needed, so we can't go > without them. Could you please provide an example of such option? Is it absolutely necessary to keep in binary? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 14:45:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from sneakerz.org (sneakerz.org [216.33.66.254]) by hub.freebsd.org (Postfix) with ESMTP id EFD0D37B400 for ; Fri, 18 Jan 2002 14:45:02 -0800 (PST) Received: by sneakerz.org (Postfix, from userid 1023) id 4C9335D006; Fri, 18 Jan 2002 16:44:57 -0600 (CST) Date: Fri, 18 Jan 2002 16:44:57 -0600 From: Maxime Henrion To: freebsd-arch@freebsd.org Cc: Alexander Kabaev Subject: Re: Proposal for a new mount API Message-ID: <20020118164457.B52542@sneakerz.org> References: <20020118112224.236b4754.ak03@gte.com> <12159.1011371137@critter.freebsd.dk> <20020118114124.61c39faf.ak03@gte.com> <20020118105335.A50775@sneakerz.org> <20020118144715.587a577c.ak03@gte.com> <20020118152920.A52542@sneakerz.org> <20020118172820.21f4e2b5.ak03@gte.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20020118172820.21f4e2b5.ak03@gte.com>; from ak03@gte.com on Fri, Jan 18, 2002 at 05:28:20PM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Alexander Kabaev wrote: > I do not see what 32bit restriction and inability to call vfs_mount from > kernel space have in common. The new API clearly does not solve 32-bit > flags problem, as no such problem really existed. It just provides more > elegant way of doing what practically each filesystem has done on its > own before. Because if you can't use the void * pointer from the kernel (or with gross hacks) you will resort to use the flags parameter. Now that we have an easy way to pass an indeterminate number of options, from the kernel or from userland, we don't care about this flags parameter being only 32bits. > It was possible to use ugly stackgap hack to call vfs_mount from kernel > before. The new method is obviously infinitely more elegant. That is the > only advantage of the new API I see so far. Yes, using stackgap would be a gross hack IMO. > > Moreover, mount options are now retained in the mountpoint, and it > > will be possible to retrieve them any time. This wasn't the case > > before as well. > Almost every filesystem in the tree has arg_ struct as the first > member of its private mount structure. If parameters can only be decoded > by the filesystem itself, what is advantage of keeping it in > FS-independent mount struct? What good is in discouraging binary > parameters if you cannot tell beforehand which parameter is safe to > access as a string and which is not? We can tell if a parameter is safe to access as a string or not. The API described in my original mail allows it very easily, though maybe a vfs_getopt_string() that does the checks would be needed. The advantage of retaining the mount options is because some code may want to look at it. For example, Robert Watson recently told me that the MAC code will be a consumer of mount options, this at execve() time. And it's good to discourage use of binary parameters because with a bit of luck (if people cares) the noone will use them if they are not really needed. Note that binary parameters in themselves don't ncecessarily prevent us from being able to mount the fs in a generic way, they only prevents it if this binary parameter is mandatory to the filesystem. > > > Which version of Linux are you talking about ? Linux 2.2 has a void * > > and I thought this didn't change in Linux 2.4. > It did not, but all filesytems in Linux kernel simply treat it as a > string, containing comma-separated options from the mount -o > command. > > > Which explains why we discourage the use of these binary parameters. > > However, there are still cases where they are needed, so we can't go > > without them. > Could you please provide an example of such option? Is it absolutely > necessary to keep in binary? There was examples in my original post that you apparently didn't read entirely. Cheers, Maxime To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 14:51:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id C0C5537B43F for ; Fri, 18 Jan 2002 14:51:07 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id g0IMmVA18159; Fri, 18 Jan 2002 23:48:31 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Maxime Henrion Cc: freebsd-arch@FreeBSD.ORG, Alexander Kabaev Subject: Re: Proposal for a new mount API In-Reply-To: Your message of "Fri, 18 Jan 2002 16:44:57 CST." <20020118164457.B52542@sneakerz.org> Date: Fri, 18 Jan 2002 23:48:31 +0100 Message-ID: <18157.1011394111@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Let me try to clarify how the new mount api is much stronger and completely abolishes the 32 bit thing. If I issue the command: mount -t devfs -o thisopt -o thatopt=3 -o yetanopt=DTRT /home/dev A uiovec will be passed into the kernel which looks like this: uiovec[0] = "thisopt" uiovec[1] = "" uiovec[2] = "thatopt" uiovec[3] = "3" uiovec[4] = "yetanopt" uiovec[5] = "DTRT" uiovec[6] = "vfstype" uiovec[7] = "devfs" uiovec[8] = "mountpoint" uiovec[9] = "/home/dev" As you can see this is extensible in all directions: As long as I stay with ascii options, I don't have to write a "mount_devfs" program, and I can introduce new options and filesystems without rewriting the current mount program. Now, it may be that a particular filesystem needs to pass binary data, a UNIcode table has been mentioned as an example, in that case you can do that, but then you need to write a mount_foofs program to do that. Does that address your concerns ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 16:27:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 4C07437B417; Fri, 18 Jan 2002 16:27:36 -0800 (PST) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 0F40C78307; Sat, 19 Jan 2002 10:57:34 +1030 (CST) Date: Sat, 19 Jan 2002 10:57:34 +1030 From: Greg Lehey To: Robert Watson Cc: Dan Langille , Ruslan Ermilov , Joerg Wunsch , cvs-committers@FreeBSD.ORG, cvs-all@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist Message-ID: <20020119105733.A50299@wantadilla.lemis.com> References: <20020117053925.A18072-100000@lists.unixathome.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.23i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Friday, 18 January 2002 at 13:58:56 -0500, Robert Watson wrote: > On Thu, 17 Jan 2002, Dan Langille wrote: > >> We've had users coming into IRC channels asking about the man page >> changes. I don't fully understand the implications yet. From the >> comments above, it sounds like each time I issue 'man ls', it's going to >> decompress the man page unless I do some tweaking. Is that correct? >> >> If so, that's not good for me, and I suspect for a large number of other >> people. Else, please ignore the next paragraph. > > I don't understand the criticism here. Here are the two cases in the > current scenario: > > (1) catman doesn't exist, so man decompresses the nroff source, processes > the nroff, spits it into a compressed catman file, and displays it to > the user (I'm not sure if there's a double decompress here or not). > > (2) catman does exist, so man decompresses the processed nroff and > displays it for the user. > > In the new scenario, (1) is eschewed under almost all situations so as to > avoid using a setuid application. In no situation, new or old, was > compression ever not performed. Certainly the compression's not the issue. But the formatting time could be. It's not as fast as that. I can think of a couple of compromises: 1. Just don't install /usr/share/man/catman. 2. Use a different user ID or group for catman, and make man(1) setuid or setgid to that. >> Please don't force the majority of people to tweak their systems in >> order to avoid a theoretical problem for a small minority. But if you >> insist upon doing so, then I feel you must provide a knob in >> /etc/defaults/make.conf. > > Even the people who are objecting to providing no tweak here admit they > are not in the majority. Not in this forum. But we're not typical of the user base. I will continue to use catman, probably making it world writeable, since in my situation this isn't a compromise. But what about the man in the street? Greg -- See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 16:34:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id C875F37B402; Fri, 18 Jan 2002 16:34:25 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id g0J0YJD30868; Fri, 18 Jan 2002 19:34:20 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Fri, 18 Jan 2002 19:34:19 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Greg Lehey Cc: Dan Langille , Ruslan Ermilov , Joerg Wunsch , cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org, arch@FreeBSD.org Subject: Re: cvs commit: src/gnu/usr.bin/man/man Makefile man.c src/etc/mtree BSD.local.dist BSD.usr.dist BSD.x11-4.dist BSD.x11.dist In-Reply-To: <20020119105733.A50299@wantadilla.lemis.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 19 Jan 2002, Greg Lehey wrote: > > In the new scenario, (1) is eschewed under almost all situations so as to > > avoid using a setuid application. In no situation, new or old, was > > compression ever not performed. > > Certainly the compression's not the issue. But the formatting time > could be. It's not as fast as that. He suggested the compression was a problem, so I objected that it wasn't changed. Installing the existing catman distribution already eliminates all performance concerns by simply never rendering the pages on user-time. Making that the default might be a reasonable choice if we turn off setuid. > I can think of a couple of compromises: > > 1. Just don't install /usr/share/man/catman. That was what was being proposed initially: remove the setuid bit on man, and then allow the user to either install pre-generated catman, or not install it at all. > 2. Use a different user ID or group for catman, and make man(1) > setuid or setgid to that. That's what is currently there. Is this supposed to be different somehow? > >> Please don't force the majority of people to tweak their systems in > >> order to avoid a theoretical problem for a small minority. But if you > >> insist upon doing so, then I feel you must provide a knob in > >> /etc/defaults/make.conf. > > > > Even the people who are objecting to providing no tweak here admit they > > are not in the majority. > > Not in this forum. But we're not typical of the user base. I will > continue to use catman, probably making it world writeable, since in my > situation this isn't a compromise. But what about the man in the > street? The difference between the developers and the users is that the users hardly ever change the man pages, and so would probably benefit most from simply using the catman pages in a pregenerated form, rather than having to wait for each page to render the first time they read it, gradually consuming more and more disk space as they read more manpages. This disagreement is all about how developers might change man pages, and therefore need them regenerated, which as has been pointed out, is not something that "the man on the street" does. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 16:58:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from prg.traveller.cz (prg.traveller.cz [193.85.2.77]) by hub.freebsd.org (Postfix) with ESMTP id 4F58837B404 for ; Fri, 18 Jan 2002 16:57:58 -0800 (PST) Received: from prg.traveller.cz (localhost [127.0.0.1]) by prg.traveller.cz (8.12.1[KQ-CZ](1)/8.12.1/pukvis) with ESMTP id g0J0vug8007567; Sat, 19 Jan 2002 01:57:56 +0100 (CET) Received: from localhost (mime@localhost) by prg.traveller.cz (8.12.1[KQ-CZ](1)/pukvis) with ESMTP id g0J0vuSv007564; Sat, 19 Jan 2002 01:57:56 +0100 (CET) Date: Sat, 19 Jan 2002 01:57:56 +0100 (CET) From: Michal Mertl To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Re: 64 bit counters again In-Reply-To: <3C48A0E7.F97BC01@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 18 Jan 2002, Terry Lambert wrote: > > > The additional locks required for i386 64 bit atomicity will, > > > if the counter is accessed by more than one CPU, result in > > > bus contention for inter-CPU coherency. > > > > What additional locks? The lock prefix for cmpxchg8b? It's required for 32 > > bit too and it increases time spent on operation from 3 to 21 clocks > > making the difference between 32 and 64 bit "only" 29 clocks instead on > > 47. > > The additional locks on PPC, SPARC, and Alpha. Do I understand correctly that 64 bit atomic operations are significantly more expensive/impossible without lock on these platforms? That sounds strange for 64 bit platforms. > THe lock also is a barrier instruction. You need to read the > Intel programming guid on barrier instructions. On a P4, it Yes. That's what I described with "(memory bandwidth with lock operations?) will suffer". > > > > What do you mean by that? Zero-copy operation? Like sendfile? Is Apache > > > > 1.x zero-copy? > > > > > > Yes, zero copy. Sendfile isn't ideal, but works. Apache is > > > not zero copy. The idea is to not include a lot of CPU work > > > on copies between the user space and the kernel, which aren't > > > going to happen in an extremely optimized application. > > > > An "extremely optimized" application is a thing which would have > > an administrator who doesn't enable costly counters. > > No. If we are talking a BSD-based embedded system, then it's just > one written by someone who was not playing at being an engineer > (assuming the performance requirements were there; otherwise, > their just an engineer who went after the low hanging fruit, and > it's a legitimate design decision). > Yes if the default mode of operation of mentioned counters is going to stay the same as today. I'd like to stop this thread because you're still explaining me why 64 bit is expensive when I already switched the subject. I don't push anyone to 64 bit. What I'm offering now is: 1) 64 bit atomic ops which probably could be easily added - probably even to base tree (/machine/atomic.h). 2) "API" for counters which I think can help people to change most/all counter accesses to use the right operation (what's right depends on the actual counter and/or point on kernel codepath, where it's occuring - I don't know what's right - you say we always need atomicity) - it can be atomic, "simple", per-cpu (this may be hard to implement but with inlining it should be possible) or whatever). It's everyones decision to #define what to use. I would keep default 32 bit "simple" - same performance - same (potencial) problems. I'll polish my patch to STABLE and post a link to hackers. > > > Well, you probably should collect *all* statistics you can, > > > in the most "this is the only thing I'm doing with the box" > > > way you can, before and after the code change, and then plot > > > the ones that get worse (or better) as a result of the change. > > > > Will do eventually, but unfotunately don't have the time to devote to it > > at the moment. > > I think it's a requirement to advocate this change. No if it doesn't go into the standard tree and no if the actual functionality/performance isn't affected by default. > > > I think the answer is "yes, we need atomic counters". Whether they > > > need to be 64 bit or just 32 bit is really application dependent > > > (we have all agreed to that, I think). > > > > Thanks. Do you think it's always true (STABLE/CURRENT,network device > > ISRs, /sys/netinet routines) ? > > I think it's true of all open-ended counters, where there is > a risk of overflow if they are 32 bit, and some application > could be bitten by the overflow, and still be consideted to > be "well written"... in other words, anywhere overflow is > *expected*. I don't understand again. > > > See Bruce's posting about atomicity; I think it speaks very > > > eleoquently on the issue (much more brief than what I'd write > > > to say the same thing ;^)). > > > > If you mean the email where he talks about atomic_t ("atomic_t would be > > "int" if anything") it doesn't fully apply. I am not inventing atomic_t > > anymore anyway :-). Isn't there a platform, which better works with 64 bit > > ints than with 32 bits (a-la 32/16 bits on modern i386)? > > Yes. IA64. SPARC 9b (SPARC64) and Alpha, which are 64 > bits, require locks, since they don't have the ability to > do an atomic "lock; cmpxchg8b". Can they do "lock; add const,(mem)" in 32 or 64 bit? I suppore not. I don't care about cmp part - we need addition. cmpxchg8b is used because that's the only way on X86 CPU to access (read/modify/write) 64 bit memory in one operation (without using FPU). -- Michal Mertl mime@traveller.cz To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 17: 6:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-179-27.gte.com (h132-197-179-27.gte.com [132.197.179.27]) by hub.freebsd.org (Postfix) with ESMTP id B0D2837B404 for ; Fri, 18 Jan 2002 17:06:10 -0800 (PST) Received: (from ak03@localhost) by h132-197-179-27.gte.com (8.11.6/8.11.4) id g0J169O98585; Fri, 18 Jan 2002 20:06:09 -0500 (EST) (envelope-from ak03) Date: Fri, 18 Jan 2002 20:06:09 -0500 From: "Alexander N. Kabaev" To: Maxime Henrion Cc: freebsd-arch@freebsd.org Subject: Re: Proposal for a new mount API Message-ID: <20020119010608.GA98105@kanpc.gte.com> References: <20020118112224.236b4754.ak03@gte.com> <12159.1011371137@critter.freebsd.dk> <20020118114124.61c39faf.ak03@gte.com> <20020118105335.A50775@sneakerz.org> <20020118144715.587a577c.ak03@gte.com> <20020118152920.A52542@sneakerz.org> <20020118172820.21f4e2b5.ak03@gte.com> <20020118164457.B52542@sneakerz.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020118164457.B52542@sneakerz.org> User-Agent: Mutt/1.3.25i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > Because if you can't use the void * pointer from the kernel (or with gross > hacks) you will resort to use the flags parameter. If you can be satisfied with a simple flag, nothing prevents you from simply adding corresponding field into _args struct and copyin whole struct at the mount time and even save it as a part of your private mount structure for future use. You could then reference fields of the structure directly whithout calling the new accessor function which is needed for new API. > We can tell if a parameter is safe to access as a string or not. The API > described in my original mail allows it very easily, though maybe a > vfs_getopt_string() that does the checks would be needed. How are you going to distinguish 32-bit integer 0x00202020 from " " on i386 unless you specifically mark it as integer somehow? > The advantage of retaining the mount options is because some code may > want to look at it. For example, Robert Watson recently told me that > the MAC code will be a consumer of mount options, this at execve() time. > > And it's good to discourage use of binary parameters because with a bit > of luck (if people cares) the noone will use them if they are not really > needed. Note that binary parameters in themselves don't ncecessarily > prevent us from being able to mount the fs in a generic way, they only > prevents it if this binary parameter is mandatory to the filesystem. Agreed. I was just arguing that all custom mount parameters going in and out of the kernel should preferably be strings for so that they safely can be included in a mount(8) output or read by automounter from a text map and then used to mount a new filesystem, for instance, even though neither mount(8) nor automounter have slightest idea what they actually mean. > There was examples in my original post that you apparently didn't read > entirely. If only that was so simple. Your example of recode tables is actually not representative. Custom loaded recode tables were supposed to be a temporary hack. Now, when libiconv is available as a kernel module, these filesystems should follow smbfs lead and use iconv instead. All they will need is to pass from/to charset names to use and let their mount programs to preload necessary conversion tables. If Linux managed to get by without binary parameters in mount then there is no reason we can't. All in all I am pretty much in favor of the new API. I just want the message about binary parameters not welcome to be stressed more strongly. Thanks for taking time to answer my questions. -- Alexander Kabaev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 17:17:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-179-27.gte.com (h132-197-179-27.gte.com [132.197.179.27]) by hub.freebsd.org (Postfix) with ESMTP id 5111537B400 for ; Fri, 18 Jan 2002 17:17:18 -0800 (PST) Received: (from ak03@localhost) by h132-197-179-27.gte.com (8.11.6/8.11.4) id g0J1H4x98655; Fri, 18 Jan 2002 20:17:04 -0500 (EST) (envelope-from ak03) Date: Fri, 18 Jan 2002 20:17:04 -0500 From: "Alexander N. Kabaev" To: Poul-Henning Kamp Cc: Maxime Henrion , freebsd-arch@FreeBSD.ORG Subject: Re: Proposal for a new mount API Message-ID: <20020119011702.GB98105@kanpc.gte.com> References: <20020118164457.B52542@sneakerz.org> <18157.1011394111@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <18157.1011394111@critter.freebsd.dk> User-Agent: Mutt/1.3.25i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > Does that address your concerns ? Yes. I think this API can be made even more powerful when paired with helper generic parameter processing library which is able to convert data from string to the binary format or handle weird parameters names such as 'suid'/'nosuid' which actually refer to single actual parameter. -- Alexander Kabaev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 19:50:39 2002 Delivered-To: freebsd-arch@freebsd.org Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by hub.freebsd.org (Postfix) with ESMTP id B7DDC37B41C for ; Fri, 18 Jan 2002 19:50:31 -0800 (PST) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.11.4/8.11.4) id g0J3oNN08944; Fri, 18 Jan 2002 22:50:23 -0500 (EST) (envelope-from wollman) Date: Fri, 18 Jan 2002 22:50:23 -0500 (EST) From: Garrett Wollman Message-Id: <200201190350.g0J3oNN08944@khavrinen.lcs.mit.edu> To: mime@traveller.cz Subject: Re: 64 bit counters again X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: References: Organization: MIT Laboratory for Computer Science Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In article you write: >> Yes. IA64. SPARC 9b (SPARC64) and Alpha, which are 64 >> bits, require locks, since they don't have the ability to >> do an atomic "lock; cmpxchg8b". > >Can they do "lock; add const,(mem)" in 32 or 64 bit? Terry is talking out of his ear again. Alpha most definitely does have 64-bit load-linked/store-conditional, which is entirely equivalent to i386's 64-bit compare-exchange. (For some reason in the Alpha ARM it's called ``load locked'' instead.) If we cared about running on MIPS architectures, they have LL/SC as well. SPARC v9 has 64-bit compare-and-swap (although unlike in IA32 a memory barrier is also required); see the SPARC v9 Architecture Reference Manual. IA64 has 64-bit compare-exchange as well; see the Itanium Architecture Software Developer's Manual, volume 2 (Intel document number 245318-003), where there's a whole section on implementing synchronization primitives on IA84; it also has an atomic FETCHADD instruction which I didn't bother to look more closely at. -GAWollman -- Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same wollman@lcs.mit.edu | O Siem / The fires of freedom Opinions not those of| Dance in the burning flame MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 18 20:58:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id E2F5737B416 for ; Fri, 18 Jan 2002 20:58:47 -0800 (PST) Received: from pool0211.cvx40-bradley.dialup.earthlink.net ([216.244.42.211] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16RnaE-0000vb-00; Fri, 18 Jan 2002 20:58:26 -0800 Message-ID: <3C48FCEF.9190CA08@mindspring.com> Date: Fri, 18 Jan 2002 20:58:23 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Garrett Wollman Cc: mime@traveller.cz, arch@FreeBSD.org Subject: Re: 64 bit counters again References: <200201190350.g0J3oNN08944@khavrinen.lcs.mit.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Garrett Wollman wrote: > >> Yes. IA64. SPARC 9b (SPARC64) and Alpha, which are 64 > >> bits, require locks, since they don't have the ability to > >> do an atomic "lock; cmpxchg8b". > > > >Can they do "lock; add const,(mem)" in 32 or 64 bit? > > Terry is talking out of his ear again. > > Alpha most definitely does have 64-bit load-linked/store-conditional, > which is entirely equivalent to i386's 64-bit compare-exchange. (For > some reason in the Alpha ARM it's called ``load locked'' instead.) If > we cared about running on MIPS architectures, they have LL/SC as well. > SPARC v9 has 64-bit compare-and-swap (although unlike in IA32 a memory > barrier is also required); see the SPARC v9 Architecture Reference > Manual. IA64 has 64-bit compare-exchange as well; see the Itanium > Architecture Software Developer's Manual, volume 2 (Intel document > number 245318-003), where there's a whole section on implementing > synchronization primitives on IA84; it also has an atomic FETCHADD > instruction which I didn't bother to look more closely at. John Baldwin said: | Well, SMP on Pentium's maybe, but not on Alpha, sparc64, or ia64, all of which | support OOE and looser memory models than x86, meaning that you really need | locks unless you are going to have 386-specific code all over the place. I | suppose you can wrap it behind an MI API but that seems like a lot of work for | fairly small gain that can end up making the code uglier. So you can argue it out with him, Garrett. Meanwhile, with a 32 bit cmpxchg, I don't need locks at all, unless I want 64 bit counters so I can measure the wrong thing. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 1:21: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mta7.pltn13.pbi.net (mta7.pltn13.pbi.net [64.164.98.8]) by hub.freebsd.org (Postfix) with ESMTP id E492D37B400 for ; Sat, 19 Jan 2002 01:21:07 -0800 (PST) Received: from elischer.org ([64.164.10.141]) by mta7.pltn13.pbi.net (iPlanet Messaging Server 5.1 (built May 7 2001)) with ESMTP id <0GQ600G32HZ7RF@mta7.pltn13.pbi.net> for arch@freebsd.org; Sat, 19 Jan 2002 01:21:07 -0800 (PST) Date: Sat, 19 Jan 2002 01:21:07 -0800 From: Julian Elischer Subject: doreti() and userret() To: arch@freebsd.org Message-id: <3C493A80.F77A2283@elischer.org> MIME-version: 1.0 X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) Content-type: text/plain; charset=iso-8859-2 Content-transfer-encoding: 7BIT X-Accept-Language: en, hu Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Userret() is supposed to be called the last thing before the processor is returned to userland.. As such it is called from syscall() and from fork_return(). However AFTER they call it doreti() is called, which in turn calls ast which sometimes calls userret() again..... I can't help but feel that there could be some retionalisation there.. Also, either userret() is the last thing to be called or ast() is, or maybe doreti is, but certainly they can't ALL be the last thing called before the kernel returns to user mode. Is it possible that we could take the userret() out of syscall() and fork_return() and just make sure that it is always called from doreti()? Julian -- +------------------------------------+ ______ _ __ | __--_|\ Julian Elischer | \ U \/ / hard at work in | / \ julian@elischer.org +------>x USA \ a very strange | ( OZ ) \___ ___ | country ! +- X_.---._/ presently in San Francisco \_/ \\ v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 4:22:53 2002 Delivered-To: freebsd-arch@freebsd.org Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by hub.freebsd.org (Postfix) with ESMTP id 9CCA837B402 for ; Sat, 19 Jan 2002 04:22:50 -0800 (PST) Received: from sheldonh (helo=axl.seasidesoftware.co.za) by axl.seasidesoftware.co.za with local-esmtp (Exim 3.33 #1) id 16RuYE-0006qe-00; Sat, 19 Jan 2002 14:24:50 +0200 From: Sheldon Hearn To: Giorgos Keramidas Cc: Ted Mittelstaedt , arch@freebsd.org Subject: Re: termcap versus terminfo In-reply-to: Your message of "Fri, 18 Jan 2002 21:56:55 +0200." <20020118195654.GA73428@hades.hell.gr> Date: Sat, 19 Jan 2002 14:24:50 +0200 Message-ID: <26327.1011443090@axl.seasidesoftware.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 18 Jan 2002 21:56:55 +0200, Giorgos Keramidas wrote: > This looks nice (well, apart from double spaces after '#' in > comments). Any src/ committers around to help us getting this > committed to -CURRENT? Sure, as soon as the ports guys have sorted out the ncurses mess as per misc/33906. :-) Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 9:31:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 0185E37B41D for ; Sat, 19 Jan 2002 09:30:58 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id EAA22551; Sun, 20 Jan 2002 04:30:51 +1100 Date: Sun, 20 Jan 2002 04:32:16 +1100 (EST) From: Bruce Evans X-X-Sender: To: Julian Elischer Cc: Subject: Re: doreti() and userret() In-Reply-To: <3C493A80.F77A2283@elischer.org> Message-ID: <20020120034844.A5320-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 19 Jan 2002, Julian Elischer wrote: > Userret() is supposed to be called the last thing before the > processor is returned to userland.. At least on i386's, this is only true for the pessimized returns from syscall() and trap(). Returns from interrupt handlers normally just return. They sometimes (rarely) go through ast() to handle cases where userret() would do something. In my version of FreeBSD, userret() is almost a no-op and almost all the things that were in userret() are in ast(). > As such it is called from syscall() and from fork_return(). > However AFTER they call it doreti() is called, which in turn > calls ast which sometimes calls userret() again..... > I can't help but feel that there could be some retionalisation there.. There may be any number of calls to userret() for a single return to userland, to handle events that arrive while previous calls are being processed. Unnecessary calls are just pessimizations. In -current there is one call in syscall(), etc. and a loop in ast() that may make any number of calls. The loop must check for new events atomically with exiting. The complications for this are best handled in ast(). > Also, either userret() is the last thing to be called or > ast() is, or maybe doreti is, but > certainly they can't ALL be the last thing called before the kernel returns > to user mode. The logical order is (doreti, ast, userret, iret). > Is it possible that we could take the userret() out of > syscall() and fork_return() > and just make sure that it is always called from doreti()? I haven't completely taken it out. I do the pri_level setting inline and call userret() only to handle the PS_PROFIL case. I think -current only needs it for precisely these things too. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 10: 0:41 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id DFFD637B404 for ; Sat, 19 Jan 2002 10:00:13 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020119180010.KIEG26243.rwcrmhc51.attbi.com@InterJet.elischer.org>; Sat, 19 Jan 2002 18:00:10 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id JAA02213; Sat, 19 Jan 2002 09:49:49 -0800 (PST) Date: Sat, 19 Jan 2002 09:49:48 -0800 (PST) From: Julian Elischer To: Bruce Evans Cc: arch@FreeBSD.ORG Subject: Re: doreti() and userret() In-Reply-To: <20020120034844.A5320-100000@gamplex.bde.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG My interest in this is because of KSEs. I have added code to doreti (assuming it was the last place I could add C code (due to the name) and now I find that it's not quite so simple.. (I never really looked ast ASTs much before) WHat I do before returning to userland is: 1/ (Only in KSE processes) check if there are other threads also wanting to return. If so, the current thread, having saved it's state to userland already, just exits, (thrtead_exic() calls cpu_throw(). If we are the last returning thread, we have also already written our frame out to userland, and we instead, do an UPCALL to the userland scheduler. 2/ check if we should be STOPPED or Single threadding (a variant of STOPPED where a particular thread is immune). If so, then the thread can not return to userland and in put in a per-process "suspended" queue, and mi_switch() is called. Some of this code is also done in non KSE processes. I'm now trying to work out where the real correct place to put these code segments is. On Sun, 20 Jan 2002, Bruce Evans wrote: > On Sat, 19 Jan 2002, Julian Elischer wrote: > > > Userret() is supposed to be called the last thing before the > > processor is returned to userland.. > > At least on i386's, this is only true for the pessimized returns from > syscall() and trap(). Returns from interrupt handlers normally just > return. They sometimes (rarely) go through ast() to handle cases where > userret() would do something. In my version of FreeBSD, userret() is > almost a no-op and almost all the things that were in userret() are in > ast(). So, Bruce, can you outline to use what is in BDEBSD that we can expect to see in the future? It soulds to em like it would be nice to get some more of your work in.. > > > As such it is called from syscall() and from fork_return(). > > However AFTER they call it doreti() is called, which in turn > > calls ast which sometimes calls userret() again..... > > I can't help but feel that there could be some retionalisation there.. > > There may be any number of calls to userret() for a single return to > userland, to handle events that arrive while previous calls are being > processed. Unnecessary calls are just pessimizations. In -current > there is one call in syscall(), etc. and a loop in ast() that may make > any number of calls. The loop must check for new events atomically > with exiting. The complications for this are best handled in ast(). Maybe there needs to be a single place for "really last" things. > > > Also, either userret() is the last thing to be called or > > ast() is, or maybe doreti is, but > > certainly they can't ALL be the last thing called before the kernel returns > > to user mode. > > The logical order is (doreti, ast, userret, iret). So you would remove userret from syscall() and fork_return()? How abou from ast()? > > > Is it possible that we could take the userret() out of > > syscall() and fork_return() > > and just make sure that it is always called from doreti()? > > I haven't completely taken it out. I do the pri_level setting inline and > call userret() only to handle the PS_PROFIL case. I think -current > only needs it for precisely these things too. I have quite a bit more to do in a threaded process. > > Bruce > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 11: 4:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id D235337B405 for ; Sat, 19 Jan 2002 11:04:28 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id 628EF10DDF8; Sat, 19 Jan 2002 11:04:28 -0800 (PST) Date: Sat, 19 Jan 2002 11:04:28 -0800 From: Alfred Perlstein To: Julian Elischer Cc: Bruce Evans , arch@FreeBSD.ORG Subject: Re: doreti() and userret() Message-ID: <20020119110428.Z13686@elvis.mu.org> References: <20020120034844.A5320-100000@gamplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from julian@elischer.org on Sat, Jan 19, 2002 at 09:49:48AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Julian Elischer [020119 10:01] wrote: > > On Sun, 20 Jan 2002, Bruce Evans wrote: > > > On Sat, 19 Jan 2002, Julian Elischer wrote: > > > > > Userret() is supposed to be called the last thing before the > > > processor is returned to userland.. > > > > At least on i386's, this is only true for the pessimized returns from > > syscall() and trap(). Returns from interrupt handlers normally just > > return. They sometimes (rarely) go through ast() to handle cases where > > userret() would do something. In my version of FreeBSD, userret() is > > almost a no-op and almost all the things that were in userret() are in > > ast(). > > So, Bruce, can you outline to use what is in BDEBSD that we can expect > to see in the future? It soulds to em like it would be nice to get some > more of your work in.. I would too, Bruce please stop holding out on us. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductable donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 19:38:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from D00015.dialonly.kemerovo.su (www2.svzserv.kemerovo.su [213.184.65.86]) by hub.freebsd.org (Postfix) with ESMTP id C98F537B402 for ; Sat, 19 Jan 2002 19:38:25 -0800 (PST) Received: (from eugen@localhost) by D00015.dialonly.kemerovo.su (8.11.6/8.11.6) id g0K3PLj00479 for arch@freebsd.org; Sun, 20 Jan 2002 10:25:21 +0700 (KRAT) (envelope-from eugen) Date: Sun, 20 Jan 2002 10:25:21 +0700 From: Eugene Grosbein To: arch@freebsd.org Subject: [PATCH] msdosfs: differrent masks for directories and other files Message-ID: <20020120102521.A450@grosbein.pp.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi! There is very frequently asked (at least, in Russian newsgroups) question: is it possible to make Midnight Commander enter to archives on msdosfs and stop his attempts to execute files there? The only way is to use '-m 644', but then you can enter directories as root only. Finally I got tired of these questions and patched kernel, mount_msdosfs and its man page to accept new option '-M mask'. This mask is used for directories only, if supplied. If -M is used and -m is not then supplied mask is used for all objects. I do not run CURRENT so this patch is for 4.5RC1 That is my first 6K kernel patch so please review. It works at least for me :-) Index: sbin/mount_msdos/mount_msdos.8 =================================================================== RCS file: /home/ncvs/src/sbin/mount_msdos/Attic/mount_msdos.8,v retrieving revision 1.19.2.1 diff -u -r1.19.2.1 mount_msdos.8 --- sbin/mount_msdos/mount_msdos.8 8 Dec 2000 14:03:59 -0000 1.19.2.1 +++ sbin/mount_msdos/mount_msdos.8 19 Jan 2002 05:55:22 -0000 @@ -42,6 +42,7 @@ .Op Fl u Ar uid .Op Fl g Ar gid .Op Fl m Ar mask +.Op Fl M Ar mask .Op Fl s .Op Fl l .Op Fl 9 @@ -105,11 +106,22 @@ for more information about octal file modes.) Only the nine low-order bits of .Ar mask -are used. +are used. The value of +.Ar -M +is used if it is supplied and +.Ar -m +is omitted. The default .Ar mask is taken from the directory on which the file system is being mounted. +.It Fl M Ar mask +Specify the maximum file permissions for directories +in the file system. The value of +.Ar -m +is used if it is supplied and +.Ar -M +is omitted. See description of previous option for details. .It Fl s Force behaviour to ignore and not generate Win'95 long filenames. Index: sbin/mount_msdos/mount_msdos.c =================================================================== RCS file: /home/ncvs/src/sbin/mount_msdos/Attic/mount_msdos.c,v retrieving revision 1.19.2.1 diff -u -r1.19.2.1 mount_msdos.c --- sbin/mount_msdos/mount_msdos.c 20 Jul 2000 10:35:13 -0000 1.19.2.1 +++ sbin/mount_msdos/mount_msdos.c 19 Jan 2002 04:30:27 -0000 @@ -88,15 +88,15 @@ { struct msdosfs_args args; struct stat sb; - int c, error, mntflags, set_gid, set_uid, set_mask; + int c, error, mntflags, set_gid, set_uid, set_mask, set_dirmask; char *dev, *dir, mntpath[MAXPATHLEN]; struct vfsconf vfc; - mntflags = set_gid = set_uid = set_mask = 0; + mntflags = set_gid = set_uid = set_mask = set_dirmask = 0; (void)memset(&args, '\0', sizeof(args)); args.magic = MSDOSFS_ARGSMAGIC; - while ((c = getopt(argc, argv, "sl9u:g:m:o:L:W:")) != -1) { + while ((c = getopt(argc, argv, "sl9u:g:m:M:o:L:W:")) != -1) { switch (c) { #ifdef MSDOSFSMNT_GEMDOSFS case 'G': @@ -124,6 +124,10 @@ args.mask = a_mask(optarg); set_mask = 1; break; + case 'M': + args.dirmask = a_mask(optarg); + set_dirmask = 1; + break; case 'L': load_ultable(&args, optarg); args.flags |= MSDOSFSMNT_ULTABLE; @@ -144,7 +148,16 @@ if (optind + 2 != argc) usage(); - + + if (set_mask && !set_dirmask) { + args.dirmask = args.mask; + set_dirmask = 1; + } + else if (set_dirmask && !set_mask) { + args.mask = args.dirmask; + set_mask = 1; + } + dev = argv[optind]; dir = argv[optind + 1]; @@ -170,7 +183,8 @@ if (!set_gid) args.gid = sb.st_gid; if (!set_mask) - args.mask = sb.st_mode & (S_IRWXU | S_IRWXG | S_IRWXO); + args.mask = args.dirmask = + sb.st_mode & (S_IRWXU | S_IRWXG | S_IRWXO); } error = getvfsbyname("msdos", &vfc); Index: sys/msdosfs/msdosfs_vfsops.c =================================================================== RCS file: /home/ncvs/src/sys/msdosfs/Attic/msdosfs_vfsops.c,v retrieving revision 1.60.2.5 diff -u -r1.60.2.5 msdosfs_vfsops.c --- sys/msdosfs/msdosfs_vfsops.c 4 Nov 2001 18:57:51 -0000 1.60.2.5 +++ sys/msdosfs/msdosfs_vfsops.c 19 Jan 2002 04:47:43 -0000 @@ -113,6 +113,7 @@ pmp->pm_gid = argp->gid; pmp->pm_uid = argp->uid; pmp->pm_mask = argp->mask & ALLPERMS; + pmp->pm_dirmask = argp->dirmask & ALLPERMS; pmp->pm_flags |= argp->flags & MSDOSFSMNT_MNTOPT; if (pmp->pm_flags & MSDOSFSMNT_U2WTABLE) { bcopy(argp->u2w, pmp->pm_u2w, sizeof(pmp->pm_u2w)); @@ -184,7 +185,7 @@ args.flags = 0; args.uid = 0; args.gid = 0; - args.mask = 0777; + args.mask = args.dirmask = 0777; if ((error = mountmsdosfs(rootvp, mp, p, &args)) != 0) { free(mp, M_MOUNT); Index: sys/msdosfs/msdosfs_vnops.c =================================================================== RCS file: /home/ncvs/src/sys/msdosfs/Attic/msdosfs_vnops.c,v retrieving revision 1.95.2.1 diff -u -r1.95.2.1 msdosfs_vnops.c --- sys/msdosfs/msdosfs_vnops.c 18 Jul 2000 13:19:13 -0000 1.95.2.1 +++ sys/msdosfs/msdosfs_vnops.c 19 Jan 2002 05:36:00 -0000 @@ -259,7 +259,7 @@ file_mode = (S_IXUSR|S_IXGRP|S_IXOTH) | (S_IRUSR|S_IRGRP|S_IROTH) | ((dep->de_Attributes & ATTR_READONLY) ? 0 : (S_IWUSR|S_IWGRP|S_IWOTH)); - file_mode &= pmp->pm_mask; + file_mode &= (vp->v_type == VDIR ? pmp->pm_dirmask : pmp->pm_mask); /* * Disallow write attempts on read-only file systems; @@ -358,7 +358,8 @@ mode = S_IRWXU|S_IRWXG|S_IRWXO; else mode = S_IRUSR|S_IXUSR|S_IRGRP|S_IXGRP|S_IROTH|S_IXOTH; - vap->va_mode = mode & pmp->pm_mask; + vap->va_mode = mode & + (ap->a_vp->v_type == VDIR ? pmp->pm_dirmask : pmp->pm_mask); vap->va_uid = pmp->pm_uid; vap->va_gid = pmp->pm_gid; vap->va_nlink = 1; Index: sys/msdosfs/msdosfsmount.h =================================================================== RCS file: /home/ncvs/src/sys/msdosfs/Attic/msdosfsmount.h,v retrieving revision 1.20.2.2 diff -u -r1.20.2.2 msdosfsmount.h --- sys/msdosfs/msdosfsmount.h 27 Oct 2000 09:45:07 -0000 1.20.2.2 +++ sys/msdosfs/msdosfsmount.h 19 Jan 2002 04:42:41 -0000 @@ -65,7 +65,10 @@ dev_t pm_dev; /* block special device mounted */ uid_t pm_uid; /* uid to set as owner of the files */ gid_t pm_gid; /* gid to set as owner of the files */ - mode_t pm_mask; /* mask to and with file protection bits */ + mode_t pm_mask; /* mask to and with file protection bits + for files */ + mode_t pm_dirmask; /* mask to and with file protection bits + for directories */ struct vnode *pm_devvp; /* vnode for block device mntd */ struct bpb50 pm_bpb; /* BIOS parameter blk for this fs */ u_long pm_BlkPerSec; /* How many DEV_BSIZE blocks fit inside a physical sector */ @@ -211,7 +214,8 @@ struct export_args export; /* network export information */ uid_t uid; /* uid that owns msdosfs files */ gid_t gid; /* gid that owns msdosfs files */ - mode_t mask; /* mask to be applied for msdosfs perms */ + mode_t mask; /* file mask to be applied for msdosfs perms */ + mode_t dirmask; /* dir mask to be applied for msdosfs perms */ int flags; /* see below */ int magic; /* version number */ u_int16_t u2w[128]; /* Local->Unicode table */ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 20:21:30 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 9A4AD37B417 for ; Sat, 19 Jan 2002 20:21:27 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id PAA16478; Sun, 20 Jan 2002 15:21:18 +1100 Date: Sun, 20 Jan 2002 15:22:43 +1100 (EST) From: Bruce Evans X-X-Sender: To: Alfred Perlstein Cc: Julian Elischer , Subject: Re: doreti() and userret() In-Reply-To: <20020119110428.Z13686@elvis.mu.org> Message-ID: <20020120150232.R6889-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 19 Jan 2002, Alfred Perlstein wrote: > * Julian Elischer [020119 10:01] wrote: > > > > On Sun, 20 Jan 2002, Bruce Evans wrote: > > > ... In my version of FreeBSD, userret() is > > > almost a no-op and almost all the things that were in userret() are in > > > ast(). > > > > So, Bruce, can you outline to use what is in BDEBSD that we can expect > > to see in the future? It soulds to em like it would be nice to get some > > more of your work in.. Only lots of optimizations (but not enough to compensate for pessimizations in -current) and cleanups. > I would too, Bruce please stop holding out on us. Sorry, I'm getting further behind committing things (2MB of diffs -c2 behind). The ast() changes were originally written by luoqi (http://www.freebsd.org/~luoqi/old/). I just provided ideas for parts of them originally and started keeping them up to date lately. Half of the infrastructure was already in -current since it was needed for SMP. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 22:20:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id D173F37B419 for ; Sat, 19 Jan 2002 22:20:09 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020120062009.SSG26243.rwcrmhc51.attbi.com@InterJet.elischer.org>; Sun, 20 Jan 2002 06:20:09 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id WAA04735; Sat, 19 Jan 2002 22:03:46 -0800 (PST) Date: Sat, 19 Jan 2002 22:03:45 -0800 (PST) From: Julian Elischer To: Bruce Evans Cc: Alfred Perlstein , arch@FreeBSD.ORG Subject: Re: doreti() and userret() In-Reply-To: <20020120150232.R6889-100000@gamplex.bde.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG yeah but we'd LOVE to see them.. can you at least make diffs available? I'm in userret() at the moment and I'd love to see what you would like to change. On Sun, 20 Jan 2002, Bruce Evans wrote: > On Sat, 19 Jan 2002, Alfred Perlstein wrote: > > > * Julian Elischer [020119 10:01] wrote: > > > > > > On Sun, 20 Jan 2002, Bruce Evans wrote: > > > > ... In my version of FreeBSD, userret() is > > > > almost a no-op and almost all the things that were in userret() are in > > > > ast(). > > > > > > So, Bruce, can you outline to use what is in BDEBSD that we can expect > > > to see in the future? It soulds to em like it would be nice to get some > > > more of your work in.. > > Only lots of optimizations (but not enough to compensate for pessimizations > in -current) and cleanups. > > > I would too, Bruce please stop holding out on us. > > Sorry, I'm getting further behind committing things (2MB of diffs -c2 > behind). The ast() changes were originally written by luoqi > (http://www.freebsd.org/~luoqi/old/). I just provided ideas for parts > of them originally and started keeping them up to date lately. Half of > the infrastructure was already in -current since it was needed for SMP. > > Bruce > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 19 23:30:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id D7B0537B402 for ; Sat, 19 Jan 2002 23:30:17 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id SAA25713; Sun, 20 Jan 2002 18:30:05 +1100 Date: Sun, 20 Jan 2002 18:31:31 +1100 (EST) From: Bruce Evans X-X-Sender: To: Julian Elischer Cc: Alfred Perlstein , Subject: Re: doreti() and userret() In-Reply-To: Message-ID: <20020120182437.R7452-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 19 Jan 2002, Julian Elischer wrote: > yeah but we'd LOVE to see them.. > can you at least make diffs available? > > I'm in userret() at the moment and I'd love to see what you would like to > change. See sys.dif.gz in ~bde on freefall. For optimizing userret(), the idea is to set flags for ast() to check. E.g., there is a flag for signals so that usrret() doesn't need the CURSIG() loop. CURSIG() is much more expensive than it used to be, since it has to check 128 signals instead of 32 and aquire and release 2 locksE.g., there is a flag for signals so that usrret() doesn't need the CURSIG() loop. CURSIG() is much more expensive than it used to be, since it has to check 128 signals instead of 32 and aquire and release 2 locks instead of none. > On Sun, 20 Jan 2002, Bruce Evans wrote: > > > On Sat, 19 Jan 2002, Alfred Perlstein wrote: > > > > > * Julian Elischer [020119 10:01] wrote: > > > > > > > > On Sun, 20 Jan 2002, Bruce Evans wrote: Please don't top post (top mail?). Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message