From owner-freebsd-hackers Sun Sep 23 0:19:22 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from root.com (root.com [209.102.106.178]) by hub.freebsd.org (Postfix) with ESMTP id E045A37B401 for ; Sun, 23 Sep 2001 00:19:19 -0700 (PDT) Received: (from dg@localhost) by root.com (8.11.2/8.11.2) id f8N6v4T48857; Sat, 22 Sep 2001 23:57:04 -0700 (PDT) (envelope-from dg) Date: Sat, 22 Sep 2001 23:57:04 -0700 From: David Greenman To: Matt Dillon Cc: Poul-Henning Kamp , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine Message-ID: <20010922235704.D28469@nexus.root.com> References: <88901.1001191415@critter> <200109222121.f8MLLHe82202@earth.backplane.com> <20010922141934.C28469@nexus.root.com> <200109230642.f8N6gPj84955@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200109230642.f8N6gPj84955@earth.backplane.com>; from dillon@earth.backplane.com on Sat, Sep 22, 2001 at 11:42:25PM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > Well, this has turned into a rather sticky little problem. I've > spent all day going through the vnode/name-cache reclaim code, looking > both at Seigo's cache_purgeleafdirs() and my own patch. Can you forward me your patch? I'd like to try it out on some machines in the TSI lab. > The bigger problem is exactly as DG has stated... it isn't the namei > cache that is our enemy, it's the VM Page cache preventing vnodes > from being recycled. > > For the moment I believe that cache_purgeleafdirs() or my patch solves > the problem well enough that we can run with it for a while. The real > solution, I believe, is to give us the ability to take cached VM Pages > associated with a file and rename them to cached VM Pages associated > with the filesystem device - we can do this only for page-aligned > blocks of course, not fragments (which we would simply throw away)... > but it would allow us to reclaim vnodes independant of the VM Page cache > without losing the cached pages. I think this is doable but it will > require a considerable amount of work. It isn't something I can do in a > day. I also believe that this can dovetail quite nicely into the I/O > model that we have slowly been moving towards over the last year > (Poul's work). Inevitably we will have to manage device-based I/O > on a page-by-page basis and being able to do it via a VM Object seems > to fit the bill in my opinion. Hmmm. This would seem to be a step back to the days when caching was done relative to the device as opposed to the file-relative scheme we have now. One of the problems with the old scheme as I recall is that some filesystems like NFS don't have a 'device' and thus no physical block numbers to associate the cached pages with. There is also some cost in moving the pages between the file object and the device object. For these reasons, I would prefer that we keep the existing model, but just make sure that we can handle the degenerate case of one page per file object. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 0:46:34 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id CA91F37B420 for ; Sun, 23 Sep 2001 00:46:28 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8N7kMM85221; Sun, 23 Sep 2001 00:46:22 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 00:46:22 -0700 (PDT) From: Matt Dillon Message-Id: <200109230746.f8N7kMM85221@earth.backplane.com> To: David Greenman Cc: Poul-Henning Kamp , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <88901.1001191415@critter> <200109222121.f8MLLHe82202@earth.backplane.com> <20010922141934.C28469@nexus.root.com> <200109230642.f8N6gPj84955@earth.backplane.com> <20010922235704.D28469@nexus.root.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :> Well, this has turned into a rather sticky little problem. I've :> spent all day going through the vnode/name-cache reclaim code, looking :> both at Seigo's cache_purgeleafdirs() and my own patch. : : Can you forward me your patch? I'd like to try it out on some machines in :the TSI lab. Absolutely, any and all testing is good. Here's the patch for stable and current. Stable first: Index: kern/vfs_subr.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_subr.c,v retrieving revision 1.249.2.11 diff -u -r1.249.2.11 vfs_subr.c --- kern/vfs_subr.c 2001/09/11 09:49:53 1.249.2.11 +++ kern/vfs_subr.c 2001/09/23 07:33:51 @@ -506,10 +506,12 @@ TAILQ_REMOVE(&vnode_free_list, vp, v_freelist); TAILQ_INSERT_TAIL(&vnode_tmp_list, vp, v_freelist); continue; +#if 0 } else if (LIST_FIRST(&vp->v_cache_src)) { /* Don't recycle if active in the namecache */ simple_unlock(&vp->v_interlock); continue; +#endif } else { break; } And here is the patch for current: Index: kern/vfs_cache.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_cache.c,v retrieving revision 1.61 diff -u -r1.61 vfs_cache.c --- kern/vfs_cache.c 2001/09/12 08:37:46 1.61 +++ kern/vfs_cache.c 2001/09/23 07:27:05 @@ -101,8 +101,10 @@ SYSCTL_ULONG(_debug, OID_AUTO, numcache, CTLFLAG_RD, &numcache, 0, ""); static u_long numcachehv; /* number of cache entries with vnodes held */ SYSCTL_ULONG(_debug, OID_AUTO, numcachehv, CTLFLAG_RD, &numcachehv, 0, ""); +#if 0 static u_long numcachepl; /* number of cache purge for leaf entries */ SYSCTL_ULONG(_debug, OID_AUTO, numcachepl, CTLFLAG_RD, &numcachepl, 0, ""); +#endif struct nchstats nchstats; /* cache effectiveness statistics */ static int doingcache = 1; /* 1 => enable the cache */ @@ -499,6 +501,8 @@ } } +#if 0 + /* * Flush all dirctory entries with no child directories held in * the cache. @@ -554,6 +558,8 @@ } numcachepl++; } + +#endif /* * Perform canonical checks and cache lookup and pass on to filesystem Index: kern/vfs_subr.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_subr.c,v retrieving revision 1.319 diff -u -r1.319 vfs_subr.c --- kern/vfs_subr.c 2001/09/12 08:37:47 1.319 +++ kern/vfs_subr.c 2001/09/22 20:15:11 @@ -110,6 +110,8 @@ /* Number of vnodes in the free list. */ static u_long freevnodes = 0; SYSCTL_LONG(_debug, OID_AUTO, freevnodes, CTLFLAG_RD, &freevnodes, 0, ""); + +#if 0 /* Number of vnode allocation. */ static u_long vnodeallocs = 0; SYSCTL_LONG(_debug, OID_AUTO, vnodeallocs, CTLFLAG_RD, &vnodeallocs, 0, ""); @@ -125,6 +127,7 @@ /* Number of vnodes attempted to recycle at a time. */ static u_long vnoderecyclenumber = 3000; SYSCTL_LONG(_debug, OID_AUTO, vnoderecyclenumber, CTLFLAG_RW, &vnoderecyclenumber, 0, ""); +#endif /* * Various variables used for debugging the new implementation of @@ -556,8 +559,13 @@ * Don't recycle if active in the namecache or * if it still has cached pages or we cannot get * its interlock. + * + * XXX the namei cache can hold onto vnodes too long, + * causing us to run out of MALLOC space. Instead, we + * should make path lookups requeue any vnodes on the free + * list. */ - if (LIST_FIRST(&vp->v_cache_src) != NULL || + if (/* LIST_FIRST(&vp->v_cache_src) != NULL || */ (VOP_GETVOBJECT(vp, &object) == 0 && (object->resident_page_count || object->ref_count)) || !mtx_trylock(&vp->v_interlock)) { @@ -636,6 +644,7 @@ vfs_object_create(vp, td, td->td_proc->p_ucred); +#if 0 vnodeallocs++; if (vnodeallocs % vnoderecycleperiod == 0 && freevnodes < vnoderecycleminfreevn && @@ -643,6 +652,7 @@ /* Recycle vnodes. */ cache_purgeleafdirs(vnoderecyclenumber); } +#endif return (0); } Index: sys/vnode.h =================================================================== RCS file: /home/ncvs/src/sys/sys/vnode.h,v retrieving revision 1.157 diff -u -r1.157 vnode.h --- sys/vnode.h 2001/09/13 22:52:42 1.157 +++ sys/vnode.h 2001/09/23 07:26:54 @@ -559,7 +559,6 @@ struct componentname *cnp)); void cache_purge __P((struct vnode *vp)); void cache_purgevfs __P((struct mount *mp)); -void cache_purgeleafdirs __P((int ndir)); void cvtstat __P((struct stat *st, struct ostat *ost)); void cvtnstat __P((struct stat *sb, struct nstat *nsb)); int getnewvnode __P((enum vtagtype tag, To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 0:50:29 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id E8E4737B401 for ; Sun, 23 Sep 2001 00:50:26 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8N7oLI85253; Sun, 23 Sep 2001 00:50:21 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 00:50:21 -0700 (PDT) From: Matt Dillon Message-Id: <200109230750.f8N7oLI85253@earth.backplane.com> To: David Greenman Cc: Poul-Henning Kamp , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <88901.1001191415@critter> <200109222121.f8MLLHe82202@earth.backplane.com> <20010922141934.C28469@nexus.root.com> <200109230642.f8N6gPj84955@earth.backplane.com> <20010922235704.D28469@nexus.root.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : Hmmm. This would seem to be a step back to the days when caching was done :relative to the device as opposed to the file-relative scheme we have now. :One of the problems with the old scheme as I recall is that some filesystems :like NFS don't have a 'device' and thus no physical block numbers to :associate the cached pages with. There is also some cost in moving the pages :between the file object and the device object. For these reasons, I would :prefer that we keep the existing model, but just make sure that we can :handle the degenerate case of one page per file object. : :-DG : :David Greenman Yah. For NFS we'd have to just throw the pages away. I'm not changing anything any time soon, I've got lots of other interesting stuff on my plate. It's just an idea. The filesystem already has a VM Object that it uses for inodes and bitmap blocks and such, this would simply be a way to leverage it. Ultimately it may not matter... perhaps we can devise a way of hanging onto the inode/vnode reference without eating up so much KVM. For example, the VM Object could be associated with the filesystem inode cache at the filesystem level. Or something like that. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 2:35:34 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id E86D037B417 for ; Sun, 23 Sep 2001 02:35:30 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.4/8.11.4) with ESMTP id f8N9Y1v96471; Sun, 23 Sep 2001 11:34:02 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Matt Dillon Cc: David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine In-Reply-To: Your message of "Sat, 22 Sep 2001 23:42:25 PDT." <200109230642.f8N6gPj84955@earth.backplane.com> Date: Sun, 23 Sep 2001 11:34:01 +0200 Message-ID: <96469.1001237641@critter> From: Poul-Henning Kamp Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200109230642.f8N6gPj84955@earth.backplane.com>, Matt Dillon writes: > My patch doesn't make a distinction but assumes that (A) will tend to > hold for higher level directories: that is, that higher level directories > tend to be accessed more often and thus will tend to have pages in the > VM Page Cache, and thus not be candidates for reuse anyway. So my patch > has a very similar effect but without the overhead. Back when I rewrote the VFS namecache back in 1997 I added that clause because I saw directories getting nuked in no time because there were no pages holding on to them (device nodes were even worse!) So refresh my memory here, does directories get pages cached in VM if you have vfs.vmiodirenable=0 ? What about !UFS filesystems ? Do they show a performance difference ? Also, don't forget that if the VM system gave preferential caching to directory pages, we wouldn't need the VFS-cache very much in the first place... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 3:40:49 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id D790C37B41E for ; Sun, 23 Sep 2001 03:40:41 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8NAeXw86352; Sun, 23 Sep 2001 03:40:33 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 03:40:33 -0700 (PDT) From: Matt Dillon Message-Id: <200109231040.f8NAeXw86352@earth.backplane.com> To: Poul-Henning Kamp Cc: David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <96469.1001237641@critter> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :> VM Page Cache, and thus not be candidates for reuse anyway. So my patch :> has a very similar effect but without the overhead. : :Back when I rewrote the VFS namecache back in 1997 I added that :clause because I saw directories getting nuked in no time because :there were no pages holding on to them (device nodes were even worse!) : :So refresh my memory here, does directories get pages cached in VM if :you have vfs.vmiodirenable=0 ? : :What about !UFS filesystems ? Do they show a performance difference ? : :Also, don't forget that if the VM system gave preferential caching to :directory pages, we wouldn't need the VFS-cache very much in the first :place... : :-- :Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 Ah yes, vmiodirenable. We should just turn it on by default now. I've been waffling too long on that. With it off the buffer cache will remember at most vfs.maxmallocspace worth of directory data (read: not very much), and without VMIO backing, which means vnodes could be reclaimed immediately. Ah! Now I see why that clause was put in... but it's obsolete now if vmiodirenable is turned on, and it doesn't scale well to large-memory machines if it is left in. If we turn vmiodirenable on then directory blocks get cached by the VM system. There is no preferential treatment of directory blocks but there doesn't need to be, the VM system does a very good job figuring out which blocks to keep and which not to. vfs.vmiodirenable=0 works well for small lightly loaded systems but doesn't scale at all. vfs.vmiodirenable=1 works well for any sized system, even though there's considerable storage ineffeciencies with small directories, because the VM Page algorithms compensate (and scale). Small systems with fewer directories don't see the vnode scaling problem because there are simply not enough directories to saturate the vnode/inode malloc areas. Large systems with a greater number of directories blow up the vnode/inode malloc space. I'll run some buildworld tests tomorrow. Er, later today. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 3:47:10 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from smtp8.xs4all.nl (smtp8.xs4all.nl [194.109.127.134]) by hub.freebsd.org (Postfix) with ESMTP id 5735E37B42A for ; Sun, 23 Sep 2001 03:47:04 -0700 (PDT) Received: from freebie.xs4all.nl (freebie.xs4all.nl [213.84.32.253]) by smtp8.xs4all.nl (8.9.3/8.9.3) with ESMTP id MAA13453; Sun, 23 Sep 2001 12:47:02 +0200 (CEST) Received: (from wkb@localhost) by freebie.xs4all.nl (8.11.6/8.11.4) id f8NAl2M09958; Sun, 23 Sep 2001 12:47:02 +0200 (CEST) (envelope-from wkb) Date: Sun, 23 Sep 2001 12:47:02 +0200 From: Wilko Bulte To: Matt Dillon Cc: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine Message-ID: <20010923124702.B9914@freebie.xs4all.nl> References: <96469.1001237641@critter> <200109231040.f8NAeXw86352@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109231040.f8NAeXw86352@earth.backplane.com>; from dillon@earth.backplane.com on Sun, Sep 23, 2001 at 03:40:33AM -0700 X-OS: FreeBSD 4.4-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Sep 23, 2001 at 03:40:33AM -0700, Matt Dillon wrote: > > :> VM Page Cache, and thus not be candidates for reuse anyway. So my patch > :> has a very similar effect but without the overhead. > : > :Back when I rewrote the VFS namecache back in 1997 I added that > :clause because I saw directories getting nuked in no time because > :there were no pages holding on to them (device nodes were even worse!) > : > :So refresh my memory here, does directories get pages cached in VM if > :you have vfs.vmiodirenable=0 ? > : > :What about !UFS filesystems ? Do they show a performance difference ? > : > :Also, don't forget that if the VM system gave preferential caching to > :directory pages, we wouldn't need the VFS-cache very much in the first > :place... > : > :-- > :Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > > Ah yes, vmiodirenable. We should just turn it on by default now. I've Has the problem of small-memory machines (< 64M IIRC) solved now? As I understand it vmiodirenable is counter-productive for these boxes. Maybe one could decide on-boot whether the amount of mem is enough to make it useful? Just a thought of course. > been waffling too long on that. With it off the buffer cache will > remember at most vfs.maxmallocspace worth of directory data (read: not > very much), and without VMIO backing, which means vnodes could be > reclaimed immediately. Ah! Now I see why that clause was put > in... but it's obsolete now if vmiodirenable is turned on, and it > doesn't scale well to large-memory machines if it is left in. -- | / o / /_ _ email: wilko@FreeBSD.org |/|/ / / /( (_) Bulte Arnhem, The Netherlands To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 6: 6:55 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 4E5FC37B428 for ; Sun, 23 Sep 2001 06:06:50 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.4/8.11.4) with ESMTP id f8ND6bv98333; Sun, 23 Sep 2001 15:06:37 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Matt Dillon Cc: David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine In-Reply-To: Your message of "Sun, 23 Sep 2001 03:40:33 PDT." <200109231040.f8NAeXw86352@earth.backplane.com> Date: Sun, 23 Sep 2001 15:06:36 +0200 Message-ID: <98331.1001250396@critter> From: Poul-Henning Kamp Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200109231040.f8NAeXw86352@earth.backplane.com>, Matt Dillon writes: > Ah yes, vmiodirenable. We should just turn it on by default now. I've > been waffling too long on that. With it off the buffer cache will > remember at most vfs.maxmallocspace worth of directory data (read: not > very much), and without VMIO backing, which means vnodes could be > reclaimed immediately. Ah! Now I see why that clause was put > in... but it's obsolete now if vmiodirenable is turned on, and it > doesn't scale well to large-memory machines if it is left in. > > If we turn vmiodirenable on then directory blocks get cached by the > VM system. There is no preferential treatment of directory blocks > but there doesn't need to be, the VM system does a very good job figuring > out which blocks to keep and which not to. Well, benchmarks will show that. A directory page may hold the access to hundred files so it should not be pushed out by one file. I think getting directory pages VM cached is The Right Way for this, because it means that also the not-earlier used filenames become cheaply accessible. Things to look out for: 1. !ufs filesystems 2. inode->vnode hash/search algorithms may become much more important. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 7: 5:38 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from smtp.noos.fr (descartes.noos.net [212.198.2.74]) by hub.freebsd.org (Postfix) with ESMTP id DC54F37B410 for ; Sun, 23 Sep 2001 07:05:33 -0700 (PDT) Received: (qmail 14292239 invoked by uid 0); 23 Sep 2001 14:05:33 -0000 Received: from unknown (HELO gits.dyndns.org) ([212.198.231.187]) (envelope-sender ) by 212.198.2.74 (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 23 Sep 2001 14:05:33 -0000 Received: (from root@localhost) by gits.dyndns.org (8.11.6/8.11.6) id f8NE5Rf28302; Sun, 23 Sep 2001 16:05:27 +0200 (CEST) (envelope-from root) Message-Id: <200109231405.f8NE5Rf28302@gits.dyndns.org> Subject: Re: Version of XFree86 in FreeBSD Release 4.4 In-Reply-To: <20010918145722.A97809@dragon.nuxi.com> To: obrien@freebsd.org Date: Sun, 23 Sep 2001 16:05:27 +0200 (CEST) Cc: Jordan Hubbard , glenngombert@zdnetonebox.com, freebsd-hackers@freebsd.org Reply-To: clefevre@citeweb.net From: Cyrille Lefevre Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[<97Zd*>^#%Y5Cxv;%Y[PT-LW3;A:fRrJ8+^k"e7@+30g0YD0*^^3jgyShN7o?a]C la*Zv'5NA,=963bM%J^o]C X-Mailer: ELM [version 2.4ME+ PL94b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG David O'Brien wrote: > On Mon, Sep 17, 2001 at 05:42:23PM -0700, Jordan Hubbard wrote: > > We're still waiting for 4.0's "support footprint" to widen > > a bit more before subjecting people to it by default. Hopefully > > by 4.5. > > Are you really considering using XFree86 4.x in FreeBSD-4.5? > When I asked you about this in the past, you had said you wanted to keep > the same X in RELENG_4 (presumable to not "rock the boat" mid-branch). isn't it possible to provide both versions and left the user to choice between both w/ a information box relating problems found in the one or the other ? Cyrille. -- Cyrille Lefevre mailto:clefevre@citeweb.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 8:59:49 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from cs.bu.edu (CS.BU.EDU [128.197.10.2]) by hub.freebsd.org (Postfix) with ESMTP id E50DC37B417 for ; Sun, 23 Sep 2001 08:59:45 -0700 (PDT) Received: from csa.bu.edu (evms@csa [128.197.12.3]) by cs.bu.edu (8.10.1/8.10.1) with ESMTP id f8NFxiA07531 for ; Sun, 23 Sep 2001 11:59:44 -0400 (EDT) Received: (from evms@localhost) by csa.bu.edu (8.10.1/8.10.1) id f8NFxeC13760; Sun, 23 Sep 2001 11:59:40 -0400 (EDT) Date: Sun, 23 Sep 2001 11:59:40 -0400 (EDT) Message-Id: <200109231559.f8NFxeC13760@csa.bu.edu> From: Evan Sarmiento To: freebsd-hackers@freebsd.org Subject: panic on mount Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, After compiling a new kernel, installing it, when my laptop tries to mount its drive, it panics with this message: panic: lock (sleep mutex) vnode interlock not locked @ ../../../kern/vfs_default.c:460 which is: if (ap->a_flags & LK_INTERLOCK) mtx_unlock(&ap->a_vp->v_interlock); within the function vop_nolock. Thanks, Evan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 9:43:34 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from hokkshideh.jetcafe.org (hokkshideh.jetcafe.org [205.147.43.4]) by hub.freebsd.org (Postfix) with ESMTP id 7738237B438 for ; Sun, 23 Sep 2001 09:43:26 -0700 (PDT) Received: from hokkshideh.jetcafe.org (localhost [127.0.0.1]) by hokkshideh.jetcafe.org (8.8.8/8.8.5) with ESMTP id JAA09454 for ; Sun, 23 Sep 2001 09:43:25 -0700 (PDT) Message-Id: <200109231643.JAA09454@hokkshideh.jetcafe.org> X-Mailer: exmh version 2.2 06/23/2000 with version: MH 6.8.4 #1[UCI] To: freebsd-hackers@freebsd.org Subject: Problems with many ATA drives Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 23 Sep 2001 09:43:25 -0700 From: Dave Hayes Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG We've been attempting to set up a vinum raid box with a bunch of IDE drives. Each drive is partitioned with a vinum partition on A, such that the entire drive is on partition a. Initial partitioning is done with /stand/sysinstall so it "fixes" my geometry, this has always worked in the past. I had been getting "funny" stuff from the drives, so I devised the following simple test: # dd if=/dev/rad1a of=/dev/null This eventually produces: ad1: READ command timeout tag=0 serv=0 - resetting ata0: resetting devices .. done ad1a: hard error reading fsbn 5068879 (ad1 bn 5068879; cn 315 tn 133 sn 25)ad1a: hard error reading fsbn 5068879 (ad1 bn 5068879; cn 315 tn 133 sn 25) status=59 error=40 I notice 3 out of 11 drives produce this error, so far one on each controller (ruling out a specific controller issue). I didn't want to just assume the failure rate of 80GB IDE drives is that large, so I'm asking this list for it's opinion: a) Is this a bug or consequence of software drivers? (see bug kern/17592) b) Or is it just that IDE drives are cheap and fail this much? Relevant data from dmesg: atapci0: port 0xb000-0xb00f,0xb400-0xb403,0xb800-0x b807,0xd000-0xd003,0xd400-0xd407 mem 0xf5800000-0xf5803fff irq 6 at device 10.0 on pci2 ata2: at 0xd400 on atapci0 ata3: at 0xb800 on atapci0 atapci1: port 0x9400-0x940f,0x9800-0x9803,0xa000-0x a007,0xa400-0xa403,0xa800-0xa807 mem 0xf5000000-0xf5003fff irq 9 at device 11.0 on pci2 ata4: at 0xa800 on atapci1 ata5: at 0xa000 on atapci1 ... atapci2: port 0x8800-0x880f at device 31.1 on pci0 ata0: at 0x1f0 irq 14 on atapci2 ata1: at 0x170 irq 15 on atapci2 ... ad0: 78167MB [158816/16/63] at ata0-master UDMA100 ad1: 78167MB [158816/16/63] at ata0-slave UDMA100 ad2: 78167MB [158816/16/63] at ata1-master UDMA100 ad3: 78167MB [158816/16/63] at ata1-slave UDMA100 ad4: 78167MB [158816/16/63] at ata2-master WDMA2 ad5: 78167MB [158816/16/63] at ata2-slave WDMA2 ad6: 78167MB [158816/16/63] at ata3-master WDMA2 ad7: 78167MB [158816/16/63] at ata3-slave WDMA2 ad8: 78167MB [158816/16/63] at ata4-master WDMA2 ad9: 78167MB [158816/16/63] at ata4-slave WDMA2 Yes, we know that the "WDMA2" is happening, this state proved to be independant of a drive failing. It has to do with 10 drives in a tower and cable lengths... =( ------ Dave Hayes - Consultant - Altadena CA, USA - dave@jetcafe.org >>> The opinions expressed above are entirely my own <<< There is no distinctly native American criminal class except Congress. -- Mark Twain To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 10:13:12 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by hub.freebsd.org (Postfix) with ESMTP id 458BA37B41B; Sun, 23 Sep 2001 10:13:07 -0700 (PDT) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.6/8.11.1) id f8NHCiV10264; Sun, 23 Sep 2001 10:12:44 -0700 (PDT) (envelope-from obrien) Date: Sun, 23 Sep 2001 10:12:44 -0700 From: "David O'Brien" To: Cyrille Lefevre Cc: Jordan Hubbard , glenngombert@zdnetonebox.com, freebsd-hackers@freebsd.org Subject: Re: Version of XFree86 in FreeBSD Release 4.4 Message-ID: <20010923101244.A9616@dragon.nuxi.com> Reply-To: obrien@freebsd.org References: <20010918145722.A97809@dragon.nuxi.com> <200109231405.f8NE5Rf28302@gits.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109231405.f8NE5Rf28302@gits.dyndns.org>; from clefevre@citeweb.net on Sun, Sep 23, 2001 at 04:05:27PM +0200 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Sep 23, 2001 at 04:05:27PM +0200, Cyrille Lefevre wrote: > David O'Brien wrote: > > On Mon, Sep 17, 2001 at 05:42:23PM -0700, Jordan Hubbard wrote: > > > We're still waiting for 4.0's "support footprint" to widen > > > a bit more before subjecting people to it by default. Hopefully > > > by 4.5. > > > > Are you really considering using XFree86 4.x in FreeBSD-4.5? > > When I asked you about this in the past, you had said you wanted to keep > > the same X in RELENG_4 (presumable to not "rock the boat" mid-branch). > > isn't it possible to provide both versions and left the user to > choice between both w/ a information box relating problems found > in the one or the other ? There are issues for the pre-compiled packages due to differences between the two versions of XFree86. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 12:19:46 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 25C3737B421 for ; Sun, 23 Sep 2001 12:19:43 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.4/8.11.4) with ESMTP id f8NJJUv01622; Sun, 23 Sep 2001 21:19:30 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Matt Dillon Cc: David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine In-Reply-To: Your message of "Sat, 22 Sep 2001 23:42:25 PDT." <200109230642.f8N6gPj84955@earth.backplane.com> Date: Sun, 23 Sep 2001 21:19:30 +0200 Message-ID: <1620.1001272770@critter> From: Poul-Henning Kamp Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I ran one of my trivial benchmarks here: "make -j 12 buildworld" on a dual 866MHz P3 with 640M RAM. Most of the stuff reported by "/usr/bin/time -l" here is useless: almost all the numbers are all inside the standard deviation I have recorded for them on this box. Block input operations is the one notable exception and it tells a very interesting story: Matts patch results in a 4% increase, but combined with vmdirioenable it results in a 21.5% decrease. That's pretty darn significant: one out of every five I/O have been saved. The reason it has not manifested itself in the "real" number is probably the high degree of parallelism in the task which practically ensures that the CPU will not go idle. I suggest we let Matt's patch depend on the vmiodirenable sysctl and change the default for that. If there are no bad side effects found in the next couple of months, then kill the sysctl and lets be done with it. Poul-Henning -current | Matts patch | Matts patch | | vmiodirenable=1 -------------+-------------+------------------------------------------- 2206.27| 2277.97| 2190.35|real 2118.62| 2119.07| 2128.68|user 828.77| 825.03| 835.24|system 15220 | 15220 | 15216 |maximum resident set size 1006 | 1006 | 1005 |average shared memory size 1036 | 1035 | 1034 |average unshared data size 128 | 128 | 128 |average unshared stack size 10800112 | 10812895 | 10799144 |page reclaims 2530 | 3002 | 2686 |page faults 0 | 0 | 0 |swaps 36971 | 38462 | 29030 |block input operations 6694 | 6637 | 6610 |block output operations 0 | 0 | 0 |messages sent 0 | 0 | 0 |messages received 6 | 6 | 6 |signals received 17857329 | 17161716 | 17914470 |voluntary context switches 843181 | 898530 | 778408 |involuntary context switches -------------+-------------+------------------------------------------- -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 13:19:59 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id AFBD937B439 for ; Sun, 23 Sep 2001 13:19:53 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8NKJZL88354; Sun, 23 Sep 2001 13:19:35 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 13:19:35 -0700 (PDT) From: Matt Dillon Message-Id: <200109232019.f8NKJZL88354@earth.backplane.com> To: Poul-Henning Kamp Cc: David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <1620.1001272770@critter> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Block input operations is the one notable exception and it tells a :very interesting story: Matts patch results in a 4% increase, but :combined with vmdirioenable it results in a 21.5% decrease. : :That's pretty darn significant: one out of every five I/O have :been saved. : :The reason it has not manifested itself in the "real" number is :probably the high degree of parallelism in the task which practically :ensures that the CPU will not go idle. : :I suggest we let Matt's patch depend on the vmiodirenable sysctl :and change the default for that. : :If there are no bad side effects found in the next couple of months, :then kill the sysctl and lets be done with it. : :Poul-Henning Very interesting! I agree completely. I will instrument the code to make the namei cache check conditional on vmiodirenable. I will also add a sysctl to change the check condition to test for leaf directories or not (closer to what Seigo's code accomplished) so we can test for differences there. If we can get that 4% down the code will be more acceptable. The results make a lot of sense, especially if the machine has sufficient memory to cache most of the source tree. Increasing the number of vnode reclaims will increase I/O on directories, especially for something like a buildworld which is doing major path expansions up the wazoo. Turning on vmiodirenable in a buildworld situation will decrease I/O significantly. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 14: 9:58 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 9287C37B41E for ; Sun, 23 Sep 2001 14:09:54 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8NL9kU88657; Sun, 23 Sep 2001 14:09:46 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 14:09:46 -0700 (PDT) From: Matt Dillon Message-Id: <200109232109.f8NL9kU88657@earth.backplane.com> To: Wilko Bulte Cc: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <96469.1001237641@critter> <200109231040.f8NAeXw86352@earth.backplane.com> <20010923124702.B9914@freebie.xs4all.nl> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Has the problem of small-memory machines (< 64M IIRC) solved now? As I :understand it vmiodirenable is counter-productive for these boxes. :Maybe one could decide on-boot whether the amount of mem is enough to :make it useful? : :Just a thought of course. : :| / o / /_ _ email: wilko@FreeBSD.org Small memory machines never had a problem. Even though there can be considerable memory inefficiencies using vmiodirenable (e.g. a directory less then 512 bytes eats 512 bytes of physical ram with vmiodirenable turned off, and 4K with it turned on), there are two compensating factors: First, the VM Paging cache is a cache, so the cached directory blocks can be thrown away. Second, the VM Page cache has all of memory to play with while the buffer cache with vmiodirenable turned off on a 64MB machine will reserve less then a megabyte to cache directories. #2 is important because it leads to more I/O which has a far greater effect on the system then memory waste. Also, if you look at the typical program's memory footprint and assume that actively accessed directories eat 4K, the memory used to hold the active directories winds up being almost nothing compared to the RSS of the program using those directories. Single-use directories (e.g. make buildworld) are recycled in the VM Page cache and do not eat as much memory as you might otherwise think. So even though the memory inefficiency can be up to 8:1 (4096/512), this factor is somewhat deceptive in regards to the actual effect on the system. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 14:13: 9 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from freebsd.dk (fw-rl0.freebsd.dk [212.242.86.114]) by hub.freebsd.org (Postfix) with ESMTP id 0B71137B435 for ; Sun, 23 Sep 2001 14:13:00 -0700 (PDT) Received: (from sos@localhost) by freebsd.dk (8.11.3/8.11.3) id f8NLCsE42136; Sun, 23 Sep 2001 23:12:54 +0200 (CEST) (envelope-from sos) From: Søren Schmidt Message-Id: <200109232112.f8NLCsE42136@freebsd.dk> Subject: Re: Problems with many ATA drives In-Reply-To: <200109231643.JAA09454@hokkshideh.jetcafe.org> "from Dave Hayes at Sep 23, 2001 09:43:25 am" To: Dave Hayes Date: Sun, 23 Sep 2001 23:12:54 +0200 (CEST) Cc: freebsd-hackers@FreeBSD.ORG Reply-To: sos@freebsd.dk X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG It seems Dave Hayes wrote: > > ad1: READ command timeout tag=0 serv=0 - resetting > ata0: resetting devices .. done > ad1a: hard error reading fsbn 5068879 (ad1 bn 5068879; cn 315 tn 133 sn > 25)ad1a: hard error reading fsbn 5068879 (ad1 bn 5068879; cn 315 tn 133 sn 25) > status=59 error=40 > > I notice 3 out of 11 drives produce this error, so far one on each > controller (ruling out a specific controller issue). I didn't want to > just assume the failure rate of 80GB IDE drives is that large, so > I'm asking this list for it's opinion: > > a) Is this a bug or consequence of software drivers? (see > bug kern/17592) > > b) Or is it just that IDE drives are cheap and fail this much? > > Relevant data from dmesg: > > atapci0: port 0xb000-0xb00f,0xb400-0xb403,0xb800-0x > b807,0xd000-0xd003,0xd400-0xd407 mem 0xf5800000-0xf5803fff irq 6 at device > 10.0 on pci2 > ata2: at 0xd400 on atapci0 > ata3: at 0xb800 on atapci0 > atapci1: port 0x9400-0x940f,0x9800-0x9803,0xa000-0x > a007,0xa400-0xa403,0xa800-0xa807 mem 0xf5000000-0xf5003fff irq 9 at device > 11.0 on pci2 > ata4: at 0xa800 on atapci1 > ata5: at 0xa000 on atapci1 > ... > atapci2: port 0x8800-0x880f at device 31.1 on > pci0 > ata0: at 0x1f0 irq 14 on atapci2 > ata1: at 0x170 irq 15 on atapci2 > ... > ad0: 78167MB [158816/16/63] at ata0-master UDMA100 > ad1: 78167MB [158816/16/63] at ata0-slave UDMA100 > ad2: 78167MB [158816/16/63] at ata1-master UDMA100 > ad3: 78167MB [158816/16/63] at ata1-slave UDMA100 > ad4: 78167MB [158816/16/63] at ata2-master WDMA2 > ad5: 78167MB [158816/16/63] at ata2-slave WDMA2 > ad6: 78167MB [158816/16/63] at ata3-master WDMA2 > ad7: 78167MB [158816/16/63] at ata3-slave WDMA2 > ad8: 78167MB [158816/16/63] at ata4-master WDMA2 > ad9: 78167MB [158816/16/63] at ata4-slave WDMA2 > > Yes, we know that the "WDMA2" is happening, this state proved to be > independant of a drive failing. It has to do with 10 drives in a tower > and cable lengths... =( Hmm, first of the error above looks very much to be a genuine media error on the disks, are the bad spot always the same or random ? Anyhow does the 3 bad ones produce the error regardless of what controller they are put on? I assume that its always the same 3 drives that are failing right ? Oh, and you should take cable length seriously, remember you only get ICRC errors (which the ATA driver retries) on UDMA33 and above, at WDMA2 speed there is *NO* CRC check at all (the HW doesn't support that), so you wont know when your data has been currupted :) So thinking that you solved the problem by going to WDMA2 mode is extremly dangerous, you are just hiding the problem as data corruption will very likely still happen when you use off-spec cableing. -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 14:28:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 1716D37B428 for ; Sun, 23 Sep 2001 14:28:05 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8NLRuW88761; Sun, 23 Sep 2001 14:27:56 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 14:27:56 -0700 (PDT) From: Matt Dillon Message-Id: <200109232127.f8NLRuW88761@earth.backplane.com> To: Poul-Henning Kamp Cc: David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: cache purge cache for -stable (Was Re: Conclusions on... cache_purgeleafdirs()) References: <1620.1001272770@critter> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Here is the latest patch for -stable. vmiodirenable is turned on by default, the cache purge code is enabled based on vmiodirenable, and I added a new sysctl called nameileafonly which defaults to ON (1). nameileafonly vmiodirenable action 1 1 (DEFAULT) purge leaf dirs on vnode reclaim 0 1 purge any dir on vnode reclaim 1 0 purge leaf dirs on vnode reclaim 0 0 do not purge dirs on vnode reclaim -1 0 puger any dir on vnode reclaim Index: sys/vnode.h =================================================================== RCS file: /home/ncvs/src/sys/sys/vnode.h,v retrieving revision 1.111.2.12 diff -u -r1.111.2.12 vnode.h --- sys/vnode.h 2001/09/22 09:21:48 1.111.2.12 +++ sys/vnode.h 2001/09/23 21:18:00 @@ -550,6 +550,7 @@ struct componentname *cnp)); void cache_purge __P((struct vnode *vp)); void cache_purgevfs __P((struct mount *mp)); +int cache_leaf_test __P((struct vnode *vp)); void cvtstat __P((struct stat *st, struct ostat *ost)); void cvtnstat __P((struct stat *sb, struct nstat *nsb)); int getnewvnode __P((enum vtagtype tag, Index: kern/vfs_bio.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_bio.c,v retrieving revision 1.242.2.9 diff -u -r1.242.2.9 vfs_bio.c --- kern/vfs_bio.c 2001/06/03 05:00:09 1.242.2.9 +++ kern/vfs_bio.c 2001/09/23 20:24:47 @@ -82,7 +82,7 @@ * but the code is intricate enough already. */ vm_page_t bogus_page; -int vmiodirenable = FALSE; +int vmiodirenable = TRUE; int runningbufspace; static vm_offset_t bogus_offset; Index: kern/vfs_cache.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_cache.c,v retrieving revision 1.42.2.4 diff -u -r1.42.2.4 vfs_cache.c --- kern/vfs_cache.c 2001/03/21 10:50:58 1.42.2.4 +++ kern/vfs_cache.c 2001/09/23 21:18:24 @@ -405,6 +405,31 @@ } /* + * cache_leaf_test() + * + * Test whether this (directory) vnode's namei cache entry contains + * subdirectories or not. Used to determine whether the directory is + * a leaf in the namei cache or not. Note: the directory may still + * contain files in the namei cache. + * + * Returns 0 if the directory is a leaf, -1 if it isn't. + */ +int +cache_leaf_test(struct vnode *vp) +{ + struct namecache *ncpc; + + for (ncpc = LIST_FIRST(&vp->v_cache_src); + ncpc != NULL; + ncpc = LIST_NEXT(ncpc, nc_src) + ) { + if (ncpc->nc_vp != NULL && ncpc->nc_vp->v_type == VDIR) + return(0); + } + return(-1); +} + +/* * Perform canonical checks and cache lookup and pass on to filesystem * through the vop_cachedlookup only if needed. */ Index: kern/vfs_subr.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_subr.c,v retrieving revision 1.249.2.11 diff -u -r1.249.2.11 vfs_subr.c --- kern/vfs_subr.c 2001/09/11 09:49:53 1.249.2.11 +++ kern/vfs_subr.c 2001/09/23 21:18:20 @@ -113,6 +113,8 @@ SYSCTL_INT(_vfs, OID_AUTO, reassignbufsortbad, CTLFLAG_RW, &reassignbufsortbad, 0, ""); static int reassignbufmethod = 1; SYSCTL_INT(_vfs, OID_AUTO, reassignbufmethod, CTLFLAG_RW, &reassignbufmethod, 0, ""); +static int nameileafonly = 1; +SYSCTL_INT(_vfs, OID_AUTO, nameileafonly, CTLFLAG_RW, &nameileafonly, 0, ""); #ifdef ENABLE_VFS_IOOPT int vfs_ioopt = 0; @@ -506,13 +508,32 @@ TAILQ_REMOVE(&vnode_free_list, vp, v_freelist); TAILQ_INSERT_TAIL(&vnode_tmp_list, vp, v_freelist); continue; - } else if (LIST_FIRST(&vp->v_cache_src)) { - /* Don't recycle if active in the namecache */ - simple_unlock(&vp->v_interlock); - continue; - } else { - break; } + if (LIST_FIRST(&vp->v_cache_src)) { + /* + * If nameileafonly is set, do not throw + * away directory vnodes unless they are + * leaf nodes in the namecache. + * + * If nameileafonly is not set then throw-aways + * are based on vmiodirenable. If + * vmiodirenable is turned off we do not throw + * away directory vnodes active in the + * namecache. The (nameileafonly < 0) test + * is for further debugging only. + */ + if (nameileafonly > 0) { + if (cache_leaf_test(vp) == 0) { + simple_unlock(&vp->v_interlock); + continue; + } + } else if (nameileafonly < 0 || + vmiodirenable == 0) { + simple_unlock(&vp->v_interlock); + continue; + } + } + break; } } To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 14:29:15 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 6760D37B415 for ; Sun, 23 Sep 2001 14:29:07 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8NLT0188769; Sun, 23 Sep 2001 14:29:00 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 14:29:00 -0700 (PDT) From: Matt Dillon Message-Id: <200109232129.f8NLT0188769@earth.backplane.com> To: Poul-Henning Kamp Cc: David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: cache purge cache for -current (Was Re: Conclusions on... cache_purgeleafdirs()) References: <1620.1001272770@critter> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Here is the latest patch for -current. vmiodirenable is turned on by default, the cache purge code is enabled based on vmiodirenable, and I added a new sysctl called nameileafonly which defaults to ON (1). The old cache_purgeleafdirs() stuff is #if 0'd out. nameileafonly vmiodirenable action 1 1 (DEFAULT) purge leaf dirs on vnode reclaim 0 1 purge any dir on vnode reclaim 1 0 purge leaf dirs on vnode reclaim 0 0 do not purge dirs on vnode reclaim -1 0 puger any dir on vnode reclaim Index: sys/vnode.h =================================================================== RCS file: /home/ncvs/src/sys/sys/vnode.h,v retrieving revision 1.157 diff -u -r1.157 vnode.h --- sys/vnode.h 2001/09/13 22:52:42 1.157 +++ sys/vnode.h 2001/09/23 21:17:23 @@ -559,7 +559,7 @@ struct componentname *cnp)); void cache_purge __P((struct vnode *vp)); void cache_purgevfs __P((struct mount *mp)); -void cache_purgeleafdirs __P((int ndir)); +int cache_leaf_test __P((struct vnode *vp)); void cvtstat __P((struct stat *st, struct ostat *ost)); void cvtnstat __P((struct stat *sb, struct nstat *nsb)); int getnewvnode __P((enum vtagtype tag, Index: kern/vfs_bio.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_bio.c,v retrieving revision 1.288 diff -u -r1.288 vfs_bio.c --- kern/vfs_bio.c 2001/09/12 08:37:46 1.288 +++ kern/vfs_bio.c 2001/09/23 21:13:02 @@ -90,7 +90,7 @@ * but the code is intricate enough already. */ vm_page_t bogus_page; -int vmiodirenable = FALSE; +int vmiodirenable = TRUE; int runningbufspace; static vm_offset_t bogus_offset; Index: kern/vfs_cache.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_cache.c,v retrieving revision 1.61 diff -u -r1.61 vfs_cache.c --- kern/vfs_cache.c 2001/09/12 08:37:46 1.61 +++ kern/vfs_cache.c 2001/09/23 21:17:47 @@ -101,8 +101,10 @@ SYSCTL_ULONG(_debug, OID_AUTO, numcache, CTLFLAG_RD, &numcache, 0, ""); static u_long numcachehv; /* number of cache entries with vnodes held */ SYSCTL_ULONG(_debug, OID_AUTO, numcachehv, CTLFLAG_RD, &numcachehv, 0, ""); +#if 0 static u_long numcachepl; /* number of cache purge for leaf entries */ SYSCTL_ULONG(_debug, OID_AUTO, numcachepl, CTLFLAG_RD, &numcachepl, 0, ""); +#endif struct nchstats nchstats; /* cache effectiveness statistics */ static int doingcache = 1; /* 1 => enable the cache */ @@ -247,6 +249,31 @@ } /* + * cache_leaf_test() + * + * Test whether this (directory) vnode's namei cache entry contains + * subdirectories or not. Used to determine whether the directory is + * a leaf in the namei cache or not. Note: the directory may still + * contain files in the namei cache. + * + * Returns 0 if the directory is a leaf, -1 if it isn't. + */ +int +cache_leaf_test(struct vnode *vp) +{ + struct namecache *ncpc; + + for (ncpc = LIST_FIRST(&vp->v_cache_src); + ncpc != NULL; + ncpc = LIST_NEXT(ncpc, nc_src) + ) { + if (ncpc->nc_vp != NULL && ncpc->nc_vp->v_type == VDIR) + return(0); + } + return(-1); +} + +/* * Lookup an entry in the cache * * We don't do this if the segment name is long, simply so the cache @@ -499,6 +526,8 @@ } } +#if 0 + /* * Flush all dirctory entries with no child directories held in * the cache. @@ -554,6 +583,8 @@ } numcachepl++; } + +#endif /* * Perform canonical checks and cache lookup and pass on to filesystem Index: kern/vfs_subr.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_subr.c,v retrieving revision 1.319 diff -u -r1.319 vfs_subr.c --- kern/vfs_subr.c 2001/09/12 08:37:47 1.319 +++ kern/vfs_subr.c 2001/09/23 21:19:26 @@ -110,6 +110,8 @@ /* Number of vnodes in the free list. */ static u_long freevnodes = 0; SYSCTL_LONG(_debug, OID_AUTO, freevnodes, CTLFLAG_RD, &freevnodes, 0, ""); + +#if 0 /* Number of vnode allocation. */ static u_long vnodeallocs = 0; SYSCTL_LONG(_debug, OID_AUTO, vnodeallocs, CTLFLAG_RD, &vnodeallocs, 0, ""); @@ -125,6 +127,7 @@ /* Number of vnodes attempted to recycle at a time. */ static u_long vnoderecyclenumber = 3000; SYSCTL_LONG(_debug, OID_AUTO, vnoderecyclenumber, CTLFLAG_RW, &vnoderecyclenumber, 0, ""); +#endif /* * Various variables used for debugging the new implementation of @@ -142,6 +145,8 @@ /* Set to 0 for old insertion-sort based reassignbuf, 1 for modern method. */ static int reassignbufmethod = 1; SYSCTL_INT(_vfs, OID_AUTO, reassignbufmethod, CTLFLAG_RW, &reassignbufmethod, 0, ""); +static int nameileafonly = 1; +SYSCTL_INT(_vfs, OID_AUTO, nameileafonly, CTLFLAG_RW, &nameileafonly, 0, ""); #ifdef ENABLE_VFS_IOOPT /* See NOTES for a description of this setting. */ @@ -556,15 +561,46 @@ * Don't recycle if active in the namecache or * if it still has cached pages or we cannot get * its interlock. + * + * XXX the namei cache can hold onto vnodes too long, + * causing us to run out of MALLOC space. Instead, we + * should make path lookups requeue any vnodes on the free + * list. */ - if (LIST_FIRST(&vp->v_cache_src) != NULL || - (VOP_GETVOBJECT(vp, &object) == 0 && + if ((VOP_GETVOBJECT(vp, &object) == 0 && (object->resident_page_count || object->ref_count)) || !mtx_trylock(&vp->v_interlock)) { TAILQ_INSERT_TAIL(&vnode_free_list, vp, v_freelist); vp = NULL; continue; } + if (LIST_FIRST(&vp->v_cache_src)) { + /* + * If nameileafonly is set, do not throw + * away directory vnodes unless they are + * leaf nodes in the namecache. + * + * If nameileafonly is not set then throw-aways + * are based on vmiodirenable. If + * vmiodirenable is turned off we do not throw + * away directory vnodes active in the + * namecache. The (nameileafonly < 0) test + * is for further debugging only. + */ + if (nameileafonly > 0) { + if (cache_leaf_test(vp) == 0) { + mtx_unlock(&vp->v_interlock); + vp = NULL; + continue; + } + } else if (nameileafonly < 0 || + vmiodirenable == 0) { + mtx_unlock(&vp->v_interlock); + vp = NULL; + continue; + } + } + /* * Skip over it if its filesystem is being suspended. */ @@ -636,6 +672,7 @@ vfs_object_create(vp, td, td->td_proc->p_ucred); +#if 0 vnodeallocs++; if (vnodeallocs % vnoderecycleperiod == 0 && freevnodes < vnoderecycleminfreevn && @@ -643,6 +680,7 @@ /* Recycle vnodes. */ cache_purgeleafdirs(vnoderecyclenumber); } +#endif return (0); } To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 14:30: 6 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 8E6E437B407 for ; Sun, 23 Sep 2001 14:29:55 -0700 (PDT) Received: from elischer.org (InterJet.elischer.org [192.168.1.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA46997; Sun, 23 Sep 2001 15:14:13 -0700 (PDT) Message-ID: <3BAE5283.C7B434CF@elischer.org> Date: Sun, 23 Sep 2001 14:22:11 -0700 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Matt Dillon Cc: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <1620.1001272770@critter> <200109232019.f8NKJZL88354@earth.backplane.com> Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matt Dillon wrote: > > :Block input operations is the one notable exception and it tells a > :very interesting story: Matts patch results in a 4% increase, but > :combined with vmdirioenable it results in a 21.5% decrease. > : > :That's pretty darn significant: one out of every five I/O have > :been saved. Notice that both the user and system times increased.. if there had been another parallel task, the overall system throughout may have decreased.. I'm not saying this is wrong, just that we should look at other workloads too. no point in optimising the system for compiling itself.. that's not really a real-world task.. lmost noo-one buys a freeBSD box so that they can compile freeBS.. they usually have other plans for what they want out of it... (of course this may effect other tasks in an even more positive way, but we should know that..) > : > :The reason it has not manifested itself in the "real" number is > :probably the high degree of parallelism in the task which practically > :ensures that the CPU will not go idle. > : > :I suggest we let Matt's patch depend on the vmiodirenable sysctl > :and change the default for that. > : > :If there are no bad side effects found in the next couple of months, > :then kill the sysctl and lets be done with it. > : > :Poul-Henning > > Very interesting! I agree completely. I will instrument the code > to make the namei cache check conditional on vmiodirenable. I will > also add a sysctl to change the check condition to test for leaf > directories or not (closer to what Seigo's code accomplished) so we can > test for differences there. If we can get that 4% down the code will > be more acceptable. > > The results make a lot of sense, especially if the machine has sufficient > memory to cache most of the source tree. Increasing the number of > vnode reclaims will increase I/O on directories, especially for something > like a buildworld which is doing major path expansions up the wazoo. > Turning on vmiodirenable in a buildworld situation will decrease I/O > significantly. > > -Matt > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message -- +------------------------------------+ ______ _ __ | __--_|\ Julian Elischer | \ U \/ / hard at work in | / \ julian@elischer.org +------>x USA \ a very strange | ( OZ ) \___ ___ | country ! +- X_.---._/ presently in San Francisco \_/ \\ v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 14:48:17 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 5C6E937B429 for ; Sun, 23 Sep 2001 14:48:15 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8NLmEN88896; Sun, 23 Sep 2001 14:48:14 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 14:48:14 -0700 (PDT) From: Matt Dillon Message-Id: <200109232148.f8NLmEN88896@earth.backplane.com> To: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: correction... nameileafonly=-1 is 'do not purge dirs on vnode reclaim'. (was cache purge cache for ...) Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG fdirs()) References: <1620.1001272770@critter> : : Here is the latest patch for -stable. vmiodirenable is turned on by : default, the cache purge code is enabled based on vmiodirenable, and : I added a new sysctl called nameileafonly which defaults to ON (1). : : nameileafonly vmiodirenable action : 1 1 (DEFAULT) purge leaf dirs on vnode reclaim : 0 1 purge any dir on vnode reclaim : 1 0 purge leaf dirs on vnode reclaim : 0 0 do not purge dirs on vnode reclaim : -1 0 puger any dir on vnode reclaim Correction. -1 any do NOT purge dirs on vnode reclaim Sorry for the mixup. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 14:49:47 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 5EC6B37B41D for ; Sun, 23 Sep 2001 14:49:42 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8NLnAV88904; Sun, 23 Sep 2001 14:49:10 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 14:49:10 -0700 (PDT) From: Matt Dillon Message-Id: <200109232149.f8NLnAV88904@earth.backplane.com> To: Julian Elischer Cc: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <1620.1001272770@critter> <200109232019.f8NKJZL88354@earth.backplane.com> <3BAE5283.C7B434CF@elischer.org> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Notice that both the user and system times increased.. :if there had been another parallel task, the overall system throughout may have :decreased.. : :I'm not saying this is wrong, just that we should look at other workloads too. :no point in optimising the system for compiling itself.. that's not really a :real-world task.. :lmost noo-one buys a freeBSD box so that they can compile freeBS.. they usually :have other plans for what they want out of it... : :(of course this may effect other tasks in an even more positive way, but we :should know that..) Yah, but buildworld is all we can really do on test boxes. We will have to wait for people to try these things on production squid, news, and web server systems to get real-world numbers. The timing difference is 1.1%. On two identical machines I got 1564 and 1553 seconds with the same config, which is 0.7%, so I expect the std-dev is going to be, what, 1.4% or so? I'm running some comprehensive buildworld tests now using /usr/bin/time -l and will post them when they finish. These are very fast machines: 26 minute buildworld on -stable! So I don't expect this to take very long. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 15: 8:32 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id DDEBD37B408 for ; Sun, 23 Sep 2001 15:08:27 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8NM8RN89015; Sun, 23 Sep 2001 15:08:27 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 15:08:27 -0700 (PDT) From: Matt Dillon Message-Id: <200109232208.f8NM8RN89015@earth.backplane.com> To: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: another correction (self negating) Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I got the cache_leaf_test() return values backwards. But that's ok, because my cache_leaf_test() if() statement is also backwards :-). I'll turn them around in a later patch set. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 20:58:24 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 1C35237B41B for ; Sun, 23 Sep 2001 20:58:18 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8O3wCb90533; Sun, 23 Sep 2001 20:58:12 -0700 (PDT) (envelope-from dillon) Date: Sun, 23 Sep 2001 20:58:12 -0700 (PDT) From: Matt Dillon Message-Id: <200109240358.f8O3wCb90533@earth.backplane.com> To: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: stable buildworld results w/ vmiodirenable & nameileafonly combos Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ok, here are the first set of results. I am going to rerun the entire suite of tests again with the machines limited to 128M of ram to see what happens then. BTW, these are really nice machines! I highly recommend DELL2550's. The results w/ 512M are basically that it doesn't matter what we do with the namei cache. vmiodirenable is the only thing that really makes a difference. I expect we will see more differentiation in the 128M tests. -Matt WIDE TERMINAL WINDOW REQUIRED! --------------------------------------------------------------------------------------------------- TEST SUITE 1 (512M ram) buildworld of -stable. DELL2550 (Duel PIII-1.2GHz / 512M ram / SCSI) 23 September 2001 SMP kernel, softupdates-enabled, dirpref'd local /usr/src (no nfs), make -j 12 buildworld UFS_DIRHASH. 2 identical machines tested in parallel (test1, test2) /usr/bin/time -l timings note: atime updates left enabled in all tests REUSE LEAF DIR VNODES: directory vnodes with no subdirectories in the namei cache can be reused REUSE ALL DIR VNODES: directory vnodes can be reused (namei cache ignored) DO NOT REUSE DIR...: (Poul's original 1995 algo) directory vnode can only be reused if no subdirectories or files in the namei cache VMIODIRENABLE ENABLED [------------ A ------------] [------------ B ------------] [------------ C ------------] [BEST CASE ] [BEST CASE ] [BEST CASE ] machine test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 pass (2) R 1 1 2 2 R 1 1 2 2 R 1 1 2 2 vfs.vmiodirenable E 1 1 1 1 E 1 1 1 1 E 1 1 1 1 vfs.nameileafonly B 1 1 1 1 B 0 0 0 0 B -1 -1 -1 -1 O O O O REUSE LEAF DIR VNODES O REUSE ALL DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI T T T 25:46 25:44 25:19 25:05 25:49 25:40 25:14 25:04 25:46 25:42 25:07 25:14 real 1546 1544 1519 1505 1549 1540 1514 1504 1546 1542 1507 1514 user 1361 1352 1359 1356 1362 1354 1361 1354 1361 1352 1358 1355 sys 636 637 645 640 632 633 642 641 636 637 642 640 max resident 16292 16276 16268 16288 16284 16288 16280 16280 16288 16288 16284 16288 avg shared mem 1026 1025 1025 1025 1027 1028 1025 1023 1025 1026 1026 1025 avg unshared data 1018 1009 1014 1007 1007 1010 1007 1008 1010 1018 1006 1002 avg unshared stack 129 129 129 129 129 129 129 128 129 129 129 129 page reclaims 11.15M 11.15M 11.16M 11.15M 11.15M 11.16M 11.15M 11.16M 11.15M 11.15M 11.15M 11.15M page faults 1812 1800 1316 1348 1797 1798 1320 1273 1795 1797 1307 1321 swaps 0 0 0 0 0 0 0 0 0 0 0 0 block input ops 26542 26535 9272 9233 26555 26577 9258 8720 26470 26552 8905 9237 block output ops 5400 5217 5248 5266 5450 5109 5328 5389 5332 5371 5300 5341 messages sent 34582 34572 33533 33538 34579 34538 33539 33510 34610 34587 33525 33543 messages received 34582 34578 33533 33539 34580 34540 33546 33524 34610 34589 33525 33543 signals received 8 8 8 8 8 8 8 8 8 8 8 8 voluntary ctx sw 594390 595868 575038 574626 594677 594028 571265 572912 593500 593548 571394 573575 invol. ctx switch 380583 381897 378850 376941 380393 381540 374408 376260 380385 379585 375318 374844 desiredvnodes 36157 36157 36157 36157 36157 36157 36157 36157 36157 36157 36157 36157 maxvnodes (sysstat)(1) 37099 37064 37122 37208 37180 36928 37193 37314 37175 37152 37152 37152 VMIODIRENABLE DISABLED [------------ D ------------] [------------ E ------------] [BEST CASE ] [BEST CASE ] machine test1 test2 test1 test2 test1 test2 test1 test2 pass (2) R 1 1 2 2 R 1 1 2 2 vfs.vmiodirenable E 0 0 0 0 E 0 0 0 0 vfs.nameileafonly B 1 1 1 1 B <=0 <=0 <=0 <=0 O O O REUSE LEAF DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI T T 26:19 26:03 25:50 25:37 26:16 26:10 25:46 25:43 real 1579 1563 1550 1537 1576 1570 1546 1543 user 1361 1356 1360 1355 1359 1349 1361 1354 sys 634 632 640 637 636 639 641 641 max resident 16264 16276 16276 16280 16292 16280 16284 16283 avg shared mem 1026 1025 1026 1025 1027 1025 1025 1026 avg unshared data 1012 1005 1010 1009 1009 1007 1010 1010 avg unshared stack 129 129 129 129 129 129 129 129 page reclaims 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M page faults 1716 1720 1272 1263 1717 1758 1264 1244 swaps 0 0 0 0 0 0 0 0 block input ops 33716 33749 18888 18930 33788 33849 18760 18794 block output ops 5283 5304 5305 5365 5525 5411 5423 5465 messages sent 34759 34732 33659 33650 34710 34707 33645 33686 messages received 34760 34738 33660 33651 34711 34707 33649 33693 signals received 8 8 8 8 8 8 8 8 voluntary ctx sw 604747 602379 586746 584227 603949 603830 584766 585322 invol. ctx switch 384576 383500 381032 379730 383183 383798 379336 380705 desiredvnodes 36157 36157 36157 36157 36157 36157 36157 36157 maxvnodes (sysstat)(1) 37096 37062 37488 37231 37107 37228 37107 37228 note(1): there aren't enough discrete directories in /usr/src and /usr/obj to make a difference in the maxvnode numbers no matter what algorithm is chosen when vmiodirenable is turned on. /usr/src has 6607 directories and 62635 files. Running the entire suite of tests again with the machine artificially reduced to 128M of memory should yield more interesting results. note(2): PASS 1 vs PASS 2. A second buildworld pass is taken after the first completes. Considering the amount of ram in the machines the second pass is expected to perform much better due to prior caching of data. TESTS 1A, 1B, 1C: All results are within a standard-deviation of each other. With vmiodirenabled turned on, it does not appear to matter what type of namecache restriction(s) we place on directory vnode reuse. The existance of direcory pages in the VM Page cache appear to override all other considerations (at least on these boxes, with 512M of physical ram). Again, running the entire suite of tests again with the machines artifically reduced to 128M of memory should yield more interesting results. TESTS 1D, 1E: These tests were run with vmiodirenable turned off. This has the effect of reducing (by two orderes of magnitude) the amount of memory available to cache directories. Therefore the namei cache tests in the vnode reuse code become much more important. Turning off vmiodirenable resulted in around 40 second longer buildworlds even in the best case. Again, these numbers should alter more drastically in SUITE 2 when we test with less physical memory. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 22:14:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 60BE137B42A for ; Sun, 23 Sep 2001 22:14:08 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.4/8.11.4) with ESMTP id f8O5Dmv07142; Mon, 24 Sep 2001 07:13:49 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Julian Elischer Cc: Matt Dillon , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine In-Reply-To: Your message of "Sun, 23 Sep 2001 14:22:11 PDT." <3BAE5283.C7B434CF@elischer.org> Date: Mon, 24 Sep 2001 07:13:48 +0200 Message-ID: <7140.1001308428@critter> From: Poul-Henning Kamp Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <3BAE5283.C7B434CF@elischer.org>, Julian Elischer writes: >Matt Dillon wrote: >> >> :Block input operations is the one notable exception and it tells a >> :very interesting story: Matts patch results in a 4% increase, but >> :combined with vmdirioenable it results in a 21.5% decrease. >> : >> :That's pretty darn significant: one out of every five I/O have >> :been saved. > >Notice that both the user and system times increased.. Not significantly. As I already said: they are inside the standard deviation and therefore concluding anything from them is bogus. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Sep 23 23:56:23 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from smtp.noos.fr (racine.noos.net [212.198.2.71]) by hub.freebsd.org (Postfix) with ESMTP id 39E7037B40E for ; Sun, 23 Sep 2001 23:56:19 -0700 (PDT) Received: (qmail 72718872 invoked by uid 0); 24 Sep 2001 06:56:18 -0000 Received: from unknown (HELO gits.dyndns.org) ([212.198.231.187]) (envelope-sender ) by 212.198.2.71 (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 24 Sep 2001 06:56:18 -0000 Received: (from root@localhost) by gits.dyndns.org (8.11.6/8.11.6) id f8O6u8w64347; Mon, 24 Sep 2001 08:56:08 +0200 (CEST) (envelope-from root) Message-Id: <200109240656.f8O6u8w64347@gits.dyndns.org> Subject: Re: Version of XFree86 in FreeBSD Release 4.4 In-Reply-To: <20010923101244.A9616@dragon.nuxi.com> To: obrien@freebsd.org Date: Mon, 24 Sep 2001 08:56:08 +0200 (CEST) Cc: Jordan Hubbard , glenngombert@zdnetonebox.com, freebsd-hackers@freebsd.org Reply-To: clefevre@citeweb.net From: Cyrille Lefevre Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[<97Zd*>^#%Y5Cxv;%Y[PT-LW3;A:fRrJ8+^k"e7@+30g0YD0*^^3jgyShN7o?a]C la*Zv'5NA,=963bM%J^o]C X-Mailer: ELM [version 2.4ME+ PL94b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG David O'Brien wrote: > On Sun, Sep 23, 2001 at 04:05:27PM +0200, Cyrille Lefevre wrote: > > David O'Brien wrote: > > > On Mon, Sep 17, 2001 at 05:42:23PM -0700, Jordan Hubbard wrote: > > > > We're still waiting for 4.0's "support footprint" to widen > > > > a bit more before subjecting people to it by default. Hopefully > > > > by 4.5. > > > > > > Are you really considering using XFree86 4.x in FreeBSD-4.5? > > > When I asked you about this in the past, you had said you wanted to keep > > > the same X in RELENG_4 (presumable to not "rock the boat" mid-branch). > > > > isn't it possible to provide both versions and left the user to > > choice between both w/ a information box relating problems found > > in the one or the other ? > > There are issues for the pre-compiled packages due to differences between > the two versions of XFree86. what kind of issues ? I'm using both XFree86-4 and ports in package form (pre-compiled stuffs) w/o any problems. Cyrille. -- Cyrille Lefevre mailto:clefevre@citeweb.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 0: 6:21 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by hub.freebsd.org (Postfix) with ESMTP id 8335F37B40E for ; Mon, 24 Sep 2001 00:06:19 -0700 (PDT) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.6/8.11.1) id f8O76D209403; Mon, 24 Sep 2001 00:06:13 -0700 (PDT) (envelope-from obrien) Date: Mon, 24 Sep 2001 00:06:13 -0700 From: "David O'Brien" To: Cyrille Lefevre Cc: freebsd-hackers@freebsd.org Subject: Re: Version of XFree86 in FreeBSD Release 4.4 Message-ID: <20010924000613.A9322@dragon.nuxi.com> Reply-To: obrien@freebsd.org References: <20010923101244.A9616@dragon.nuxi.com> <200109240656.f8O6u8w64347@gits.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109240656.f8O6u8w64347@gits.dyndns.org>; from clefevre@citeweb.net on Mon, Sep 24, 2001 at 08:56:08AM +0200 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Sep 24, 2001 at 08:56:08AM +0200, Cyrille Lefevre wrote: > what kind of issues ? I'm using both XFree86-4 and ports in package form > (pre-compiled stuffs) w/o any problems. Please RTF /usr/ports/Mk/bsd.port.mk and look at what "XFREE86_VERSION" does. -- -- David (obrien@FreeBSD.org) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 1:44: 1 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id DE38737B40C for ; Mon, 24 Sep 2001 01:43:55 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8O8htO91785; Mon, 24 Sep 2001 01:43:55 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 01:43:55 -0700 (PDT) From: Matt Dillon Message-Id: <200109240843.f8O8htO91785@earth.backplane.com> To: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Second set of stable buildworld results w/ vmiodirenable & nameileafonly combos Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ok, here is the second set of results. I didn't run all the tests because nothing I did appeared to really have much of an effect. In this set of tests I set MAXMEM to 128M. As you can see the buildworld took longer verses 512M (no surprise), and vmiodirenable still helped verses otherwise. If one takes into consideration the standard deviation, the directory vnode reclamation parameters made absolutely no difference in the tests. The primary differentiator in all the tests is 'block input ops'. With vmiodirenable turned on it sits at around 51000. With it off it sits at around 56000. In the 512M tests the pass-1 numbers were 26000 with vmiodirenable turned on and 33000 with it off. Pass-2 numbers were 9000 with it on and 18000 with it off. The directory leaf reuse parameters had almost no effect on either the 128M or 512M numbers. I'm not sure why test2 wound up doing a better job then test1 in the 128M tests with vmiodirenable disabled. Both machines are configured identically with only some extra junk on test1's /usr from prior tests. In anycase, the differences point to a rather significant error spread in regards to possible outcomes, at least with vmiodirenable=0. My conclusion from all of this is: * vmiodirenable should be turned on by default. * We should rip out the cache_purgeleafdirs() code entirely and use my simpler version to fix the vnode-growth problem. * We can probably also rip out my cache_leaf_test() .. we do not need to add any sophistication to reuse only directory vnodes without subdirectories in the cache. If it had been a problem we would have seen it. I can leave the sysctl's in place on the commit to allow further testing, and I can leave it conditional on vmiodirenable. I'll set the default vmiodirenable to 1 (which will also enable directory vnode reuse) and the default nameileafonly to 0 (i.e. to use the less sophisticated check). In a few weeks I will rip-out nameileafonly and cache_leaf_test(). -Matt WIDE TERMINAL WINDOW REQUIRED! --------------------------------------------------------------------------------------------------- TEST SUITE 2 (128M ram) buildworld of -stable. DELL2550 (Duel PIII-1.2GHz / 128M ram (via MAXMEM) / SCSI) 23 September 2001 SMP kernel, softupdates-enabled, dirpref'd local /usr/src (no nfs), make -j 12 buildworld UFS_DIRHASH. 2 identical machines tested in parallel (test1, test2) /usr/bin/time -l timings note: atime updates left enabled in all tests REUSE LEAF DIR VNODES: directory vnodes with no subdirectories in the namei cache can be reused REUSE ALL DIR VNODES: directory vnodes can be reused (namei cache ignored) DO NOT REUSE DIR...: (Poul's original 1995 algo) directory vnode can only be reused if no subdirectories or files in the namei cache I stopped bothering with pass-2 after it became evident that the numbers were not changing significantly. VMIODIRENABLE ENABLED [------------ A ------------] [------------ B ------------] [------------ C ------------] [BEST CASE ] [BEST CASE ] [BEST CASE ] machine test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 test1 test2 pass (2) R 1 1 2 2 R 1 1 2 2 R 1 1 2 2 vfs.vmiodirenable E 1 1 1 1 E 1 1 1 1 E 1 1 1 1 vfs.nameileafonly B 1 1 1 1 B 0 0 0 0 B -1 -1 -1 -1 O O O O REUSE LEAF DIR VNODES O REUSE ALL DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI T T T 26:49 26:30 26:41 26:24 real 1609 1590 1601 1584 user 1361 1354 1361 1356 sys 617 615 617 614 max resident 16264 16256 16260 16264 avg shared mem 1030 1030 1030 1030 avg unshared data 1004 1005 1006 1004 avg unshared stack 129 129 129 129 page reclaims 11.16M 11.16M 11.15M 11.15M page faults 3321 3674 2940 2801 swaps 0 0 0 0 block input ops 51748 51881 50777 50690 block output ops 5532 6497 5680 6089 messages sent 35847 35848 35789 35715 messages received 35848 35852 35792 35721 signals received 8 8 8 8 voluntary ctx sw 634633 637640 633166 630426 invol. ctx switch 389944 391048 389868 390128 desiredvnodes 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993 11993 maxvnodes (sysstat)(1) 10775 10624 10775 10624 VMIODIRENABLE DISABLED [------------ D ------------] [------------ E ------------] [BEST CASE ] [BEST CASE ] machine test1 test2 test1 test2 test1 test2 test1 test2 pass (2) R 1 1 2 2 R 1 1 2 2 vfs.vmiodirenable E 0 0 0 0 E 0 0 0 0 vfs.nameileafonly B 1 1 1 1 B <=0 <=0 <=0 <=0 O O O REUSE LEAF DIR VNODES O DO NOT REUSE DIR VNODES W/ACTIVE NAMEI T T 27:12 26:42 27:06 26:33 27:16 26:42 27:12 26:35 real 1632 1602 1626 1593 1636 1602 1632 1595 user 1359 1354 1361 1352 1361 1352 1359 1354 sys 616 617 614 616 614 618 617 610 max resident 16268 16268 16264 16252 16272 16260 16248 16264 avg shared mem 1030 1028 1031 1029 1030 1028 1030 1029 avg unshared data 1004 1005 1007 1003 1004 1003 1006 1006 avg unshared stack 129 129 129 128 129 129 129 129 page reclaims 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M 11.15M page faults 3299 3201 2677 2716 3179 3268 3391 2788 swaps 0 0 0 0 0 0 0 0 block input ops 56118 56310 55623 55287 56228 56187 44508 55618 block output ops 6181 5454 6179 5305 7935 5355 7758 5357 messages sent 35927 35945 35814 35731 35946 35866 35813 35846 messages received 35927 35945 35815 35732 35948 35866 35819 35848 signals received 8 8 8 8 8 8 8 8 voluntary ctx sw 643370 640072 641656 637785 644750 638374 641644 637633 invol. ctx switch 392619 392669 393116 392564 392246 390639 390585 391506 desiredvnodes 11993 11993 11993 11993 11993 11993 11993 11993 maxvnodes (sysstat)(1) 10657 10606 10657 10606 10698 10630 10698 10630 note(1): Again there aren't enough directory vnodes to force maxvnodes into overflow. note(2): PASS 1 vs PASS 2. Here the lack of memory comes into play. The second pass has no pre-cached advantage over the first pass. The numbers are essentially the same between pass1 and pass2. TESTS 1A, 1B, 1C: I didn't bother with 1B and 1C. See below. TESTS 1D, 1E: I'm not sure why the 'test2' host has 800 or so fewer output ops. I reran the test twice. /usr on test2 has less data then /usr on test1 (due to other unrelated tests), but I dunno if that could account for it. In anycase, output ops have no real bearing on the test. Focusing on block input ops one can see that only vmiodirenable makes a real difference in regards to read caching. The difference between the numbers for 1A verses 1D and 1E is almost certainly associated with directory data. nameileafonly seems to have made no difference at all. And if it made no difference in 1D and 1E, it certainly would not make a difference in 1B or 1C. So I didn't bother with 1B and 1C. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 4: 7: 5 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from scribble.fsn.hu (scribble.fsn.hu [193.224.40.95]) by hub.freebsd.org (Postfix) with SMTP id 833AD37B411 for ; Mon, 24 Sep 2001 04:07:02 -0700 (PDT) Received: (qmail 94928 invoked by uid 1000); 24 Sep 2001 11:07:01 -0000 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 24 Sep 2001 11:07:01 -0000 Date: Mon, 24 Sep 2001 13:07:00 +0200 (CEST) From: Attila Nagy To: Subject: Disk based file system cache Message-ID: <20010918165302.V17360-100000@scribble.fsn.hu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, I'm just curious: is it possible to set up an NFS server and a client where the client has very big (28 GB maximum for FreeBSD?) swap area on multiple disks and caches the NFS exported data on it? This could save a lot of bandwidth on the NFS server and also redues load on that. Thanks, -------------------------------------------------------------------------- Attila Nagy e-mail: Attila.Nagy@fsn.hu Budapest Polytechnic (BMF.HU) @work: +361 210 1415 (194) H-1084 Budapest, Tavaszmezo u. 15-17. cell.: +3630 306 6758 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 4:23: 1 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from rfhs8012.fh-regensburg.de (rfhs8012.fh-regensburg.de [194.95.108.29]) by hub.freebsd.org (Postfix) with ESMTP id 1718937B40B for ; Mon, 24 Sep 2001 04:22:59 -0700 (PDT) Received: from rfhpc8320.fh-regensburg.de (wup34966@rfhpc8320.fh-regensburg.de [194.95.108.32]) by rfhs8012.fh-regensburg.de (8.11.6/8.11.6) with ESMTP id f8OBMtP21315 for ; Mon, 24 Sep 2001 13:22:55 +0200 (MEST) Received: (from wup34966@localhost) by rfhpc8320.fh-regensburg.de (8.9.1/8.8.3) id NAA02698 for freebsd-hackers@freebsd.org; Mon, 24 Sep 2001 13:22:53 +0200 (MET DST) Date: Mon, 24 Sep 2001 13:22:53 +0200 From: Peter Wullinger To: freebsd-hackers@freebsd.org Subject: Re: Disk based file system cache Message-ID: <20010924132253.A2599@rfhpc8320.fh-regensburg.de> References: <20010918165302.V17360-100000@scribble.fsn.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010918165302.V17360-100000@scribble.fsn.hu>; from bra@fsn.hu on Mon, Sep 24, 2001 at 01:07:00PM +0200 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Sep 24, 2001 at 01:07:00PM +0200, Attila Nagy wrote: > Hello, > > I'm just curious: is it possible to set up an NFS server and a client > where the client has very big (28 GB maximum for FreeBSD?) swap area on > multiple disks and caches the NFS exported data on it? > This could save a lot of bandwidth on the NFS server and also redues load > on that. > I'm not familiar with nfsiod(was it?), but I think, that this NFS run in kernel mode and uses kernel malloc(9) memory for caching. And kernel memory is quite different from user space memory ... Correct me, if I'm wrong. Even if it worked, you will possibly get REAL problems due to synchronisation problems. If your client machines are Linux, Solaris or (;-)) FreeBSD, you can setup CODA from the ports collection, it's much more suitable for this. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 4:49:42 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mss.rdc2.nsw.optushome.com.au (ha1.rdc2.nsw.optushome.com.au [203.164.2.50]) by hub.freebsd.org (Postfix) with ESMTP id 8384437B414 for ; Mon, 24 Sep 2001 04:49:29 -0700 (PDT) Received: from co3045456-a.sunsh1.vic.optushome.com.au ([203.164.32.100]) by mss.rdc2.nsw.optushome.com.au (InterMail vM.4.01.03.20 201-229-121-120-20010223) with ESMTP id <20010924114927.CYSO13193.mss.rdc2.nsw.optushome.com.au@co3045456-a.sunsh1.vic.optushome.com.au> for ; Mon, 24 Sep 2001 21:49:27 +1000 Received: from optushome.com.au (tbird-fe1.home.lan [192.168.2.5]) by co3045456-a.sunsh1.vic.optushome.com.au (8.11.3/8.11.0) with ESMTP id f8OBnZ601040 for ; Mon, 24 Sep 2001 21:49:36 +1000 (EST) (envelope-from markhannon@optushome.com.au) Message-ID: <3BAF1DCE.4BA21B8D@optushome.com.au> Date: Mon, 24 Sep 2001 21:49:34 +1000 From: Mark Hannon X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: dump files too large, nodump related?? Content-Type: multipart/mixed; boundary="------------558CB1854F512336B4523BA1" Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. --------------558CB1854F512336B4523BA1 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi, I have noticed some strange behaviour with 4.3-RELEASE and dump. I have been dumping my filesystems through gzip into a compressed dumpfile. Some of the resulting dumps have been MUCH larger than I would expect. As an example, I have just dumped my /home partition .... note that lots of directories on this partition are marked nodump, eg /home/ftp which is one of the biggest users of diskspace. Building 8 level dump of /home and writing it to /var/dumps//home8.gz (gzipped) DUMP: Date of this level 8 dump: Mon Sep 24 21:13:55 2001 DUMP: Date of last level 1 dump: Tue Sep 18 20:15:43 2001 DUMP: Dumping /dev/ad0s1h (/home) to standard output DUMP: mapping (Pass I) [regular files] DUMP: mapping (Pass II) [directories] DUMP: estimated 360780 tape blocks. DUMP: dumping (Pass III) [directories] DUMP: dumping (Pass IV) [regular files] DUMP: 30.76% done, finished in 0:11 DUMP: 60.89% done, finished in 0:06 DUMP: DUMP: 360664 tape blocks DUMP: finished in 849 seconds, throughput 424 KBytes/sec DUMP: level 8 dump on Mon Sep 24 21:13:55 2001 DUMP: DUMP IS DONE The GZIPPED dumpfile is 289 MB!!! I wrote a little perl script to check the table of contents and estimate how big the dump should be (see attached) and this gives an interesting result. doorway:~> proj/dumpsize/dumpsize.pl /home /var/dumps/home8.gz Level 8 dump of /home on doorway.home.lan:/dev/ad0s1h Label: none The level 0 dump of /home partition written to /var/dumps/home8.gz contains 689 files totalling 146450 KB, cf size of dumpfile = 282063 ( 360660 ) KB The following files are larger than 1024 KB in size: 163264 ./mark/.netscape/xover-cache/host-news/athome.aus.service.snm 1343488 ./mark/.netscape/xover-cache/host-news/athome.aus.support.snm 2097152 ./mark/.netscape/xover-cache/host-news/athome.aus.users.linux.snm 1754819 ./mark/.netscape/xover-cache/host-news/hostinfo.dat 1122336 ./samba/profile.9x/mark/USER.DAT 1441792 ./samba/profile.9x/tuija/History/History.IE5/index.dat 92440996 ./tuija/Mail/Archive/Sent Items 2001 2985510 ./tuija/My Documents/gas1.JPG 2528914 ./tuija/My Documents/gas2.JPG The interesting thing here is that the sum of all the file sizes in the dump is only 147MB cf the 361MB uncompressed dump size!!!!!!! This is a discrepancy of 210MB. (This would line up with the 180MB ISO image plus other dribs and drabs that I have stored in a nodump flagged directory since my last dump) Any ideas of what is wrong? Are the nodumped files stored on the dump for some reason (even though they don't appear in the restore table of contents) Regards/Mark --------------558CB1854F512336B4523BA1 Content-Type: application/x-perl; name="dumpsize.pl" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="dumpsize.pl" #!/usr/bin/perl # # $Id: dumpsize.pl,v 1.4 2001/09/23 05:33:22 mark Exp mark $ # # Usage: dumpsize.pl [-list] partition gzipped_dumpfilename # use strict; my ($progname) = "dumpsize.pl"; # Name of this program for errors my ($progusage) = "[-list] partition gzipped_dumpfilename"; my ($list_flag) = 0; # 1 if -list specified on command line my ($threshold) = 1024 * 1024; # Threshold to print out files my ($tmp_dump_gzip) = "/tmp/dump_gzip"; my ($tmp_dump_toc) = "/tmp/dump_toc"; my ($partition); # Name of partition in dumpfile my ($dumpfile); # Name of dumpfile my ($dumplevel) = 0; # Dump level - not implemented yet! my ($dump_size); # Size, in bytes, of unzipped dumpfile my ($dump_size_gzip); # Size, in bytes, of gzipped dumpfile my ($dump_is_gzipped) = 1; # 1 if dumpfile has been gzipped my ($i); # Loop counter my ($total_size) = 0; # Sum of file sizes included in dump my (@line); # All lines for TOC stored in this array my (@token); # Temporary variable used to split lines my (@leaf_file); # Unsorted array of leafnodes found in TOC my (@file_list); # Sorted array of leafnode filenames my (@file_size); # Parallel array of file sizes ... my ($no_files) = 0; # Number of files on tape # ----------------------------------------------------------------------------------- # Parse command line, open dump file and table of contents unless( $ARGV[0] ne "" ){ die( "Usage %s %s\n", $progname, $progusage); } if ( $ARGV[0] eq "-list" ){ $list_flag = 1; $partition = $ARGV[1]; $dumpfile = $ARGV[2]; } else { $list_flag = 0; $partition = $ARGV[0]; $dumpfile = $ARGV[1]; } unless( chdir ( $partition ) ){ die ( "$progname : Unable to chdir to partition $partition\n" ); } unless( -e $dumpfile ){ die( "$progname : Unable to open gzipped dumpfile $dumpfile\n"); } # ----------------------------------------------------------------------------------- # Calculate uncompressed size of dumpfile $dump_size_gzip = -s $dumpfile; system( "/usr/bin/gzip -l $dumpfile > $tmp_dump_gzip.$$" ); unless( open( GZIP_DETAILS, "$tmp_dump_gzip.$$" ) ){ die( "$progname : Unable to open TOC file $tmp_dump_gzip.$$\n" ); } @line = ; $line[1] =~ s/^ +//; @token = split( / +/, $line[1] ); $dump_size = $token[1]; # ----------------------------------------------------------------------------------- # Parse restore TOC and look for all leaf entries, store contents into @file_list system( "/usr/bin/zcat $dumpfile | /sbin/restore tvf - > $tmp_dump_toc.$$" ); unless( open( RESTORE_TOC, "$tmp_dump_toc.$$" ) ){ die( "$progname : Unable to open TOC file $tmp_dump_toc.$$\n" ); } @line = ; for ( $i = 0 ; $i < @line ; $i++ ){ if ( $line[$i] =~ /^leaf/ ){ @token = split(/\t/, $line[$i] ); $leaf_file[$i] = $token[1]; chop( $leaf_file[$i] ); } } @file_list = sort( @leaf_file ); for ( $i = 0 ; $i < @file_list ; $i++ ){ $file_size[$i] = -s $file_list[$i]; if ( $file_size[$i] > 0 ){ $no_files++; } $total_size += $file_size[$i]; } # ----------------------------------------------------------------------------------- # Print detailed list of dumpfiles if ( $list_flag == 1 ){ for ( $i = 0 ; $i < @file_list ; $i++ ){ printf( "%d\t%s\n" , $file_size[$i], $file_list[$i] ); } } # ----------------------------------------------------------------------------------- # Print summary of results printf( "The level %d dump of %s partition written to %s\n", $dumplevel, $partition, $dumpfile ); printf( "contains %d files totalling %d KB, cf size of dumpfile = %d ( %d ) KB\n", $no_files, $total_size/1024, $dump_size_gzip/1024, $dump_size/1024 ); printf( "\nThe following files are larger than %d KB in size:\n", $threshold/1024 ); for ( $i = 0 ; $i < @file_list ; $i++ ){ if ( $file_size[$i] > $threshold ){ printf( "%d\t%s\n" , $file_size[$i], $file_list[$i] ); } } # ----------------------------------------------------------------------------------- # Cleanup temporary files etc. unlink ( "$tmp_dump_gzip.$$" ); unlink ( "$tmp_dump_toc.$$" ); --------------558CB1854F512336B4523BA1-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 7:26: 1 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by hub.freebsd.org (Postfix) with SMTP id 6697437B403 for ; Mon, 24 Sep 2001 07:25:58 -0700 (PDT) Received: from walton.maths.tcd.ie by salmon.maths.tcd.ie with SMTP id ; 24 Sep 2001 15:25:57 +0100 (BST) Date: Mon, 24 Sep 2001 15:25:56 +0100 From: David Malone To: Attila Nagy Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Disk based file system cache Message-ID: <20010924152556.A90833@walton.maths.tcd.ie> References: <20010918165302.V17360-100000@scribble.fsn.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010918165302.V17360-100000@scribble.fsn.hu>; from bra@fsn.hu on Mon, Sep 24, 2001 at 01:07:00PM +0200 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Sep 24, 2001 at 01:07:00PM +0200, Attila Nagy wrote: > I'm just curious: is it possible to set up an NFS server and a client > where the client has very big (28 GB maximum for FreeBSD?) swap area on > multiple disks and caches the NFS exported data on it? > This could save a lot of bandwidth on the NFS server and also redues load > on that. This would really be more than NFS is supposed to do. There other filesystems which can do this sort of thing - I think Coda might be one of them. David. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 7:57:34 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from acl.lanl.gov (acl.lanl.gov [128.165.147.1]) by hub.freebsd.org (Postfix) with SMTP id D520137B407 for ; Mon, 24 Sep 2001 07:57:31 -0700 (PDT) Received: (qmail 1451974 invoked from network); 24 Sep 2001 08:57:31 -0600 Received: from snaresland.acl.lanl.gov (128.165.147.113) by acl.lanl.gov with SMTP; 24 Sep 2001 08:57:31 -0600 Received: (qmail 2441 invoked by uid 3499); 24 Sep 2001 08:57:30 -0600 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 24 Sep 2001 08:57:30 -0600 Date: Mon, 24 Sep 2001 08:57:30 -0600 (MDT) From: Ronald G Minnich X-X-Sender: To: David Malone Cc: Attila Nagy , Subject: Re: Disk based file system cache In-Reply-To: <20010924152556.A90833@walton.maths.tcd.ie> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 24 Sep 2001, David Malone wrote: > On Mon, Sep 24, 2001 at 01:07:00PM +0200, Attila Nagy wrote: > > I'm just curious: is it possible to set up an NFS server and a client > > where the client has very big (28 GB maximum for FreeBSD?) swap area on > > multiple disks and caches the NFS exported data on it? > > This could save a lot of bandwidth on the NFS server and also redues load > > on that. > > This would really be more than NFS is supposed to do. There other > filesystems which can do this sort of thing - I think Coda might > be one of them. see the autocacher paper at http://www.acl.lanl.gov/~rminnich let me know if this is what you mean. I recently built a new one "from scratch" and it works. BUT, at least with Linux, my measurements show that the dcache really eliminates most of the benefits. ron To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 9:22:19 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from brunel.uk1.vbc.net (brunel.uk1.vbc.net [194.207.2.8]) by hub.freebsd.org (Postfix) with ESMTP id D178337B40A for ; Mon, 24 Sep 2001 09:22:15 -0700 (PDT) Received: from localhost (jcv@localhost) by brunel.uk1.vbc.net (8.11.6/8.11.6) with ESMTP id f8OGMCD39934; Mon, 24 Sep 2001 17:22:13 +0100 (BST) X-Authentication-Warning: brunel.uk1.vbc.net: jcv owned process doing -bs Date: Mon, 24 Sep 2001 17:22:12 +0100 (BST) From: Jean-Christophe Varaillon X-Sender: jcv@brunel.uk1.vbc.net To: Marco Wertejuk Cc: freebsd-hackers@FREEBSD.ORG Subject: Re: Boot proccess In-Reply-To: <20010920000212.B60704@localhost.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > Hello, > > | In short, which program gives enough knowledge to the microprocessor (?) > | and allow him to use kern.flp & mfsroot.flp in order to boot and make the > | operating system running. > > your BIOS reads the first sektor from your floppy which consists > of a boot loader, which usually loads the 2nd step boot loader > and this one loads the kernel. Tell me if I am wrong but from the floppy, the files kern.flp & mfsroot.flp are compressed and then uncompressed into memory. If so, that means that the FreeBSD box is running this programs from the RAM and not from the floppy, right ? If so, is it possible to do the same but from a hard disk instead of the floppy ? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 9:38:13 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from bdr-xcon.matchlogic.com (mail.matchlogic.com [205.216.147.127]) by hub.freebsd.org (Postfix) with ESMTP id ED38237B406 for ; Mon, 24 Sep 2001 09:38:09 -0700 (PDT) Received: by mail.matchlogic.com with Internet Mail Service (5.5.2653.19) id ; Mon, 24 Sep 2001 10:37:19 -0600 Message-ID: <5FE9B713CCCDD311A03400508B8B30130828F475@bdr-xcln.corp.matchlogic.com> From: Charles Randall To: 'David Malone' , Attila Nagy Cc: freebsd-hackers@FreeBSD.ORG Subject: RE: Disk based file system cache Date: Mon, 24 Sep 2001 10:37:18 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG As a side note, Irix and Solaris provide cachefs for this purpose and use NFS filesystems as examples (others examples may include CD-ROM, etc). Charles -----Original Message----- From: David Malone [mailto:dwmalone@maths.tcd.ie] Sent: Monday, September 24, 2001 8:26 AM To: Attila Nagy Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Disk based file system cache On Mon, Sep 24, 2001 at 01:07:00PM +0200, Attila Nagy wrote: > I'm just curious: is it possible to set up an NFS server and a client > where the client has very big (28 GB maximum for FreeBSD?) swap area on > multiple disks and caches the NFS exported data on it? > This could save a lot of bandwidth on the NFS server and also redues load > on that. This would really be more than NFS is supposed to do. There other filesystems which can do this sort of thing - I think Coda might be one of them. David. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 11:19:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mailgw3a.lmco.com (mailgw3a.lmco.com [192.35.35.24]) by hub.freebsd.org (Postfix) with ESMTP id ADA7E37B425 for ; Mon, 24 Sep 2001 11:19:04 -0700 (PDT) Received: from emss01g01.ems.lmco.com ([129.197.181.54]) by mailgw3a.lmco.com (8.8.8/8.8.8) with ESMTP id OAA08009; Mon, 24 Sep 2001 14:18:52 -0400 (EDT) Received: from CONVERSION-DAEMON by lmco.com (PMDF V5.2-32 #38886) id <0GK600M01EMUPV@lmco.com>; Mon, 24 Sep 2001 11:18:47 -0700 (PDT) Received: from cui1.lmms.lmco.com ([129.197.1.64]) by lmco.com (PMDF V5.2-32 #38886) with ESMTP id <0GK600CF0H1I5T@lmco.com>; Mon, 24 Sep 2001 10:39:19 -0700 (PDT) Received: from lmco.com (CONNECTICUT1.lmms.lmco.com [129.197.23.84]) by cui1.lmms.lmco.com (8.11.0/8.9.2) with ESMTP id f8OHdH618267; Mon, 24 Sep 2001 10:39:18 -0700 (PDT) Date: Mon, 24 Sep 2001 10:39:19 -0700 From: rick norman Subject: Re: ipfw and dummynet To: Luigi Rizzo Cc: freebsd-hackers@freebsd.org Message-id: <3BAF6FC7.6E548D65@lmco.com> MIME-version: 1.0 X-Mailer: Mozilla 4.77 [en] (WinNT; U) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT X-Accept-Language: en References: <200109220404.GAA64667@info.iet.unipi.it> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thanks for the responses, as expected it was an operator head space problem. My lack of understanding how the default queues and bw would make ping look. Apparently, enough delay is introduced merely by adding a pipe that the ping client timesout waiting for the reponse. The response was actually returning which became visible when I upped the timeout. I also didn't realize that the counters reflected the input to the pipe and not the output which was why I didn't see any change when I added a bw clamp. Luigi Rizzo wrote: > hi, > > can you show me the output of > ipfw show > and > ipfw pipe show > > Reading your questions, i have the feeling you are doing > something wrong in the commands. > For the last one, the client will keep generating its stream > of data, it is just after going through the pipe that you will > see the limitation in effect. > > cheers > luigi > > ----------------------------------+----------------------------------------- > Luigi RIZZO, luigi@iet.unipi.it . ACIRI/ICSI (on leave from Univ. di Pisa) > http://www.iet.unipi.it/~luigi/ . 1947 Center St, Berkeley CA 94704 > Phone (510) 666 2927 . > ----------------------------------+----------------------------------------- > > > I tried "questions" for this but no answer. > > > > I am attempting to use ipfw and dummynet to instrument some network > > traffic tests. I am running freebsd 4.3 release and have built the > > kernel > > with ipfirewall, dummynet, and default to enabled. For a simple test, I > > > > added a pipe "ipfw add pipe 1 icmp from any to any". When I ping this > > machine, I can do "ipfw pipe 1 show" and watch the counters increment, > > but the machine doing the pinging does not see a response to the ping. > > That's > > my first question. Next, when I try to delete the pipe, "ipfw pipe 1 > > delete", it > > won't delete. The only way I can get rid of it is to do a flush. That's > > > > the second > > question. Third question, if I type "ipfw pipe 1 config bw 10Bytes/s", > > I would > > expect the bw to be limited and the counters to reflect this limit. The > > > > counters > > indicate no change in the 64 byte/s generated by my windows client. > > > > I have read the man pages for ipfw, dummynet, and ipfirewall. If these > > are > > obvious questions, I would appreciate a pointer to a good reference. > > > > Thanks > > > > > > > > -- > > > > Logically speaking, logic is not the answer. > > > > Rick Norman > > rick.norman@lmco.com > > 408 742 1619 > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-hackers" in the body of the message > > -- Logically speaking, logic is not the answer. Rick Norman rick.norman@lmco.com 408 742 1619 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 11:30:58 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from seven.Alameda.net (seven.Alameda.net [64.81.63.137]) by hub.freebsd.org (Postfix) with ESMTP id 6EF0B37B40B for ; Mon, 24 Sep 2001 11:30:54 -0700 (PDT) Received: by seven.Alameda.net (Postfix, from userid 1000) id 42D6F3A23E; Mon, 24 Sep 2001 11:30:50 -0700 (PDT) Date: Mon, 24 Sep 2001 11:30:50 -0700 From: Ulf Zimmermann To: hackers@FreeBSD.org Subject: CVSup4.Freebsd.org Message-ID: <20010924113050.U2286@seven.alameda.net> Reply-To: ulf@Alameda.net Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Organization: Alameda Networks, Inc. X-Operating-System: FreeBSD 4.3-STABLE Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Seems to have still S1G bug: Connected to cvsup4.freebsd.org Server cvsup4.freebsd.org has the S1G bug -- Regards, Ulf. --------------------------------------------------------------------- Ulf Zimmermann, 1525 Pacific Ave., Alameda, CA-94501, #: 510-865-0204 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 11:38:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from hex.databits.net (hex.databits.net [207.29.192.16]) by hub.freebsd.org (Postfix) with SMTP id DAC9637B419 for ; Mon, 24 Sep 2001 11:38:08 -0700 (PDT) Received: (qmail 33402 invoked by uid 1001); 24 Sep 2001 18:38:03 -0000 Date: Mon, 24 Sep 2001 14:38:03 -0400 From: Pete Fritchman To: Ulf Zimmermann Cc: hackers@FreeBSD.org, rgrimes@FreeBSD.org Subject: Re: CVSup4.Freebsd.org Message-ID: <20010924143803.A32730@databits.net> References: <20010924113050.U2286@seven.alameda.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010924113050.U2286@seven.alameda.net>; from ulf@Alameda.net on Mon, Sep 24, 2001 at 11:30:50AM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ++ 24/09/01 11:30 -0700 - Ulf Zimmermann: | Seems to have still S1G bug: | | Connected to cvsup4.freebsd.org | Server cvsup4.freebsd.org has the S1G bug This should go to the maintainer of cvsup4.freebsd.org, available at: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvsup.html#CVSUP-MIRRORS And also CC:'d. -pete -- Pete Fritchman [petef@(databits.net|freebsd.org|csh.rit.edu)] finger petef@databits.net for PGP key To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 12:14:19 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 5025637B40B for ; Mon, 24 Sep 2001 12:14:05 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8OJE4l95477; Mon, 24 Sep 2001 12:14:04 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 12:14:04 -0700 (PDT) From: Matt Dillon Message-Id: <200109241914.f8OJE4l95477@earth.backplane.com> To: hackers@freebsd.org Cc: Alfred Perlstein , Bruce Evans , Poul-Henning Kamp , Julian Elischer Subject: VM Corruption - stumped, anyone have any ideas? Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG A number of people have been seeing these on STABLE: panic: vm_page_remove(): page not found in hash They appear to be reproducable after a random period of time on certain machines. I tracked the problem down to corruption in the vm_page_array but I cannot figure out what the cause of the corruption is. I would appreciate it if people could look at the following structural and hex dump of a corrupted vm_page_t. Does any recognize the subsystem the data is coming from? This is stumping me, and it appears to be rather serious. I can't reproduce it myself. The only other hint I have is from Mike Tancsa's messing around... when he bumps up the number of Apache children forked the problem appears to be easier to trigger. This also occurs on some Yahoo boxes. I don't think it's bad memory. -Matt $8 = 58630 (kgdb) print vm_page_buckets[$8] $9 = (struct vm_page *) 0xc08428cc (kgdb) print *vm_page_buckets[$8] $10 = {pageq = {tqe_next = 0xd715000, tqe_prev = 0x1}, hnext = 0xc0e26a34, listq = {tqe_next = 0xc0e26a3c, tqe_prev = 0xb00000}, object = 0x10015, pindex = 0, phys_addr = 255, md = {pv_list_count = -1066105816, pv_list = { tqh_first = 0xc09616b0, tqh_last = 0x0}}, queue = 0, flags = 0, pc = 5820, wire_count = 49302, hold_count = 9152, act_count = 151 '\227', busy = 214 '\xd6', valid = 1 '\001', dirty = 0 '\000'} tqe_prev is garbage. phys_addr is garbage. It's almost all garbage. The question is: how did it become garbage? The vm_page_t is a valid page in the preallocated vm_page_array[]. The VM system is physically incapable of corrupting a vm_page_t this badly. (kgdb) print vm_page_array_size $16 = 130743 (kgdb) print m $17 = 0xc0842acc (kgdb) print m - vm_page_array $18 = 55069 (kgdb) print &vm_page_array[55069] $19 = (struct vm_page *) 0xc0842acc (kgdb) 0xc08428cc: 0x0d715000 0x00000001 0xc0e26a34 0xc0e26a3c 0xc08428dc: 0x00b00000 0x00010015 0x00000000 0x000000ff 0xc08428ec: 0xc0748428 0xc09616b0 0x00000000 0x00000000 0xc08428fc: 0xc09616bc 0xd69723c0 0x00000001 0x0d716000 0xc084290c: 0x00000000 0x00000000 0xc0842910 0x00800022 0xc084291c: 0x00000016 0x00050000 0x000000ff 0xc0909564 0xc084292c: 0xc0921aec 0x00000000 0x00000000 0xd696aaf8 0xc084293c: 0xd696aae0 0x00000000 0x0d717000 0x00000000 0xc084294c: 0x00000000 0xc084294c 0x00800022 0x00000017 0xc084295c: 0x00050000 0x000000ff 0xc08d3e64 0xc0b6fde4 0xc084296c: 0x00000000 0xc0998430 0xd706ea98 0x00000000 0xc084297c: 0x00000010 0x0d718000 0x00000000 0x00000000 0xc084298c: 0xc0842988 0x00c00019 0x00000018 0x00050000 0xc084299c: 0x00000000 0xc09d9f5c 0xc0691764 0x00000000 0xc08429ac: 0xc083244c 0xc0848fa0 0xc02a6740 0x000001db -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 12:34:26 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.wrs.com (unknown-1-11.windriver.com [147.11.1.11]) by hub.freebsd.org (Postfix) with ESMTP id 012C437B421 for ; Mon, 24 Sep 2001 12:34:09 -0700 (PDT) Received: from laptop.baldwin.cx ([147.11.46.209]) by mail.wrs.com (8.9.3/8.9.1) with ESMTP id MAA02126; Mon, 24 Sep 2001 12:34:00 -0700 (PDT) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200109231559.f8NFxeC13760@csa.bu.edu> Date: Mon, 24 Sep 2001 12:33:43 -0700 (PDT) From: John Baldwin To: Evan Sarmiento Subject: RE: panic on mount Cc: freebsd-hackers@FreeBSD.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 23-Sep-01 Evan Sarmiento wrote: > Hello, > > After compiling a new kernel, installing it, when my laptop > tries to mount its drive, it panics with this message: > > panic: lock (sleep mutex) vnode interlock not locked @ > ../../../kern/vfs_default.c:460 > > which is: > > if (ap->a_flags & LK_INTERLOCK) > mtx_unlock(&ap->a_vp->v_interlock); > > within the function vop_nolock. Can you get a stack trace to see where vop_nolock is being called from? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 13:54:41 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mailsrv.otenet.gr (mailsrv.otenet.gr [195.170.0.5]) by hub.freebsd.org (Postfix) with ESMTP id 65D6937B410 for ; Mon, 24 Sep 2001 13:54:21 -0700 (PDT) Received: from hades.hell.gr (patr530-a188.otenet.gr [212.205.215.188]) by mailsrv.otenet.gr (8.11.5/8.11.5) with ESMTP id f8OKsHc08629 for ; Mon, 24 Sep 2001 23:54:17 +0300 (EEST) Received: (from charon@localhost) by hades.hell.gr (8.11.6/8.11.6) id f8OKgkD14589 for hackers@freebsd.org; Mon, 24 Sep 2001 23:42:46 +0300 (EEST) (envelope-from charon@labs.gr) Date: Mon, 24 Sep 2001 23:42:46 +0300 From: Giorgos Keramidas To: hackers@freebsd.org Subject: termcap sources Message-ID: <20010924234245.B14545@hades.hell.gr> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="5vNYLRcllDrimb99" Content-Disposition: inline User-Agent: Mutt/1.3.22.1i X-GPG-Fingerprint: C1EB 0653 DB8B A557 3829 00F9 D60F 941A 3186 03B6 X-URL: http://labs.gr/~charon/ Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --5vNYLRcllDrimb99 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I saw a duplicate in one of the capabilities that wer submitted to -bugs earlier. This had me thinking. What happens when a duplicate capability exists in termcap? Are there any other duplicates in termcap.src? If yes, which? The first attachment is a perl script that strips all cruft from termcap.src (fed through stdin) and makes every terminal entry occupy a single line of text. This is necessary for the second attachment to work correctly. The second attachment is a perl script that splits capability names of each input line in an array, and performs the (boring for a human to do by reading through the termcap sources) duplicate check in all the elements of the array. The third attachment is the output of the command (termcap.src,v 1.109): % cat termcap.src | ./tstrip.pl | ./tdupcheck.pl As you can see there are quite a few terminals that have capabilities defined more than once! I don't have THAT many terminals to check, but I'm open to suggestions. Should we do something about this? If yes, what? -giorgos --5vNYLRcllDrimb99 Content-Type: application/x-perl Content-Disposition: attachment; filename="tstrip.pl" Content-Transfer-Encoding: quoted-printable #!/usr/bin/perl -w=0A=0A#=0A# $Id$=0A#=0A# Strip comments from termcap file= s. Also join all continuation lines (those=0A# ending with backslash) in o= ne long line of text. Use this to prepare input=0A# for the tdupcheck.pl p= rogram (the duplicate capability checker).=0A=0A$buf =3D "";=0A=0ALINE:=0Aw= hile (defined($line =3D )) {=0A chomp $line;=0A=0A # skip empt= y and comment lines=0A next LINE if ($line =3D~ m/^[\s]*$/ || $line =3D~= m/^[\s]*#/);=0A=0A # terminal continuation lines (ending in backslash)= =0A if ($line =3D~ m/^[\s]+.*\\$/) {=0A $line =3D~ s/^[\s]+//;=0A= $line =3D~ s/^://;=0A $line =3D~ s/\\$//;=0A $buf .= =3D $line;=0A next LINE;=0A }=0A=0A # terminal continuation li= nes (final of a terminal entry)=0A if ($line =3D~ m/^[\s]+/) {=0A = $line =3D~ s/^[\s]+//;=0A $line =3D~ s/^://;=0A $buf .=3D $l= ine;=0A next LINE;=0A }=0A=0A # terminal entry starting line= =0A # print collected terminal entry in $buf, and start all over=0A i= f (defined($buf)) {=0A print "$buf\n";=0A $line =3D~ s/\\$//;= =0A $buf =3D $line;=0A next LINE;=0A }=0A}=0A=0A# print an= y left-over contents of the output buffer=0Aif (defined($buf)) {=0A prin= t "$buf\n";=0A}=0A --5vNYLRcllDrimb99 Content-Type: application/x-perl Content-Disposition: attachment; filename="tdupcheck.pl" Content-Transfer-Encoding: quoted-printable #!/usr/bin/perl -w=0A=0A#=0A# $Id$=0A#=0A# Check for duplicate capabilities= in terminal entries fed to us through the=0A# standard input. Feed the te= rmcap source to tstrip.pl before giving it to=0A# this program, for example= :=0A#=0A# % cat /usr/src/share/termcap/termcap.src | ./tstrip.pl | ./tdup= check.pl=0A#=0A=0ALINE:=0Awhile (defined($line =3D )) {=0A chomp = $line;=0A=0A # protect "\:" from being used in split() as a separator.= =0A $line =3D~ s/\\:/\001/g;=0A @cap =3D split /:/,$line;=0A=0A # = cleanup all @cap members from capability values.=0A # we only care for t= heir names.=0A for ($c =3D 0; $c <=3D $#cap; $c++) {=0A $cap[$c] = =3D~ s/=3D.*//;=0A $cap[$c] =3D~ s/#.*//;=0A }=0A=0A # do the = (slow) check for duplicates=0A if ($#cap >=3D 0) {=0A $term =3D $= cap[0];=0A $term =3D~ s/\|.*$//;=0A shift @cap;=0A @du= ps =3D ();=0A for ($c =3D 0; $c <=3D $#cap; $c++) {=0A fo= r ($i =3D $c + 1; $i <=3D $#cap; $i++) {=0A if ($cap[$c] eq = $cap[$i]) {=0A push @dups, $cap[$c];=0A }= =0A }=0A }=0A if ($#dups >=3D 0) {=0A p= rintf "$term - " . join(":",@dups) . "\n";=0A }=0A }=0A}=0A --5vNYLRcllDrimb99 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="dups.txt" q101 - cl 5410 - k4 AT386 - IC h1510 - do h1520 - do ibm3163 - ds:es:fs:hs abm80 - do d132 - ic tec400 - do f200 - ds:ts sol - ho it2 - do mdl110 - cd wsiris - cl:ho intext - le c100 - us:ue dtterm - op h29 - do z29a - mb:mr ztx - sr adm5 - do mime - do sexidy - le ttyWilliams - do tvi950 - do tvi955 - do abm85 - kd fos - bs --5vNYLRcllDrimb99-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 14:13:48 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 97DAE37B422 for ; Mon, 24 Sep 2001 14:13:45 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8OLDbV96558; Mon, 24 Sep 2001 14:13:37 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 14:13:37 -0700 (PDT) From: Matt Dillon Message-Id: <200109242113.f8OLDbV96558@earth.backplane.com> To: Ian Dowse Cc: hackers@freebsd.org Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <200109242144.aa79348@salmon.maths.tcd.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :In message <200109241914.f8OJE4l95477@earth.backplane.com>, Matt Dillon writes: :> :>$8 = 58630 :>(kgdb) print vm_page_buckets[$8] : :What is vm_page_hash_mask? The chunk of memory you printed out below :looks alright; it is consistent with vm_page_array == 0xc051c000. Is :it just the vm_page_buckets[] pointer that is corrupt? : :The address 0xc08428cc is (char *)&vm_page_array[55060] + 28, and :sizeof(struct vm_page) is 60, so 0xc08428cc is in the middle of :a vm_page within vm_page_array[]. : :Ian (kgdb) print vm_page_buckets[58630] $5 = (struct vm_page *) 0xc08428cc (kgdb) print vm_page_array $6 = 0xc051c000 (kgdb) print vm_page_hash_mask $7 = 262143 (kgdb) print &vm_page_array[55060] $11 = (struct vm_page *) 0xc08428b0 (kgdb) print &vm_page_array[55061] $10 = (struct vm_page *) 0xc08428ec Yowzer. How the hell did that happen! Yes, you're right, the vm_page_array[] pointer has gotten corrupted. If we assume that the vm_page_t is valid (0xc0842acc), then the vm_page_buckets[] pointer should be that. vm_page_buckets[58630] -> c08428cc panic on vm_page_t m -> c0842acc Ok, so the corruption here is that an 'a' turned into an '8'. 1010 turned into 1000... a bit got cleared. This is very similar to the corruption I found on one of Yahoo's machines. Except on that machine two bits were changed. It's as though some other subsystem is trying to manipulate a flag in a structure using a bad structure pointer. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 14:41:17 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mimer.webgiro.com (mailer2.webgiro.com [213.162.131.18]) by hub.freebsd.org (Postfix) with ESMTP id 3EAD937B412 for ; Mon, 24 Sep 2001 14:41:08 -0700 (PDT) Received: from webgiro.com (mailer2.webgiro.com [213.162.131.18]) by mimer.webgiro.com (Postfix) with ESMTP id E570A68534; Mon, 24 Sep 2001 23:41:04 +0200 (CEST) Message-ID: <3BAFA7CD.28E8C10B@webgiro.com> Date: Mon, 24 Sep 2001 23:38:21 +0200 From: Andrzej Bialecki Organization: WebGiro AB X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Peter Wullinger Cc: freebsd-hackers@freebsd.org Subject: Re: where to set hw.ata.wc References: <3BA77880.91744255@calcon.net> <20010919091313.A27353@rfhpc8320.fh-regensburg.de> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Wullinger wrote: > > While at the topic: > > Anybody yet thought about a mountable filesystem for sysctl() values? > > Would be like Linux-procfs, but a lot cleaner (wth has pci/ to do with /proc?) You are late... :-) Please see the archives of freebsd-arch (some 3-4 months ago). The conclusion was: there are no obvious benefits from having this as an fs. Personally, I don't fully agree, but I don't have enough time to implement it myself... -- Andrzej // ---------------------------------------------------------------- // Andrzej Bialecki , Chief System Architect // WebGiro AB, Sweden (http://www.webgiro.com) // ---------------------------------------------------------------- // FreeBSD developer (http://www.freebsd.org) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 15: 8:14 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id A31DC37B414 for ; Mon, 24 Sep 2001 15:08:10 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA51785; Mon, 24 Sep 2001 15:44:16 -0700 (PDT) Date: Mon, 24 Sep 2001 15:44:15 -0700 (PDT) From: Julian Elischer To: Matt Dillon Cc: Ian Dowse , hackers@freebsd.org Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109242113.f8OLDbV96558@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG remember that we hit almost this problem with the KSE stuff during debugging? The pointers in the last few entries of the vm_page_buckets array got corrupted when an agument to a function that manipulated whatever was next in ram was 0, and it turned out that it was 0 because of some PTE flushing thing (you are the one that found it... remember?) (there was a line of asm code missing) On Mon, 24 Sep 2001, Matt Dillon wrote: > > : > :In message <200109241914.f8OJE4l95477@earth.backplane.com>, Matt Dillon writes: > :> > :>$8 = 58630 > :>(kgdb) print vm_page_buckets[$8] > : > :What is vm_page_hash_mask? The chunk of memory you printed out below > :looks alright; it is consistent with vm_page_array == 0xc051c000. Is > :it just the vm_page_buckets[] pointer that is corrupt? > : > :The address 0xc08428cc is (char *)&vm_page_array[55060] + 28, and > :sizeof(struct vm_page) is 60, so 0xc08428cc is in the middle of > :a vm_page within vm_page_array[]. > : > :Ian > > (kgdb) print vm_page_buckets[58630] > $5 = (struct vm_page *) 0xc08428cc > (kgdb) print vm_page_array > $6 = 0xc051c000 > (kgdb) print vm_page_hash_mask > $7 = 262143 > (kgdb) print &vm_page_array[55060] > $11 = (struct vm_page *) 0xc08428b0 > (kgdb) print &vm_page_array[55061] > $10 = (struct vm_page *) 0xc08428ec > > Yowzer. How the hell did that happen! Yes, you're right, the > vm_page_array[] pointer has gotten corrupted. If we assume that > the vm_page_t is valid (0xc0842acc), then the vm_page_buckets[] > pointer should be that. > > vm_page_buckets[58630] -> c08428cc > panic on vm_page_t m -> c0842acc > > Ok, so the corruption here is that an 'a' turned into an '8'. 1010 turned > into 1000... a bit got cleared. > > This is very similar to the corruption I found on one of Yahoo's > machines. Except on that machine two bits were changed. It's as though > some other subsystem is trying to manipulate a flag in a structure using > a bad structure pointer. > > -Matt > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 15:19:42 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from winston.freebsd.org (adsl-64-173-15-98.dsl.sntc01.pacbell.net [64.173.15.98]) by hub.freebsd.org (Postfix) with ESMTP id 7F54437B429 for ; Mon, 24 Sep 2001 15:19:39 -0700 (PDT) Received: from localhost (jkh@localhost [127.0.0.1]) by winston.freebsd.org (8.11.6/8.11.6) with ESMTP id f8OMJ2B04369; Mon, 24 Sep 2001 15:19:02 -0700 (PDT) (envelope-from jkh@freebsd.org) To: jcv@vbc.net Cc: wertejuk@mwcis.com, freebsd-hackers@freebsd.org Subject: Re: Boot proccess In-Reply-To: References: <20010920000212.B60704@localhost.com> X-Mailer: Mew version 1.94.1 on Emacs 20.7 / Mule 4.0 (HANANOEN) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20010924151902K.jkh@freebsd.org> Date: Mon, 24 Sep 2001 15:19:02 -0700 From: Jordan Hubbard X-Dispatcher: imput version 20000228(IM140) Lines: 16 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > Tell me if I am wrong but from the floppy, the files kern.flp & > mfsroot.flp are compressed and then uncompressed into memory. > > If so, that means that the FreeBSD box is running this programs from the > RAM and not from the floppy, right ? Correct. They're running with the root device set to a memory filesystem (which has been initialized with the contents of mfsroot.flp). > If so, is it possible to do the same but from a hard disk instead of the > floppy ? That's generally how most FreeBSD systems boot, yes. :-) - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 16: 1:15 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by hub.freebsd.org (Postfix) with SMTP id 364F137B425 for ; Mon, 24 Sep 2001 16:01:12 -0700 (PDT) Received: from walton.maths.tcd.ie by salmon.maths.tcd.ie with SMTP id ; 25 Sep 2001 00:01:11 +0100 (BST) To: Julian Elischer Cc: Matt Dillon , hackers@freebsd.org Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: Your message of "Mon, 24 Sep 2001 15:44:15 PDT." Date: Tue, 25 Sep 2001 00:01:10 +0100 From: Ian Dowse Message-ID: <200109250001.aa92227@salmon.maths.tcd.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > >The pointers in the last few entries of the vm_page_buckets array got >corrupted when an agument to a function that manipulated whatever was next >in ram was 0, and it turned out that it was 0 because > of some PTE flushing thing (you are the one that found it... remember?) I think I've also seen a few reports of programs exiting with "Profiling timer expired" messages with 4.4. These can be caused by stack overflows, since the p_timer[] array in struct pstats is one of the things that I think lives below the per-process kernel stack. I wonder if they are related? Stack overflows could result in corruption of local variables, after which anything could happen. That said, hardware problems are still a possiblilty. Ian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 16:23:10 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 3391437B40B for ; Mon, 24 Sep 2001 16:23:08 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8ONMaT97469; Mon, 24 Sep 2001 16:22:36 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 16:22:36 -0700 (PDT) From: Matt Dillon Message-Id: <200109242322.f8ONMaT97469@earth.backplane.com> To: Ian Dowse Cc: Julian Elischer , hackers@freebsd.org Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <200109250001.aa92227@salmon.maths.tcd.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :>The pointers in the last few entries of the vm_page_buckets array got :>corrupted when an agument to a function that manipulated whatever was next :>in ram was 0, and it turned out that it was 0 because :> of some PTE flushing thing (you are the one that found it... remember?) : :I think I've also seen a few reports of programs exiting with :"Profiling timer expired" messages with 4.4. These can be caused :by stack overflows, since the p_timer[] array in struct pstats is :one of the things that I think lives below the per-process kernel :stack. I wonder if they are related? Stack overflows could result :in corruption of local variables, after which anything could happen. : :That said, hardware problems are still a possiblilty. : :Ian Hmm. Do we have a guard page at the base of the per process kernel stack? -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 16:26: 3 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id D62F837B427 for ; Mon, 24 Sep 2001 16:26:00 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8ONPTE97519; Mon, 24 Sep 2001 16:25:29 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 16:25:29 -0700 (PDT) From: Matt Dillon Message-Id: <200109242325.f8ONPTE97519@earth.backplane.com> To: Julian Elischer Cc: Ian Dowse , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :remember that we hit almost this problem with the KSE stuff during :debugging? : :The pointers in the last few entries of the vm_page_buckets array got :corrupted when an agument to a function that manipulated whatever was next :in ram was 0, and it turned out that it was 0 because : of some PTE flushing thing (you are the one that found it... remember?) :(there was a line of asm code missing) I've kept that in mind, but I think this may be a different issue. The memory involved is 100% statically mapped in the kernel page table array, and the errors are more like bit errors then anything else. Either the memory is bad or something in our kernel is setting or clearing flags through a bad pointer. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 16:30:16 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from smtp013.mail.yahoo.com (smtp013.mail.yahoo.com [216.136.173.57]) by hub.freebsd.org (Postfix) with SMTP id 3EEAD37B445 for ; Mon, 24 Sep 2001 16:30:08 -0700 (PDT) Received: from mkc-65-30-96-67.kc.rr.com (HELO yahoo.com) (65.30.96.67) by smtp.mail.vip.sc5.yahoo.com with SMTP; 24 Sep 2001 23:30:07 -0000 X-Apparently-From: Message-ID: <3BAF1A99.9030803@yahoo.com> Date: Mon, 24 Sep 2001 06:35:53 -0500 From: Jim Bryant Reply-To: kc5vdj@yahoo.com User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: freebsd-current@freebsd.org Cc: freebsd-hackers@freebsd.org Subject: Change request: move mt(1) to /sbin with -static compile Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG After having a recent -current issue requiring a restore from tape, I found that both dump and restore are statically linked and available in /sbin, as they should be, given their status as recovery tools. One unfortunate problem exists with this scenario... mt(1) resides in /usr/bin, and on top of that, is linked against shared libraries. Since most people do their backups using the norewind device, and preferably on as few tapes as possible, this means that to get to the next filesystem to restore you must perform a `mt fsf n` where 'n' is the number of filesets to skip on the tape. I would like to request that mt(1) be statically compiled and moved to /sbin, where dump and restore are located, in order to facilitate the use of restore as a recovery tool, especially in cases where /usr may be corrupted or contain invalid libraries or some other bogosity, allowing mt to be used in single-user mode, and assuming that /usr may have just been newfs'ed... IMHO, moving it to /bin [as it currently resides in /usr/bin] would be a questionable move, given that it's primary use of positioning the tape for programs such as dump and restore would indicate it should go into /sbin. I've heard all of the arguments against root bloat, but this is a tool that belongs inherently with the rest of the system recovery tools. On -current, the sizing info is as follows: 6:23:18am wahoo(16): size /sbin/mt text data bss dec hex filename 65896 5216 2604 73716 11ff4 /sbin/mt 6:31:36am wahoo(17): ls -l /sbin/mt -rwxr-xr-x 1 root wheel 251045 Sep 13 08:52 /sbin/mt Also, to head off any possible arguments against using tar and pax as examples "those are in /usr", I must say that tar and pax are not the official BSD recovery utilities, dump and restore are. jim -- ET has one helluva sense of humor! He's always anal-probing right-wing schizos! ----------------------------------------------------- POWER TO THE PEOPLE! ----------------------------------------------------- "Religious fundamentalism is the biggest threat to international security that exists today." United Nations Secretary General B.B.Ghali _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 16:48: 8 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 9B3EF37B407 for ; Mon, 24 Sep 2001 16:48:03 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA52161; Mon, 24 Sep 2001 17:27:41 -0700 (PDT) Date: Mon, 24 Sep 2001 17:27:40 -0700 (PDT) From: Julian Elischer To: Matt Dillon Cc: Ian Dowse , hackers@freebsd.org Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109242322.f8ONMaT97469@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG not, I believe in 4.x we do in 5.x On Mon, 24 Sep 2001, Matt Dillon wrote: > > :>The pointers in the last few entries of the vm_page_buckets array got > :>corrupted when an agument to a function that manipulated whatever was next > :>in ram was 0, and it turned out that it was 0 because > :> of some PTE flushing thing (you are the one that found it... remember?) > : > :I think I've also seen a few reports of programs exiting with > :"Profiling timer expired" messages with 4.4. These can be caused > :by stack overflows, since the p_timer[] array in struct pstats is > :one of the things that I think lives below the per-process kernel > :stack. I wonder if they are related? Stack overflows could result > :in corruption of local variables, after which anything could happen. > : > :That said, hardware problems are still a possiblilty. > : > :Ian > > Hmm. Do we have a guard page at the base of the per process kernel > stack? > > -Matt > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 16:49: 5 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by hub.freebsd.org (Postfix) with SMTP id C760037B421 for ; Mon, 24 Sep 2001 16:48:58 -0700 (PDT) Received: from walton.maths.tcd.ie by salmon.maths.tcd.ie with SMTP id ; 25 Sep 2001 00:48:57 +0100 (BST) To: Matt Dillon Cc: Julian Elischer , hackers@freebsd.org Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: Your message of "Mon, 24 Sep 2001 16:22:36 PDT." <200109242322.f8ONMaT97469@earth.backplane.com> Date: Tue, 25 Sep 2001 00:48:57 +0100 From: Ian Dowse Message-ID: <200109250048.aa97220@salmon.maths.tcd.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200109242322.f8ONMaT97469@earth.backplane.com>, Matt Dillon writes: > > Hmm. Do we have a guard page at the base of the per process kernel > stack? As I understand it, no. In RELENG_4 there are UPAGES (== 2 on i386) pages of per-process kernel state at p->p_addr. The stack grows down from the top, and struct user (sys/user.h) sits at the bottom. According to the comment in the definition of struct user, only the first three items in struct user are valid in normal running conditions: 8192 ??? 8176 <---- Top of stack stack space (4672 bytes) 3504 struct timeval p_start struct uprof p_prof struct itimerval p_timer[ITIMER_PROF] (for SIGPROF) struct itimerval p_timer[ITIMER_VIRTUAL] struct itimerval p_timer[ITIMER_REAL] struct rusage p_cru; struct rusage p_ru; u_stats 3280 u_sigacts 608 u_pcb 0 <---- p->p_addr So if the stack does overflow, p_timer[ITIMER_PROF] is about the first noticable thing that gets clobbered, causing a SIGPROF signal delivery to the process some time later. Ian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 17:27:36 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id AF21737B418 for ; Mon, 24 Sep 2001 17:27:34 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8P0RRk97980; Mon, 24 Sep 2001 17:27:27 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 17:27:27 -0700 (PDT) From: Matt Dillon Message-Id: <200109250027.f8P0RRk97980@earth.backplane.com> To: Ian Dowse Cc: Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <200109250048.aa97220@salmon.maths.tcd.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :In message <200109242322.f8ONMaT97469@earth.backplane.com>, Matt Dillon writes: :> :> Hmm. Do we have a guard page at the base of the per process kernel :> stack? : :As I understand it, no. In RELENG_4 there are UPAGES (== 2 on i386) :pages of per-process kernel state at p->p_addr. The stack grows :down from the top, and struct user (sys/user.h) sits at the bottom. :According to the comment in the definition of struct user, only :the first three items in struct user are valid in normal running :conditions: Ok. I'm going to add a magic number to the end of the process structure and check it in mi_switch() in -stable. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 17:31: 5 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 457AF37B40B for ; Mon, 24 Sep 2001 17:31:03 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8P0Uv298004; Mon, 24 Sep 2001 17:30:57 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 17:30:57 -0700 (PDT) From: Matt Dillon Message-Id: <200109250030.f8P0Uv298004@earth.backplane.com> To: Ian Dowse Cc: Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <200109250048.aa97220@salmon.maths.tcd.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :In message <200109242322.f8ONMaT97469@earth.backplane.com>, Matt Dillon writes: :> :> Hmm. Do we have a guard page at the base of the per process kernel :> stack? : :As I understand it, no. In RELENG_4 there are UPAGES (== 2 on i386) :pages of per-process kernel state at p->p_addr. The stack grows :down from the top, and struct user (sys/user.h) sits at the bottom. :According to the comment in the definition of struct user, only :the first three items in struct user are valid in normal running :conditions: Er, I mean I'll add a magic number to struct pstats. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 17:37:23 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 95E3937B414; Mon, 24 Sep 2001 17:37:17 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id UAA17186; Mon, 24 Sep 2001 20:37:06 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8P0ae171341; Mon, 24 Sep 2001 20:36:40 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15279.53656.28808.586330@grasshopper.cs.duke.edu> Date: Mon, 24 Sep 2001 20:36:40 -0400 (EDT) To: kc5vdj@yahoo.com Cc: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: Change request: move mt(1) to /sbin with -static compile In-Reply-To: <3BAF1A99.9030803@yahoo.com> References: <3BAF1A99.9030803@yahoo.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jim Bryant writes: > After having a recent -current issue requiring a restore from tape, I found that both dump and restore are statically linked and > available in /sbin, as they should be, given their status as recovery tools. > > One unfortunate problem exists with this scenario... > > mt(1) resides in /usr/bin, and on top of that, is linked against shared libraries. > > Since most people do their backups using the norewind device, and preferably on as few tapes as possible, this means that to get to > the next filesystem to restore you must perform a `mt fsf n` where 'n' is the number of filesets to skip on the tape. <..> Err.. Can't you use the -s flag to restore? Eg: -s fileno Read from the specified fileno on a multi-file tape. File num- bering starts at 1. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 17:43:33 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id EAA1C37B409 for ; Mon, 24 Sep 2001 17:43:23 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id UAA17331 for ; Mon, 24 Sep 2001 20:43:19 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8P0grH71350; Mon, 24 Sep 2001 20:42:53 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15279.54029.454089.299807@grasshopper.cs.duke.edu> Date: Mon, 24 Sep 2001 20:42:53 -0400 (EDT) To: freebsd-hackers@freebsd.org Subject: ecc on i386 X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG What happens on an ECC equipped PC when you have a multi-bit memory error that hardware scrubbing can't fix? Will there be some sort of NMI or something that will panic the box? I'm used to alphas (where you'll get a fatal machine check panic) and I am just wondering if PCs are as safe. Thanks, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 17:48: 2 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id F35EA37B420 for ; Mon, 24 Sep 2001 17:47:58 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id SAA52438; Mon, 24 Sep 2001 18:27:05 -0700 (PDT) Date: Mon, 24 Sep 2001 18:27:05 -0700 (PDT) From: Julian Elischer To: Matt Dillon Cc: Ian Dowse , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109250027.f8P0RRk97980@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG stack can be somewhat sparse depending on execution path, but it's not a bad idea.. On Mon, 24 Sep 2001, Matt Dillon wrote: > :In message <200109242322.f8ONMaT97469@earth.backplane.com>, Matt Dillon writes: > :> > :> Hmm. Do we have a guard page at the base of the per process kernel > :> stack? > : > :As I understand it, no. In RELENG_4 there are UPAGES (== 2 on i386) > :pages of per-process kernel state at p->p_addr. The stack grows > :down from the top, and struct user (sys/user.h) sits at the bottom. > :According to the comment in the definition of struct user, only > :the first three items in struct user are valid in normal running > :conditions: > > Ok. I'm going to add a magic number to the end of the process > structure and check it in mi_switch() in -stable. > > -Matt > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 17:59:45 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 25A0137B409 for ; Mon, 24 Sep 2001 17:59:41 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8P0wx998146; Mon, 24 Sep 2001 17:58:59 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 17:58:59 -0700 (PDT) From: Matt Dillon Message-Id: <200109250058.f8P0wx998146@earth.backplane.com> To: Andrew Gallatin Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: ecc on i386 References: <15279.54029.454089.299807@grasshopper.cs.duke.edu> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :What happens on an ECC equipped PC when you have a multi-bit memory :error that hardware scrubbing can't fix? Will there be some sort of :NMI or something that will panic the box? : :I'm used to alphas (where you'll get a fatal machine check panic) and :I am just wondering if PCs are as safe. : :Thanks, : :Drew ECC can typically detect and correct single bit errors and detect double bit errors. Anything beyond that is problematic... it may or may not detect the problem or may mis-correct a multi-bit error. An NMI is generated if an uncorrectable error is detected. On PC's, ECC is optional. Desktops typically do not ship with ECC memory. Branded servers typically do. A year or two ago I would have been happy to use non-ECC rams (finding bad RAM through trial and error), but now with capacities as they are and memory prices down ECC is definitely the way to go. Bit errors can come from many sources, memory being only one. Bit errors can occur inside the cpu chip, in the L1 and L2 caches, in memory, in controller chips... all over the place. Many modern processors implement parity on their caches to try to cover the problem areas. I'm not sure how Pentium III's and IV's are setup. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 18:14:27 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 2CD8237B409 for ; Mon, 24 Sep 2001 18:14:19 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id VAA18072; Mon, 24 Sep 2001 21:14:08 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8P1DgW71389; Mon, 24 Sep 2001 21:13:42 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15279.55878.110154.650940@grasshopper.cs.duke.edu> Date: Mon, 24 Sep 2001 21:13:42 -0400 (EDT) To: Matt Dillon Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: ecc on i386 In-Reply-To: <200109250058.f8P0wx998146@earth.backplane.com> References: <15279.54029.454089.299807@grasshopper.cs.duke.edu> <200109250058.f8P0wx998146@earth.backplane.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matt Dillon writes: > > :What happens on an ECC equipped PC when you have a multi-bit memory > :error that hardware scrubbing can't fix? Will there be some sort of > :NMI or something that will panic the box? > : > :I'm used to alphas (where you'll get a fatal machine check panic) and > :I am just wondering if PCs are as safe. > : > :Thanks, > : > :Drew > > ECC can typically detect and correct single bit errors and detect > double bit errors. Anything beyond that is problematic... it may or > may not detect the problem or may mis-correct a multi-bit error. > An NMI is generated if an uncorrectable error is detected. > > On PC's, ECC is optional. Desktops typically do not ship with ECC > memory. Branded servers typically do. A year or two ago I would > have been happy to use non-ECC rams (finding bad RAM through trial > and error), but now with capacities as they are and memory prices down > ECC is definitely the way to go. My sentiments exactly. > Bit errors can come from many sources, memory being only one. Bit errors > can occur inside the cpu chip, in the L1 and L2 caches, in memory, in > controller chips... all over the place. Many modern processors implement > parity on their caches to try to cover the problem areas. I'm not sure > how Pentium III's and IV's are setup. > > -Matt Hmm.. Well, it turns out that the box I"m insterested in (Thunder K7) can be set to send an SERR on multiple bit errors. I wonder what happens when a pc gets an SERR? (that's another machine check on alpha) Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 18:14:54 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from glatton.cnchost.com (glatton.cnchost.com [207.155.248.47]) by hub.freebsd.org (Postfix) with ESMTP id 552A737B417 for ; Mon, 24 Sep 2001 18:14:50 -0700 (PDT) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by glatton.cnchost.com id VAA20993; Mon, 24 Sep 2001 21:14:35 -0400 (EDT) [ConcentricHost SMTP Relay 1.14] Message-ID: <200109250114.VAA20993@glatton.cnchost.com> To: Matt Dillon Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-reply-to: Your message of "Mon, 24 Sep 2001 17:27:27 PDT." <200109250027.f8P0RRk97980@earth.backplane.com> Date: Mon, 24 Sep 2001 18:14:34 -0700 From: Bakul Shah Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG FWIW, in a Unix port we did I remember putting the user struct *above* the kernel stack. The stack grew down so you hit the red zone (the guard pages) without clobbering the user struct. Since struct user _ended_ on a page boundary, its size was needed at locore.s assembly time but that was a small price to pay for the added safety. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 18:20:48 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 2A72237B428 for ; Mon, 24 Sep 2001 18:20:42 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8P1KfM20139 for ; Mon, 24 Sep 2001 18:20:41 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id CC9613808; Mon, 24 Sep 2001 18:20:41 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Andrew Gallatin Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: ecc on i386 In-Reply-To: <15279.54029.454089.299807@grasshopper.cs.duke.edu> Date: Mon, 24 Sep 2001 18:20:41 -0700 From: Peter Wemm Message-Id: <20010925012041.CC9613808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Andrew Gallatin wrote: > > What happens on an ECC equipped PC when you have a multi-bit memory > error that hardware scrubbing can't fix? Will there be some sort of > NMI or something that will panic the box? > > I'm used to alphas (where you'll get a fatal machine check panic) and > I am just wondering if PCs are as safe. Basically it depends on how the bios has programmed the chipsets and how the motherboard is wired. The usual way goes something like this: There are two PCI signals, #PERR (pci error), #SERR (system error). Various devices can be programmed to assert these under various conditions. Things like bus master fifo underflows etc will be programmed to assert #PERR and are generally not fatal. The memory controller is usually programmed to assert #SERR on a multiple bit error and either #SERR or some other signal (a GPIO or something like #SALERT on a serverworks chip) for a single bit (corrected) error. The south bridge listens to #SERR and #PERR and can convert those into NMI events. Usually #SERR shows up as "parity error" and #PERR shows up as "IOCHK" (if it is enabled). The bad news is that many bios manufacturers **TURN OFF** ECC functionality in order to speed things up. The reason for this is that with ECC off, the cpu can read/write down to byte granularity. With ECC on, memory is rigidly enforced as 64 bit quantities (ecc-encoded out to 72 bits). If the cpu reads a byte, the memory controller actually fetches all 64 (72) bits. If the cpu writes a byte, the memory controller has to do a read-merge-write cycle where it reads the 64 bit value, merges in the 1 byte write and writes out the entire 64 bit value again. This (naturally) shows up in poor benchmarks so they like to turn it off by default in order to get a speed edge. Tyan is a notable example here (eg: the Thunder K7, the dual-athlon DDR-SDRAM board has ECC turned off by default(!!)). I am sure that others do it too. The Tyan Thunder 2510 BIOS even disables ECC -> NMI routing so you have to go to quite a bit of trouble to reprogram the serverworks chipset to actually generate NMI's so that you can find out if something got trashed. Our NMI / ECC handling really really sucks in FreeBSD. Consider: - i686_pagezero - reads before writing in order to minimize cache snooping traffic in SMP systems. However, if it gets an NMI while trying to check if the cache line is already zero, it will take the entire machine down instead of just zeroing the line. - NFS / VM / bio: when they get an NMI while trying to copy data that is clean and backed by storage, they take the machine down instead of trying to recover and re-read the page. - userland.. If userland gets an NMI, the machine dies instead of killing the process (or rereading a text page etc if possible) - our NMI handlers are a festering pile of excretement. They dont have the code to 'ack' the NMI so it isn't possible to return after recovery. - and so on. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 18:23:57 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 4E99537B491 for ; Mon, 24 Sep 2001 18:23:41 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8P1NfM20153 for ; Mon, 24 Sep 2001 18:23:41 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id E7EF63808; Mon, 24 Sep 2001 18:23:40 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Andrew Gallatin Cc: Matt Dillon , freebsd-hackers@FreeBSD.ORG Subject: Re: ecc on i386 In-Reply-To: <15279.55878.110154.650940@grasshopper.cs.duke.edu> Date: Mon, 24 Sep 2001 18:23:40 -0700 From: Peter Wemm Message-Id: <20010925012340.E7EF63808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Andrew Gallatin wrote: > > Matt Dillon writes: > > > > :What happens on an ECC equipped PC when you have a multi-bit memory > > :error that hardware scrubbing can't fix? Will there be some sort of > > :NMI or something that will panic the box? > > : > > :I'm used to alphas (where you'll get a fatal machine check panic) and > > :I am just wondering if PCs are as safe. > > : > > :Thanks, > > : > > :Drew > > > > ECC can typically detect and correct single bit errors and detect > > double bit errors. Anything beyond that is problematic... it may or > > may not detect the problem or may mis-correct a multi-bit error. > > An NMI is generated if an uncorrectable error is detected. > > > > On PC's, ECC is optional. Desktops typically do not ship with ECC > > memory. Branded servers typically do. A year or two ago I would > > have been happy to use non-ECC rams (finding bad RAM through trial > > and error), but now with capacities as they are and memory prices down > > ECC is definitely the way to go. > > My sentiments exactly. I wrote a poller for picking up correction events on various serverworks motherboards (compaq, tyan) and it was *scarey* how often single-bit errors were being corrected. > > Bit errors can come from many sources, memory being only one. Bit err ors > > can occur inside the cpu chip, in the L1 and L2 caches, in memory, in > > controller chips... all over the place. Many modern processors implem ent > > parity on their caches to try to cover the problem areas. I'm not sur e > > how Pentium III's and IV's are setup. > > > > -Matt > > Hmm.. Well, it turns out that the box I"m insterested in (Thunder K7) > can be set to send an SERR on multiple bit errors. I wonder what > happens when a pc gets an SERR? (that's another machine check > on alpha) On the Thunder K7, #SERR is routed to NMI. Trust me, you want this. And set it to ECC-SCRUB instead of "off" like the default now is. See my other email about how #SERR is converted to NMI via the ISA part of the south bridge. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 18:27:47 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 9BD6B37B411 for ; Mon, 24 Sep 2001 18:27:43 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8P1RhM20183 for ; Mon, 24 Sep 2001 18:27:43 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 43E583808; Mon, 24 Sep 2001 18:27:43 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Matt Dillon Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109242322.f8ONMaT97469@earth.backplane.com> Date: Mon, 24 Sep 2001 18:27:43 -0700 From: Peter Wemm Message-Id: <20010925012743.43E583808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matt Dillon wrote: > > :>The pointers in the last few entries of the vm_page_buckets array got > :>corrupted when an agument to a function that manipulated whatever was next > :>in ram was 0, and it turned out that it was 0 because > :> of some PTE flushing thing (you are the one that found it... remember?) > : > :I think I've also seen a few reports of programs exiting with > :"Profiling timer expired" messages with 4.4. These can be caused > :by stack overflows, since the p_timer[] array in struct pstats is > :one of the things that I think lives below the per-process kernel > :stack. I wonder if they are related? Stack overflows could result > :in corruption of local variables, after which anything could happen. > : > :That said, hardware problems are still a possiblilty. > : > :Ian > > Hmm. Do we have a guard page at the base of the per process kernel > stack? > > -Matt I did it as part of the KSE work in 5.x. It would be quite easy to do it for 4.x as well, but it makes a.out coredumps problematic. Also, "options UPAGES=4" is a pretty good defensive measure. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 18:31: 9 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from smtp011.mail.yahoo.com (smtp011.mail.yahoo.com [216.136.173.31]) by hub.freebsd.org (Postfix) with SMTP id 201AA37B406 for ; Mon, 24 Sep 2001 18:31:02 -0700 (PDT) Received: from mkc-65-30-96-67.kc.rr.com (HELO yahoo.com) (65.30.96.67) by smtp.mail.vip.sc5.yahoo.com with SMTP; 25 Sep 2001 01:31:01 -0000 X-Apparently-From: Message-ID: <3BAF36F3.90706@yahoo.com> Date: Mon, 24 Sep 2001 08:36:51 -0500 From: Jim Bryant Reply-To: kc5vdj@yahoo.com User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: Andrew Gallatin Cc: freebsd-current@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: Change request: move mt(1) to /sbin with -static compile References: <3BAF1A99.9030803@yahoo.com> <15279.53656.28808.586330@grasshopper.cs.duke.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Duh! It does help to RTFM... My mistake. Andrew Gallatin wrote: > Jim Bryant writes: > > After having a recent -current issue requiring a restore from tape, I found that both dump and restore are statically linked and > > available in /sbin, as they should be, given their status as recovery tools. > > > > One unfortunate problem exists with this scenario... > > > > mt(1) resides in /usr/bin, and on top of that, is linked against shared libraries. > > > > Since most people do their backups using the norewind device, and preferably on as few tapes as possible, this means that to get to > > the next filesystem to restore you must perform a `mt fsf n` where 'n' is the number of filesets to skip on the tape. > > <..> > > Err.. Can't you use the -s flag to restore? Eg: > > -s fileno > Read from the specified fileno on a multi-file tape. File num- > bering starts at 1. > > > Drew > > jim -- ET has one helluva sense of humor! He's always anal-probing right-wing schizos! ----------------------------------------------------- POWER TO THE PEOPLE! ----------------------------------------------------- "Religious fundamentalism is the biggest threat to international security that exists today." United Nations Secretary General B.B.Ghali _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 19:41:23 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from silby.com (cb34181-a.mdsn1.wi.home.com [24.14.173.39]) by hub.freebsd.org (Postfix) with ESMTP id 7B0B937B40A for ; Mon, 24 Sep 2001 19:41:16 -0700 (PDT) Received: (qmail 71032 invoked by uid 1000); 25 Sep 2001 02:41:14 -0000 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 25 Sep 2001 02:41:14 -0000 Date: Mon, 24 Sep 2001 21:41:14 -0500 (CDT) From: Mike Silbersack To: Matt Dillon Cc: Ian Dowse , Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109242113.f8OLDbV96558@earth.backplane.com> Message-ID: <20010924213518.G70783-100000@achilles.silby.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 24 Sep 2001, Matt Dillon wrote: > Yowzer. How the hell did that happen! Yes, you're right, the > vm_page_array[] pointer has gotten corrupted. If we assume that > the vm_page_t is valid (0xc0842acc), then the vm_page_buckets[] > pointer should be that. ... > This is very similar to the corruption I found on one of Yahoo's > machines. Except on that machine two bits were changed. It's as though > some other subsystem is trying to manipulate a flag in a structure using > a bad structure pointer. > > -Matt Ok, time to take a good stab at sticking my foot in my mouth here. Would it be possible to have a kernel mode where the read-only bit was turned on for malloc pools which shouldn't currently be accessed? This could be gated through the spl() calls (or specific mutexes on -current), ensuring that something like getpid couldn't stomp on the vm structures w/o first doing a splvm(). Obviously this wouldn't help find bugs in interrupt handlers or other high level calls, but it could help locate some memory corruption problems. Actually, since memory regions roughly follow locks, this could be an even more powerful tool on -current once it develops me. Is this even feasible in ring 0? Mike "Silby" Silbersack To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 19:41:20 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 9B34137B41F for ; Mon, 24 Sep 2001 19:41:14 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8P2f2p98637; Mon, 24 Sep 2001 19:41:02 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 19:41:02 -0700 (PDT) From: Matt Dillon Message-Id: <200109250241.f8P2f2p98637@earth.backplane.com> To: Peter Wemm Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <20010925012743.43E583808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :I did it as part of the KSE work in 5.x. It would be quite easy to do it :for 4.x as well, but it makes a.out coredumps problematic. : :Also, "options UPAGES=4" is a pretty good defensive measure. : :Cheers, :-Peter :-- :Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au Well, in 4.x: (kgdb) print p->p_addr $6 = (struct user *) 0xcb7b9000 (kgdb) print &p->p_addr->u_sigacts $7 = (struct sigacts *) 0xcb7b9260 (kgdb) print &p->p_addr->u_stats $8 = (struct pstats *) 0xcb7b9cd0 (kgdb) print &p->p_addr->u_kproc $9 = (struct kinfo_proc *) 0xcb7b9db0 (kgdb) print &p->p_addr->u_md $10 = (struct md_coredump *) 0xcb7ba1d0 (kgdb) print &p->p_addr->u_guard (my new field) $11 = (u_int32_t *) 0xcb7ba1d0 (kgdb) cb7b9000 start of kstack cb7ba1d4 end of struct user cb7bb000 top of kstack Leaving us 3628 bytes for the kernel stack. Something really weird is going on... I added u_guard to the end of the struct user structure and there are two or three processes hitting the guard immediately. All the rest are ok. I'm going to investigate further but this is very odd. Am I missing something about the UAREA? -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 20:22:58 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id CC99F37B412 for ; Mon, 24 Sep 2001 20:22:50 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8P3MoM20444 for ; Mon, 24 Sep 2001 20:22:50 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 57FF03808; Mon, 24 Sep 2001 20:22:50 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Matt Dillon Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109250241.f8P2f2p98637@earth.backplane.com> Date: Mon, 24 Sep 2001 20:22:50 -0700 From: Peter Wemm Message-Id: <20010925032250.57FF03808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matt Dillon wrote: > : > :I did it as part of the KSE work in 5.x. It would be quite easy to do it > :for 4.x as well, but it makes a.out coredumps problematic. > : > :Also, "options UPAGES=4" is a pretty good defensive measure. > : > :Cheers, > :-Peter > :-- > :Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au > > Well, in 4.x: > > (kgdb) print p->p_addr > $6 = (struct user *) 0xcb7b9000 > (kgdb) print &p->p_addr->u_sigacts > $7 = (struct sigacts *) 0xcb7b9260 > (kgdb) print &p->p_addr->u_stats > $8 = (struct pstats *) 0xcb7b9cd0 > (kgdb) print &p->p_addr->u_kproc > $9 = (struct kinfo_proc *) 0xcb7b9db0 > (kgdb) print &p->p_addr->u_md > $10 = (struct md_coredump *) 0xcb7ba1d0 > (kgdb) print &p->p_addr->u_guard (my new field) > $11 = (u_int32_t *) 0xcb7ba1d0 > (kgdb) > > cb7b9000 start of kstack > cb7ba1d4 end of struct user > cb7bb000 top of kstack > > Leaving us 3628 bytes for the kernel stack. > > Something really weird is going on... I added u_guard to the end > of the struct user structure and there are two or three processes > hitting the guard immediately. All the rest are ok. I'm going > to investigate further but this is very odd. Am I missing something > about the UAREA? Yes. u_md etc isn't used while the process is running. If you're going to have u_guard, it should come directly after u_stats, and *before* u_kproc, u_md etc. I had been contemplating making a fake 'struct user' in userland only in order to keep the a.out coredump reader code happy. The a.out coredump code (see cpu_coredump() in */*/vm_machdep.c) can generate this fake structure in order to keep gdb happy. But then I realized that a.out coredump debugging was almost totally irrelevant these days. Actually I tell a lie. In 4.x, u_kproc *can* be used on a live process.. see the **NASTY** PT_READ_U and PT_WRITE_U code in sys_process.c. It does a fill_eproc() in order to be able to read/write values from there. Nothing uses this stuff. I removed it from -current quite a while ago, and it should be MFC'ed too. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 20:31:21 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id E5F3E37B42F for ; Mon, 24 Sep 2001 20:31:16 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8P3VGM20475 for ; Mon, 24 Sep 2001 20:31:16 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 6C2C23808; Mon, 24 Sep 2001 20:31:16 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Matt Dillon Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109250241.f8P2f2p98637@earth.backplane.com> Date: Mon, 24 Sep 2001 20:31:16 -0700 From: Peter Wemm Message-Id: <20010925033116.6C2C23808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matt Dillon wrote: > : > :I did it as part of the KSE work in 5.x. It would be quite easy to do it > :for 4.x as well, but it makes a.out coredumps problematic. > : > :Also, "options UPAGES=4" is a pretty good defensive measure. > : > :Cheers, > :-Peter > :-- > :Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au > > Well, in 4.x: > > (kgdb) print p->p_addr > $6 = (struct user *) 0xcb7b9000 > (kgdb) print &p->p_addr->u_sigacts > $7 = (struct sigacts *) 0xcb7b9260 > (kgdb) print &p->p_addr->u_stats > $8 = (struct pstats *) 0xcb7b9cd0 > (kgdb) print &p->p_addr->u_kproc > $9 = (struct kinfo_proc *) 0xcb7b9db0 > (kgdb) print &p->p_addr->u_md > $10 = (struct md_coredump *) 0xcb7ba1d0 > (kgdb) print &p->p_addr->u_guard (my new field) > $11 = (u_int32_t *) 0xcb7ba1d0 > (kgdb) > > cb7b9000 start of kstack > cb7ba1d4 end of struct user > cb7bb000 top of kstack > > Leaving us 3628 bytes for the kernel stack. > > Something really weird is going on... I added u_guard to the end > of the struct user structure and there are two or three processes > hitting the guard immediately. All the rest are ok. I'm going > to investigate further but this is very odd. Am I missing something > about the UAREA? Oh, one other thing... When we had PCIBIOS active for pci config space read/write support, we had stack overflows on many systems when the SSE stuff got MFC'ed. The simple act of trimming about 300 bytes from the pcb_save structure was enough to make the difference between it working or not. We are *way* too close to the wire. I asked about raising UPAGES from 2 to 3 before RELENG_4_4 but it never happened. Julian cleaned up a couple of places stuff where we were allocating 2K of local data *twice* on local stack frames. There are some gcc patches floating around that enable you to generate a warning if your local stack frame exceedes a certain amount or the arguments are bigger than a specified amount. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 21:18:15 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id A442837B428 for ; Mon, 24 Sep 2001 21:18:11 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8P4Hv599043; Mon, 24 Sep 2001 21:17:57 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 21:17:57 -0700 (PDT) From: Matt Dillon Message-Id: <200109250417.f8P4Hv599043@earth.backplane.com> To: Peter Wemm Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <20010925033116.6C2C23808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Oh, one other thing... When we had PCIBIOS active for pci config space :read/write support, we had stack overflows on many systems when the SSE :stuff got MFC'ed. The simple act of trimming about 300 bytes from the :pcb_save structure was enough to make the difference between it working or :not. We are *way* too close to the wire. I asked about raising UPAGES :from 2 to 3 before RELENG_4_4 but it never happened. : :Julian cleaned up a couple of places stuff where we were allocating 2K of :local data *twice* on local stack frames. There are some gcc patches :floating around that enable you to generate a warning if your local stack :frame exceedes a certain amount or the arguments are bigger than a :specified amount. : :Cheers, :-Peter :-- :Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au I'm getting stack underflows with UPAGES set to 2. I've set UPAGES to 4 and preinitialized the UAREA to 0x11 and then scan it in exit1() to determine how much stack was actually used. If these numbers are correct, we are screwed with UPAGES set to 2. This is just 4 seconds worth of a buildworld. Note the '3664's showing up. That's too close. note the 3984 that came up after playing with the system for a few seconds! I'll post the patch set to use to test this stuff in a moment. -Matt process 323 exit kstackuse 2272 ... process 333 exit kstackuse 2272 process 225 exit kstackuse 3664 process 233 exit kstackuse 2272 ... process 237 exit kstackuse 2272 process 322 exit kstackuse 2676 process 334 exit kstackuse 2272 ... process 319 exit kstackuse 2272 test1# dmesg | fgrep process | sort -n +4 | tail -10 process 6 exit kstackuse 3640 process 89 exit kstackuse 3640 process 176 exit kstackuse 3664 process 186 exit kstackuse 3664 process 225 exit kstackuse 3664 process 290 exit kstackuse 3664 process 299 exit kstackuse 3664 process 300 exit kstackuse 3664 process 303 exit kstackuse 3664 process 138 exit kstackuse 3984 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 21:26:15 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 528E937B408 for ; Mon, 24 Sep 2001 21:26:07 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8P4Q7a99088; Mon, 24 Sep 2001 21:26:07 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 21:26:07 -0700 (PDT) From: Matt Dillon Message-Id: <200109250426.f8P4Q7a99088@earth.backplane.com> To: Peter Wemm Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Patch to test kstack usage. Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This isn't perfect but it should be a good start in regards to testing kstack use. This patch is against -stable. It reports kernel stack use on process exit and will generate a 'Kernel stack underflow' message if it detects an underflow. It doesn't panic, so for a fun time you can leave UPAGES at 2 and watch in horror. note: make sure you make depend before making a new kernel, or use buildkernel. -Matt Index: sys/user.h =================================================================== RCS file: /home/ncvs/src/sys/sys/user.h,v retrieving revision 1.24 diff -u -r1.24 user.h --- sys/user.h 1999/12/29 04:24:49 1.24 +++ sys/user.h 2001/09/25 03:41:04 @@ -109,9 +109,13 @@ * Remaining fields only for core dump and/or ptrace-- * not valid at other times! */ + u_int32_t u_guard2; /* guard the base of the kstack */ struct kinfo_proc u_kproc; /* proc + eproc */ struct md_coredump u_md; /* machine dependent glop */ + u_int32_t u_guard; /* guard the base of the kstack */ }; + +#define U_GUARD_MAGIC 0x51A2C3D4 /* * Redefinitions to make the debuggers happy for now... This subterfuge Index: kern/init_main.c =================================================================== RCS file: /home/ncvs/src/sys/kern/init_main.c,v retrieving revision 1.134.2.6 diff -u -r1.134.2.6 init_main.c --- kern/init_main.c 2001/06/15 09:37:55 1.134.2.6 +++ kern/init_main.c 2001/09/25 01:39:05 @@ -358,6 +358,7 @@ */ p->p_stats = &p->p_addr->u_stats; p->p_sigacts = &p->p_addr->u_sigacts; + p->p_addr->u_guard = U_GUARD_MAGIC; /* bottom of kernel stack */ /* * Charge root for one process. Index: kern/kern_exit.c =================================================================== RCS file: /home/ncvs/src/sys/kern/kern_exit.c,v retrieving revision 1.92.2.5 diff -u -r1.92.2.5 kern_exit.c --- kern/kern_exit.c 2001/07/27 14:06:01 1.92.2.5 +++ kern/kern_exit.c 2001/09/25 04:09:32 @@ -123,6 +123,16 @@ WTERMSIG(rv), WEXITSTATUS(rv)); panic("Going nowhere without my init!"); } + { + int *ua; + int *addrend = (int *)((char *)p->p_addr + UPAGES * PAGE_SIZE); + for (ua = &p->p_addr->u_guard + 1; ua < addrend; ++ua) { + if (*ua != 0x11111111) + break; + } + printf("process %d exit kstackuse %d\n", + p->p_pid, (char *)addrend - (char *)ua); + } aio_proc_rundown(p); Index: kern/kern_synch.c =================================================================== RCS file: /home/ncvs/src/sys/kern/kern_synch.c,v retrieving revision 1.87.2.3 diff -u -r1.87.2.3 kern_synch.c --- kern/kern_synch.c 2000/12/31 22:10:45 1.87.2.3 +++ kern/kern_synch.c 2001/09/25 02:54:46 @@ -44,13 +44,17 @@ #include #include #include +#include #include #include #include #include #include #include +#include +#include #include +#include #ifdef KTRACE #include #include @@ -792,6 +796,13 @@ register struct proc *p = curproc; /* XXX */ register struct rlimit *rlim; int x; + + /* + * Check to see if the kernel stack underflowed (XXX) + */ + if (p->p_addr->u_guard != U_GUARD_MAGIC) { + printf("Kernel stack underflow! %p %p %08x\n", p, p->p_addr, p->p_addr->u_guard); + } /* * XXX this spl is almost unnecessary. It is partly to allow for Index: i386/i386/pmap.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/pmap.c,v retrieving revision 1.250.2.10 diff -u -r1.250.2.10 pmap.c --- i386/i386/pmap.c 2001/07/30 23:27:59 1.250.2.10 +++ i386/i386/pmap.c 2001/09/25 04:03:52 @@ -891,6 +891,7 @@ } if (updateneeded) invltlb(); + memset(up, 0x11, UPAGES * PAGE_SIZE); } /* Index: i386/include/param.h =================================================================== RCS file: /home/ncvs/src/sys/i386/include/param.h,v retrieving revision 1.54.2.5 diff -u -r1.54.2.5 param.h --- i386/include/param.h 2001/09/15 00:50:36 1.54.2.5 +++ i386/include/param.h 2001/09/25 03:41:11 @@ -110,7 +110,7 @@ #define MAXDUMPPGS (DFLTPHYS/PAGE_SIZE) #define IOPAGES 2 /* pages of i/o permission bitmap */ -#define UPAGES 2 /* pages of u-area */ +#define UPAGES 4 /* pages of u-area */ /* * Ceiling on amount of swblock kva space. Index: vm/vm_glue.c =================================================================== RCS file: /home/ncvs/src/sys/vm/vm_glue.c,v retrieving revision 1.94.2.1 diff -u -r1.94.2.1 vm_glue.c --- vm/vm_glue.c 2000/08/02 22:15:09 1.94.2.1 +++ vm/vm_glue.c 2001/09/25 03:52:01 @@ -265,6 +265,11 @@ ((caddr_t) &up->u_stats.pstat_endcopy - (caddr_t) &up->u_stats.pstat_startcopy)); + /* + * Kernel stack guard (detection only unfortunately) + */ + up->u_guard = U_GUARD_MAGIC; + up->u_guard2 = U_GUARD_MAGIC; /* * cpu_fork will copy and update the pcb, set up the kernel stack, To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 22:19:31 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 24C6737B40D for ; Mon, 24 Sep 2001 22:19:24 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8P5JNM20685 for ; Mon, 24 Sep 2001 22:19:23 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 6CA2F3808; Mon, 24 Sep 2001 22:19:23 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Matt Dillon Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: Patch to test kstack usage. In-Reply-To: <200109250426.f8P4Q7a99088@earth.backplane.com> Date: Mon, 24 Sep 2001 22:19:23 -0700 From: Peter Wemm Message-Id: <20010925051923.6CA2F3808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matt Dillon wrote: > This isn't perfect but it should be a good start in regards to > testing kstack use. This patch is against -stable. It reports > kernel stack use on process exit and will generate a 'Kernel stack > underflow' message if it detects an underflow. It doesn't panic, > so for a fun time you can leave UPAGES at 2 and watch in horror. It is checking against the wrong guard value. It should be u_guard2. FWIW; the max stack available is 4688 bytes on a standard 4.x system. Yes, that is too freaking close. Also, the maximum usage depends on what sort of cards you have in the system.. If you have a heavy tty user (eg: a 32+ port serial card) then you have lots of tty interrupts nesting as well. Having the ppp/sl/plip drivers in the system partly negates the effect of this though since it wires the net/tty interrupt masks together. peter@thunder[10:13pm]~-111> ./tu stack base = 3504 stack size = 4688 peter@thunder[10:13pm]~-112> cat tu.c #include #include #include #include int main(int ac, char **av) { int stack_base = offsetof(struct user, u_kproc); printf("stack base = %d\n", stack_base); printf("stack size = %d\n", UPAGES * PAGE_SIZE - stack_base); } > --- sys/user.h 1999/12/29 04:24:49 1.24 > +++ sys/user.h 2001/09/25 03:41:04 > @@ -109,9 +109,13 @@ > * Remaining fields only for core dump and/or ptrace-- > * not valid at other times! > */ > + u_int32_t u_guard2; /* guard the base of the kstack */ > struct kinfo_proc u_kproc; /* proc + eproc */ > struct md_coredump u_md; /* machine dependent glop */ > + u_int32_t u_guard; /* guard the base of the kstack */ > }; Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 22:40: 9 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id C9F0437B405 for ; Mon, 24 Sep 2001 22:40:06 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8P5e6499443; Mon, 24 Sep 2001 22:40:06 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 22:40:06 -0700 (PDT) From: Matt Dillon Message-Id: <200109250540.f8P5e6499443@earth.backplane.com> To: Peter Wemm Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: Patch to test kstack usage. References: <20010925051923.6CA2F3808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :Matt Dillon wrote: :> This isn't perfect but it should be a good start in regards to :> testing kstack use. This patch is against -stable. It reports :> kernel stack use on process exit and will generate a 'Kernel stack :> underflow' message if it detects an underflow. It doesn't panic, :> so for a fun time you can leave UPAGES at 2 and watch in horror. : :It is checking against the wrong guard value. It should be u_guard2. : :FWIW; the max stack available is 4688 bytes on a standard 4.x system. Yes, :that is too freaking close. Also, the maximum usage depends on what sort :of cards you have in the system.. If you have a heavy tty user (eg: a 32+ I looked at it fairly carefully. It has got to be u_guard... at the end of struct user, at least until you do that MFC. The ptrace code appears to mess around with u_kproc quite a bit. And when you rip out u_kproc it still needs to be at the end, after the coredump structure (though for i386 the coredump structure is empty)... because interrupts can occur during a core dump. :port serial card) then you have lots of tty interrupts nesting as well. :Having the ppp/sl/plip drivers in the system partly negates the effect of :this though since it wires the net/tty interrupt masks together. :... :Cheers, :-Peter :-- :Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au :"All of this is for nothing if we don't go to the stars" - JMS/B5 : Yah... the test I ran was just a couple of seconds worth of playing around over ssh. I expect the worst case to be a whole lot worse. We're going to have to bump up UPAGES to 3 in 4.x, there's no question about it. I'm going to do it tonight. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 22:56:44 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 26C6237B421 for ; Mon, 24 Sep 2001 22:56:42 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8P5ufF99518; Mon, 24 Sep 2001 22:56:41 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 Sep 2001 22:56:41 -0700 (PDT) From: Matt Dillon Message-Id: <200109250556.f8P5ufF99518@earth.backplane.com> To: Peter Wemm Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: Patch to test kstack usage. References: <20010925051923.6CA2F3808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :stack size = 4688 Sep 24 22:47:22 test1 /kernel: process 29144 exit kstackuse 4496 closer... :-) -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Sep 24 22:56:53 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 109EE37B41D for ; Mon, 24 Sep 2001 22:56:41 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8P5ueM20772 for ; Mon, 24 Sep 2001 22:56:40 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 9472C3808; Mon, 24 Sep 2001 22:56:40 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Matt Dillon Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: Patch to test kstack usage. In-Reply-To: <200109250540.f8P5e6499443@earth.backplane.com> Date: Mon, 24 Sep 2001 22:56:40 -0700 From: Peter Wemm Message-Id: <20010925055640.9472C3808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matt Dillon wrote: > Yah... the test I ran was just a couple of seconds worth of playing > around over ssh. I expect the worst case to be a whole lot worse. > > We're going to have to bump up UPAGES to 3 in 4.x, there's no question > about it. I'm going to do it tonight. Heh. I already asked to do it a few weeks ago, in order to get it into the release. I guess I wasn't persistant enough. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 0:16:29 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from ns1.infowest.com (ns1.infowest.com [204.17.177.10]) by hub.freebsd.org (Postfix) with ESMTP id F14CE37B43C for ; Tue, 25 Sep 2001 00:16:26 -0700 (PDT) Received: from beware.dragonknight.net (unknown [208.186.107.174]) by ns1.infowest.com (Postfix) with SMTP id F1B5D20F4D for ; Tue, 25 Sep 2001 01:16:25 -0600 (MDT) Content-Type: text/plain; charset="iso-8859-1" From: Samuel Greear To: freebsd-hackers@freebsd.org Subject: dirlist mangling Date: Tue, 25 Sep 2001 01:16:04 -0600 X-Mailer: KMail [version 1.2] MIME-Version: 1.0 Message-Id: <01092501160402.10676@beware.dragonknight.net> Content-Transfer-Encoding: 8bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Never done any kernel hacking before so I'm just looking for some pointers. What's needed is a mechanism to specify a directory (or set of them) and whenever a request is made for the contents of that directory, if it exists in the list then what is returned needs to be mangled in some ways. For instance an ls in a directory in the list might only return a list of files that you own, or that you have permission to read. -- Samuel J. Greear Developer - GetMegabits, Inc. http://www.itmom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 0:55:41 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id 35FB437B40C for ; Tue, 25 Sep 2001 00:55:37 -0700 (PDT) Received: from mail.cicely.de (cicely20 [10.1.1.22]) by srv1.cosmo-project.de (8.11.0/8.11.0) with ESMTP id f8P7tX692377; Tue, 25 Sep 2001 09:55:34 +0200 (CEST) Received: (from ticso@localhost) by mail.cicely.de (8.11.0/8.11.0) id f8P7u8327761; Tue, 25 Sep 2001 09:56:08 +0200 (CEST) Date: Tue, 25 Sep 2001 09:56:07 +0200 From: Bernd Walter To: Bakul Shah Cc: Matt Dillon , Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? Message-ID: <20010925095607.B27615@cicely20.cicely.de> References: <200109250027.f8P0RRk97980@earth.backplane.com> <200109250114.VAA20993@glatton.cnchost.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109250114.VAA20993@glatton.cnchost.com>; from bakul@bitblocks.com on Mon, Sep 24, 2001 at 06:14:34PM -0700 X-Operating-System: NetBSD cicely20.cicely.de 1.5 sparc Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Sep 24, 2001 at 06:14:34PM -0700, Bakul Shah wrote: > FWIW, in a Unix port we did I remember putting the user > struct *above* the kernel stack. The stack grew down so you > hit the red zone (the guard pages) without clobbering the > user struct. Since struct user _ended_ on a page boundary, > its size was needed at locore.s assembly time but that was a > small price to pay for the added safety. I don't think a guard page can help here, because the page fault handler needs a working stack. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 2:18:31 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id CE09F37B405 for ; Tue, 25 Sep 2001 02:18:27 -0700 (PDT) Received: from mail.cicely.de (cicely20 [10.1.1.22]) by srv1.cosmo-project.de (8.11.0/8.11.0) with ESMTP id f8P9IP692757; Tue, 25 Sep 2001 11:18:25 +0200 (CEST) Received: (from ticso@localhost) by mail.cicely.de (8.11.0/8.11.0) id f8P9IxC27864; Tue, 25 Sep 2001 11:18:59 +0200 (CEST) Date: Tue, 25 Sep 2001 11:18:59 +0200 From: Bernd Walter To: Peter Wullinger Cc: Bernd Walter , freebsd-hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? Message-ID: <20010925111858.C27615@cicely20.cicely.de> References: <200109250027.f8P0RRk97980@earth.backplane.com> <200109250114.VAA20993@glatton.cnchost.com> <20010925095607.B27615@cicely20.cicely.de> <20010925100103.A4016@pc04.ipc-kallmuenz.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010925100103.A4016@pc04.ipc-kallmuenz.de>; from RivaW@gmx.de on Tue, Sep 25, 2001 at 10:01:03AM +0200 X-Operating-System: NetBSD cicely20.cicely.de 1.5 sparc Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Sep 25, 2001 at 10:01:03AM +0200, Peter Wullinger wrote: > On Tue, Sep 25, 2001 at 09:56:07AM +0200, Bernd Walter wrote: > > On Mon, Sep 24, 2001 at 06:14:34PM -0700, Bakul Shah wrote: > > > FWIW, in a Unix port we did I remember putting the user > > > struct *above* the kernel stack. The stack grew down so you > > > hit the red zone (the guard pages) without clobbering the > > > user struct. Since struct user _ended_ on a page boundary, > > > its size was needed at locore.s assembly time but that was a > > > small price to pay for the added safety. > > > > I don't think a guard page can help here, because the page fault > > handler needs a working stack. > > > Depends on what is does ... if it just panics and syncs and does > not care overwriting the user struct of the current process (which > is lost anyway), is this much of a problem? Please correct me if I'm missing something. If it is overwriting there is no page fault thus no guard page and no panic. If you would have a page fault there is no space where the CPU can write the state information to for entering the handler. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 2:20:25 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id D554D37B403 for ; Tue, 25 Sep 2001 02:20:21 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.4/8.11.4) with ESMTP id f8P9KJv25347; Tue, 25 Sep 2001 11:20:19 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Bernd Walter Cc: Peter Wullinger , freebsd-hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: Your message of "Tue, 25 Sep 2001 11:18:59 +0200." <20010925111858.C27615@cicely20.cicely.de> Date: Tue, 25 Sep 2001 11:20:19 +0200 Message-ID: <25345.1001409619@critter> From: Poul-Henning Kamp Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20010925111858.C27615@cicely20.cicely.de>, Bernd Walter writes: >On Tue, Sep 25, 2001 at 10:01:03AM +0200, Peter Wullinger wrote: >> On Tue, Sep 25, 2001 at 09:56:07AM +0200, Bernd Walter wrote: >> > On Mon, Sep 24, 2001 at 06:14:34PM -0700, Bakul Shah wrote: >> > > FWIW, in a Unix port we did I remember putting the user >> > > struct *above* the kernel stack. The stack grew down so you >> > > hit the red zone (the guard pages) without clobbering the >> > > user struct. Since struct user _ended_ on a page boundary, >> > > its size was needed at locore.s assembly time but that was a >> > > small price to pay for the added safety. >> > >> > I don't think a guard page can help here, because the page fault >> > handler needs a working stack. >> > >> Depends on what is does ... if it just panics and syncs and does >> not care overwriting the user struct of the current process (which >> is lost anyway), is this much of a problem? > >Please correct me if I'm missing something. >If it is overwriting there is no page fault thus no guard page and >no panic. >If you would have a page fault there is no space where the CPU can >write the state information to for entering the handler. And it would take a double-fault for which we have a handler with it's own stack. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 2:49:31 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from smtp.noos.fr (claudel.noos.net [212.198.2.83]) by hub.freebsd.org (Postfix) with ESMTP id C6F7D37B43A for ; Tue, 25 Sep 2001 02:49:27 -0700 (PDT) Received: (qmail 49652534 invoked by uid 0); 25 Sep 2001 09:49:23 -0000 Received: from unknown (HELO gits.dyndns.org) ([212.198.231.187]) (envelope-sender ) by 212.198.2.83 (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 25 Sep 2001 09:49:23 -0000 Received: (from root@localhost) by gits.dyndns.org (8.11.6/8.11.6) id f8P9nJT26069; Tue, 25 Sep 2001 11:49:19 +0200 (CEST) (envelope-from root) Message-Id: <200109250949.f8P9nJT26069@gits.dyndns.org> Subject: Re: termcap sources In-Reply-To: <20010924234245.B14545@hades.hell.gr> To: Giorgos Keramidas Date: Tue, 25 Sep 2001 11:49:19 +0200 (CEST) Cc: hackers@freebsd.org Reply-To: clefevre@citeweb.net From: Cyrille Lefevre Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[<97Zd*>^#%Y5Cxv;%Y[PT-LW3;A:fRrJ8+^k"e7@+30g0YD0*^^3jgyShN7o?a]C la*Zv'5NA,=963bM%J^o]C X-Mailer: ELM [version 2.4ME+ PL94b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Giorgos Keramidas wrote: [snip] > As you can see there are quite a few terminals that have capabilities defined > more than once! I don't have THAT many terminals to check, but I'm open to > suggestions. Should we do something about this? If yes, what? this isn't a problem since only the first matching capability is used. others are ignored. I've a pending termcap update related to http://www.tuxedo.org/terminfo. for instance, I don't remember if reorder works w/ termtypes.tc ? Cyrille. -- Cyrille Lefevre mailto:clefevre@citeweb.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 2:52:23 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.twwells.com (mail.twwells.com [64.38.247.128]) by hub.freebsd.org (Postfix) with ESMTP id 3B92337B41E for ; Tue, 25 Sep 2001 02:52:17 -0700 (PDT) Received: from xfermail (helo=mail.twwells.com) by mail.twwells.com with local-bsmtp (Exim 3.32 #1) id 15lost-0003QV-00 for freebsd-hackers@freebsd.org; Tue, 25 Sep 2001 02:52:11 -0700 X-Filter-Status: mail.twwells.com ok 10 Received: from twwells.com ( [65.14.140.228] ) by mail.twwells.com via tcp with esmtp id 3bb053c1-00336e; Tue, 25 Sep 2001 09:52:01 +0000 Received: from admin by twwells.com with local (Exim 3.33 #1) id 15losV-000BIQ-00; Tue, 25 Sep 2001 05:51:47 -0400 From: admin@twwells.com Subject: missing words, lots of them To: freebsd-hackers@freebsd.org Message-Id: Date: Tue, 25 Sep 2001 05:51:47 -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG These words, 830 of them, were obtained by intersecting the words in a number of lexicons and then subtracting the words in /usr/share/dict/web2. This all done with words that contain only lowercase letters. You'll find those words at the end of this message. Should you take even a cursory look at this list, I expect you'll be appalled at the words that are not in the lexicon. The point is *not* that these words should be added. The point is that a cursory, in-my-sleep check of the word list shows glaring deficiencies. A serious audit of the list will find way many more missing words (I did a preliminary -- think ~50,000-100,000 missing words if it is supposed to approximate the contents of an unabridged dictionary.) Anyway, I'm willing to create a replacement list, if it's likely to actually get used. Missing words: ============================================================================== abbreviated aborning absentminded academia acknowledgment actuate addictive adios aerospace affairs affianced aficionado aforementioned agonized agonizing agribusiness airborne airflow airline airs airspace airspeed alleged allowed aloes aloha alphanumeric ammoniac analog anechoic anomie antebellum antecedents antidisestablishmentarianism antimatter antimicrobial antipasto antiperspirant anytime appetizing apples appointed arbitrage archives arrears artwork ashtray assembled asserted attested audiovisual authors autocorrelation automate automation auxiliaries avionics awed backpack backstairs bags baklava balls bananas bandwagon bandwidth bans baptistry barbell barracks barrens baseline bawdy beatnik becalmed became bedraggled beep began begot belongings biased bidden bifocals bijection bimetallic binoculars biofeedback biomass biomedicine blabbermouth blacklist bleachers blew blinders blinkers blond bloodstream boatyard bobsledding boldface boogie boson botulinus bouncy bounds boutique box boyfriend braggadocio brainchild brainstorm bratwurst breathtaking briefcase brindle brindled brinkmanship britches brouhaha buckskins bugs bullshit bullyboy bumbling bunkmate burdened burger busboy butterflies buttocks byte cacciatore cannonball capita cards careerism cartwheel cassette castanets catalog caulk caveman challenging chambers changeover charged checklist checkpoint children chipboard chops chorale chromaticness ciao circuitry clannish classics classify classless clericals clipboard clippers clomp clueless cm coastline combo computerize coney confer cons contend contrabassoon contrived conveyor cookie cooperate cooperation cooperative coordinate coordination copywriter cordless cords corduroys corned corticosteroid councillor counterproductive courses coven coverall cowgirl cowpoke credits creeps critter crocked crud cuffs curia curtsey damaging damnedest danged dated dateline deadweight debug decencies decoder decor decrypt dedicated deli demo demodulate demur demythologize denims deprived desegregate despatch destined destruct diminished directions disadvantaged disembodied dishonesty distended disulfide disused divertimento dividers dogleg dominations dominions donnybrook doubleheader doubles downs downwards drily droppings drops dryer duce ducks duds dues dumbfound dumps dystopia eastwards eatables ecstatics ecumenicist edibles eggbeater einsteinium elan electromyography elevenses emaciated emancipated endgame enervated epoxy equities ergo erstwhile esprit esthete estranged evenings expecting expertise extramarital eyeglasses fallout falsity famed famished fantasize farmland fatigued fatso feathers feet feints femme fermion fermium ferroelectric fete fewer fiberglass fief fieldstone filigreed finicky fireworks fisticuffs fjord flamethrower flashback flashbulb flats floats floorboard fluorocarbon foiled fond footloose forensics forgave forgiven forklift forsook foxed frag frazzled fresher freshwater frostbitten frustrated fumed futures gadgetry geese gigahertz gills gimbals girlfriend gizmo glim glob globetrotter goddamn goddamned goggles goodbye gooey grapheme greatest greenbelt groceries grooved groundhog gunfight gunslinger gutsy hadron haiku hairstyle hammered handcrafted handlebar hands handyman hang hangover harassed hardboard hardened hards hardtop hardworking harken harrumph has haunted haunting heads headwaters heated heaves heist held hellfire helluva hereabouts heres heroics hibachi hid hideout hightail hijacker hipster histrionics hoagy holography homecoming honky hooray hooves hopping horrified horseshoes hubris hype icosahedron idiolect idyll immersed implied impoverished improvised incised inclined including incorporating industrials inherited innards instructions integrated interdisciplinary interfaith interferon intransigence intro ironic irons irritated isometrics itemized ivories jackboot jackpot jacks jaundiced jaws jeepers jetliner jiggered jigging jigsaw jockstrap jumbled junkie karat kcal keystroke kg kid kidding killjoy kilohertz kitsch klutz km knives knockwurst krill lamed lancers lands larger lauds lawrencium lefty leitmotif letters liabilities lice lights linebacker lines lineup lipid lists litterbug littlest lives loads loaves logjam logo loner loudspeaker lumpen machismo macintosh mademoiselle mainland mainstream makeup maladapted mama manned manpower mapping marketplace marshmallow martini maser materials mayest means meantime measures memoirs mendelevium meritocracy merits methanol metro mews microeconomics midi midpoint midsection midterm millipede minefield minesweeper mini miniaturize miniskirt minutes mitzvah ml mm mom moneymaking monies mopes moulder multidisciplinary multipurpose multiversity muon mystique nameplate nanosecond narrows nauseating near necessaries needlepoint neomycin nerves newborn nightclub nobelium nonlinearity notions nuts nylons obsequies octal offstage okay oldest oops oregano ornamented orphanage outfox outgoings outskirts outstretched ouzo overkill paid pains pairs pajamas palatalized paramedical paraprofessional paratroops parkland parts pas passed pasta payload peanuts petrology phlox pickaxe pickled pinwheel pion pissed piton plains plastered pleadings politicking polypropylene polyunsaturated pons pontificals pops postcard postmortem potage precincts preeminence preeminent preempt preemption preliminaries premises prevailing principalities principled printmaker propel proud psychedelic psycho psychoactive psycholinguistics pullover pulsar pulverable pushed pushy quadriplegic quasar questioning racehorse racquet rags rains rainwater ranking rapids rapprochement rarefied rateable rations recurved redneck reenter reentry relations resolutive retorsion retrorocket ribosome rights rockbound rompers rooftop roughneck rounders rundown salami sales salesgirl salts saltwater satiated schmuck schoolmarm scorecard scuba seafood seawards selves sexism sexist shambles sheared sheaves shelves shit shoreline shortcut shoveler showplace sideband sidelight sideman sidereal sideshow sidestep simulated sizzler skydive skyjack skyline slacks slalom sledgehammer slier sliest slipstream smashed snazzy sneakers snips snowman software soubriquet sourdough sox spacecraft spacesuit spheroid spic spoils sputnik stacked stairs stairwell stalag standby staves steels stockroom stopwatch stores striptease stripteaser strobe strongroom subroutine suited sulfate sulfide sulfite sulfur summitry sunbaked sunglasses suntan supersonics supra supremacist sushi swats sweeps sweepstakes switchblade taco tailback tailgate tailored tails takeoff takeover tangled tantalizing taxiway tears teenager temporize terms tetracycline throes tinfoil titillate toothpaste toots trademark transistorize transponder trappings trembles trunks tsarism turbojet turtleneck typeface typewritten ultrasound ultrastructure underdeveloped underpinnings undersigned understructure upbringing upperclassman upturned varistor varmint velour verified vested vibes vibrations virtues vivace waddle wares warhead watchband watercolor watercourse wavelength weeds weirdo wheelbase wheelchair wheels whimsy whosoever widget wilful windowsill wingback wingspan wino wiretap withdrew wits wives woken wolves women woodchopper woods woodwind words workhorse worldwide worthwhile wraparound wristwatch yogurt zap To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 3: 1:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from gekko.ms-agentur.de (server.ms-agentur.de [62.153.134.194]) by hub.freebsd.org (Postfix) with ESMTP id 7AE9937B42B for ; Tue, 25 Sep 2001 03:01:06 -0700 (PDT) Received: from i-clue.de (automatix.i-clue.de [192.168.0.112]) by gekko.ms-agentur.de (8.9.3/8.9.3/SuSE Linux 8.9.3-0.1) with ESMTP id MAA22073; Tue, 25 Sep 2001 12:10:45 +0200 Message-ID: <3BB055DA.20600@i-clue.de> Date: Tue, 25 Sep 2001 12:00:58 +0200 From: Christoph Sold User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; de-AT; rv:0.9.4+) Gecko/20010916 X-Accept-Language: de, en MIME-Version: 1.0 To: admin@twwells.com Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: missing words, lots of them References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG admin@twwells.com wrote: >These words, 830 of them, were obtained by intersecting the words >in a number of lexicons and then subtracting the words in >/usr/share/dict/web2. [snip] > >The point is *not* that these words should be added. The point is >that a cursory, in-my-sleep check of the word list shows glaring >deficiencies. A serious audit of the list will find way many more >missing words (I did a preliminary -- think ~50,000-100,000 >missing words if it is supposed to approximate the contents of an >unabridged dictionary.) > This is to be expected. The word list was created from an very old Webster dictionary, because the copyright had to expire before it could be used in an open source dictionary. >Anyway, I'm willing to create a replacement list, if it's likely >to actually get used. > That would be a welcome contribution to the project. >Missing words: [Lots of them deleted] > Just my EUR.02 -Christoph Sold To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 3:50: 9 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.twwells.com (mail.twwells.com [64.38.247.128]) by hub.freebsd.org (Postfix) with ESMTP id E9FE737B42F for ; Tue, 25 Sep 2001 03:50:07 -0700 (PDT) Received: from xfermail (helo=mail.twwells.com) by mail.twwells.com with local-bsmtp (Exim 3.32 #1) id 15lpms-0003nz-00 for freebsd-hackers@freebsd.org; Tue, 25 Sep 2001 03:50:02 -0700 X-Filter-Status: mail.twwells.com ok 40 Received: from twwells.com ( [65.14.140.228] ) by mail.twwells.com via tcp with esmtp id 3bb06132-00390e; Tue, 25 Sep 2001 10:49:22 +0000 Received: from root by twwells.com with local (Exim 3.33 #1) id 15lpmD-000Cmm-00; Tue, 25 Sep 2001 06:49:21 -0400 To: freebsd-hackers@freebsd.org Subject: Re: missing words, lots of them References: <3BB055DA.20600@i-clue.de> From: admin@twwells.com Message-Id: Date: Tue, 25 Sep 2001 06:49:21 -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Christoph Sold wrote: > admin@twwells.com wrote: > This is to be expected. The word list was created from an very old > Webster dictionary, because the copyright had to expire before it could > be used in an open source dictionary. I know. :) However, there are several lexicons that have acceptable copyrights. E.g., the Moby list, though I have some reservations about it, is public domain. So there's no good reason to live with an archaic list. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 4:47:45 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 5529837B41E for ; Tue, 25 Sep 2001 04:47:38 -0700 (PDT) Received: from elischer.org (InterJet.elischer.org [192.168.1.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id FAA54711; Tue, 25 Sep 2001 05:28:26 -0700 (PDT) Message-ID: <3BB06BFD.F1870676@elischer.org> Date: Tue, 25 Sep 2001 04:35:25 -0700 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Peter Wemm Cc: Matt Dillon , Ian Dowse , hackers@FreeBSD.ORG Subject: Re: Patch to test kstack usage. References: <20010925051923.6CA2F3808@overcee.netplex.com.au> Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Wemm wrote: > > Matt Dillon wrote: > > This isn't perfect but it should be a good start in regards to > > testing kstack use. This patch is against -stable. It reports > > kernel stack use on process exit and will generate a 'Kernel stack > > underflow' message if it detects an underflow. It doesn't panic, > > so for a fun time you can leave UPAGES at 2 and watch in horror. > > It is checking against the wrong guard value. It should be u_guard2. > > FWIW; the max stack available is 4688 bytes on a standard 4.x system. Yes, > that is too freaking close. Also, the maximum usage depends on what sort > of cards you have in the system.. If you have a heavy tty user (eg: a 32+ > port serial card) then you have lots of tty interrupts nesting as well. > Having the ppp/sl/plip drivers in the system partly negates the effect of > this though since it wires the net/tty interrupt masks together. usb devices allocate 2K on the stack so if you have them too....... so does openning an atapi cdrom... so a combination of interrupts and those, might consume 4K > > peter@thunder[10:13pm]~-111> ./tu > stack base = 3504 > stack size = 4688 > peter@thunder[10:13pm]~-112> cat tu.c > #include > #include > #include > #include > > int > main(int ac, char **av) > { > int stack_base = offsetof(struct user, u_kproc); > > printf("stack base = %d\n", stack_base); > printf("stack size = %d\n", UPAGES * PAGE_SIZE - stack_base); > } > > > --- sys/user.h 1999/12/29 04:24:49 1.24 > > +++ sys/user.h 2001/09/25 03:41:04 > > @@ -109,9 +109,13 @@ > > * Remaining fields only for core dump and/or ptrace-- > > * not valid at other times! > > */ > > + u_int32_t u_guard2; /* guard the base of the kstack */ > > struct kinfo_proc u_kproc; /* proc + eproc */ > > struct md_coredump u_md; /* machine dependent glop */ > > + u_int32_t u_guard; /* guard the base of the kstack */ > > }; > > Cheers, > -Peter > -- > Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au > "All of this is for nothing if we don't go to the stars" - JMS/B5 -- +------------------------------------+ ______ _ __ | __--_|\ Julian Elischer | \ U \/ / hard at work in | / \ julian@elischer.org +------>x USA \ a very strange | ( OZ ) \___ ___ | country ! +- X_.---._/ presently in San Francisco \_/ \\ v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 5: 6:28 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from brunel.uk1.vbc.net (brunel.uk1.vbc.net [194.207.2.8]) by hub.freebsd.org (Postfix) with ESMTP id A12D337B440 for ; Tue, 25 Sep 2001 05:06:24 -0700 (PDT) Received: from localhost (jcv@localhost) by brunel.uk1.vbc.net (8.11.6/8.11.6) with ESMTP id f8PC6Mw43932 for ; Tue, 25 Sep 2001 13:06:23 +0100 (BST) X-Authentication-Warning: brunel.uk1.vbc.net: jcv owned process doing -bs Date: Tue, 25 Sep 2001 13:06:22 +0100 (BST) From: Jean-Christophe Varaillon X-Sender: jcv@brunel.uk1.vbc.net To: freebsd-hackers@FREEBSD.ORG Subject: boot source code In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, I am looking for the source code used by the boot floppy to load itself into memory. At www.freebsd.org I found a lot of boot source code but I don't really know which one is intersting for me. My platform is i386. Thanks for your help, Christophe. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 5:34:14 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from balrog.rt.ru (balrog.rt.ru [195.161.0.169]) by hub.freebsd.org (Postfix) with ESMTP id A31D937B43B for ; Tue, 25 Sep 2001 05:34:09 -0700 (PDT) Received: from rt.ru (localhost [127.0.0.1]) by balrog.rt.ru (8.9.3/8.9.3) with ESMTP id QAA14878 for ; Tue, 25 Sep 2001 16:32:58 +0400 (MSD) (envelope-from dima@rt.ru) Message-ID: <3BB0797A.FB060034@rt.ru> Date: Tue, 25 Sep 2001 16:32:58 +0400 From: "Dmitry S. Rzhavin" Organization: Rostelecom X-Mailer: Mozilla 4.7 [en] (X11; I; FreeBSD 4.0-20000103-CURRENT i386) X-Accept-Language: ru, en MIME-Version: 1.0 To: hackers@FreeBSD.ORG Subject: 4.4 jail and udp Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi! Looks like jailed processes can't use udp or icmp, but can use tcp: # ping www.freebsd.org ping: socket: Operation not permitted # telnet www.freebsd.org 80 Trying 216.136.204.21... Connected to freefall.freebsd.org. Escape character is '^]'. ^] telnet> q Connection closed. # options IPFIREWALL is enabled, net.inet.ip.fw.enable is 0 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 6:36:48 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.numachi.com (numachi.numachi.com [198.175.254.2]) by hub.freebsd.org (Postfix) with SMTP id B2AF037B446 for ; Tue, 25 Sep 2001 06:36:44 -0700 (PDT) Received: (qmail 25519 invoked by uid 3001); 25 Sep 2001 13:36:43 -0000 Received: from natto.numachi.com (198.175.254.216) by numachi.numachi.com with SMTP; 25 Sep 2001 13:36:43 -0000 Received: (qmail 42475 invoked by uid 1001); 25 Sep 2001 13:36:43 -0000 Date: Tue, 25 Sep 2001 09:36:43 -0400 From: Brian Reichert To: freebsd-hackers@freebsd.org Subject: got bad cookie vp 0xe2e5ef80 bp 0xcf317328 Message-ID: <20010925093643.W49528@numachi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I'm starting to see errors in /var/log/messages under 4.2-RELEASE: Sep 23 00:31:17 bmdb1 /kernel: got bad cookie vp 0xe2e5ef80 bp 0xcf317328 Not many, but more than one, and I've never seen this in my years of using FreeBSD. The code producing this message is in /usr/src/sys/nfs/nfs_bio.c, with this associated comment: /* * Yuck! The directory has been modified on the * server. The only way to get the block is by * reading from the beginning to get all the * offset cookies. * * Leave the last bp intact unless there is an error. * Loop back up to the while if the error is another * NFSERR_BAD_COOKIE (double yuch!). */ Is this an error that I need to worry about? Is this just my NFS client loosing track of some bookkeeping details? -- Brian 'you Bastard' Reichert 37 Crystal Ave. #303 Daytime number: (603) 434-6842 Derry NH 03038-1713 USA Intel architecture: the left-hand path To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 6:45:18 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 57AFC37B409 for ; Tue, 25 Sep 2001 06:45:15 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.4/8.11.4) with ESMTP id f8PDgpv28865; Tue, 25 Sep 2001 15:43:04 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: "Dmitry S. Rzhavin" Cc: hackers@FreeBSD.ORG Subject: Re: 4.4 jail and udp In-Reply-To: Your message of "Tue, 25 Sep 2001 16:32:58 +0400." <3BB0797A.FB060034@rt.ru> Date: Tue, 25 Sep 2001 15:42:51 +0200 Message-ID: <28863.1001425371@critter> From: Poul-Henning Kamp Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <3BB0797A.FB060034@rt.ru>, "Dmitry S. Rzhavin" writes: >Hi! >Looks like jailed processes can't use udp or icmp, but can use tcp: RTFM. UDP works fine. ICMP and other raw socket magic doesn't. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 7: 2:45 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from tomts9-srv.bellnexxia.net (tomts9.bellnexxia.net [209.226.175.53]) by hub.freebsd.org (Postfix) with ESMTP id 428FC37B409 for ; Tue, 25 Sep 2001 07:02:37 -0700 (PDT) Received: from xena.gsicomp.on.ca ([65.93.38.74]) by tomts9-srv.bellnexxia.net (InterMail vM.4.01.03.16 201-229-121-116-20010115) with ESMTP id <20010925140236.ABR1679.tomts9-srv.bellnexxia.net@xena.gsicomp.on.ca>; Tue, 25 Sep 2001 10:02:36 -0400 Received: from localhost (matt@localhost) by xena.gsicomp.on.ca (8.11.1/8.11.1) with ESMTP id f8PDtnU84464; Tue, 25 Sep 2001 09:55:49 -0400 (EDT) (envelope-from matt@xena.gsicomp.on.ca) Date: Tue, 25 Sep 2001 09:54:34 -0400 (EDT) From: Matthew Emmerton To: Brian Reichert Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: got bad cookie vp 0xe2e5ef80 bp 0xcf317328 In-Reply-To: <20010925093643.W49528@numachi.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG What OS is running on the NFS client and server? -- Matthew Emmerton || matt@gsicomp.on.ca GSI Computer Services || http://www.gsicomp.on.ca On Tue, 25 Sep 2001, Brian Reichert wrote: > I'm starting to see errors in /var/log/messages under 4.2-RELEASE: > > Sep 23 00:31:17 bmdb1 /kernel: got bad cookie vp 0xe2e5ef80 bp 0xcf317328 > > Not many, but more than one, and I've never seen this in my years > of using FreeBSD. > > The code producing this message is in /usr/src/sys/nfs/nfs_bio.c, > with this associated comment: > > /* > * Yuck! The directory has been modified on the > * server. The only way to get the block is by > * reading from the beginning to get all the > * offset cookies. > * > * Leave the last bp intact unless there is an error. > * Loop back up to the while if the error is another > * NFSERR_BAD_COOKIE (double yuch!). > */ > > Is this an error that I need to worry about? Is this just my NFS > client loosing track of some bookkeeping details? > > -- > Brian 'you Bastard' Reichert > 37 Crystal Ave. #303 Daytime number: (603) 434-6842 > Derry NH 03038-1713 USA Intel architecture: the left-hand path > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 7: 3: 4 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from balrog.rt.ru (balrog.rt.ru [195.161.0.169]) by hub.freebsd.org (Postfix) with ESMTP id BB8D037B411 for ; Tue, 25 Sep 2001 07:02:58 -0700 (PDT) Received: from rt.ru (localhost [127.0.0.1]) by balrog.rt.ru (8.9.3/8.9.3) with ESMTP id SAA15023 for ; Tue, 25 Sep 2001 18:01:47 +0400 (MSD) (envelope-from dima@rt.ru) Message-ID: <3BB08E4B.A2A8479F@rt.ru> Date: Tue, 25 Sep 2001 18:01:47 +0400 From: "Dmitry S. Rzhavin" Organization: Rostelecom X-Mailer: Mozilla 4.7 [en] (X11; I; FreeBSD 4.0-20000103-CURRENT i386) X-Accept-Language: ru, en MIME-Version: 1.0 To: hackers@FreeBSD.ORG Subject: Re: 4.4 jail and udp References: <28863.1001425371@critter> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > > RTFM. thank you very much! === === # man jail | g socket Formatting page, please wait...Done. jail.socket_unixiproute_only UNIX domain sockets, IPv4 addresses, and routing sockets. To enable # man jail | g raw Exit 1 # === === can you tell me please what poor user must read to know about magic words "raw socket"? ;) sorry... > > UDP works fine. ICMP and other raw socket magic doesn't. > ... and shouldn't? Will it work in future releases, or it is better to forget about icmp for me? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 7: 7: 3 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 6DC1D37B40E for ; Tue, 25 Sep 2001 07:06:51 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id KAA03605; Tue, 25 Sep 2001 10:06:41 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8PE6EN72757; Tue, 25 Sep 2001 10:06:14 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15280.36694.786500.622681@grasshopper.cs.duke.edu> Date: Tue, 25 Sep 2001 10:06:14 -0400 (EDT) To: Peter Wemm Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: ecc on i386 In-Reply-To: <20010925012041.CC9613808@overcee.netplex.com.au> References: <15279.54029.454089.299807@grasshopper.cs.duke.edu> <20010925012041.CC9613808@overcee.netplex.com.au> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Wemm writes: Thanks for your description of how ECC is reported on PCs. That was very, very helpful. > The Tyan Thunder 2510 BIOS even disables ECC -> NMI routing so you have to > go to quite a bit of trouble to reprogram the serverworks chipset to > actually generate NMI's so that you can find out if something got trashed. Is that the He-Sl or the LE-3 chipset? Is that code available? I have some LE-3 based boxes which I'd like be certain DTRT. Unlike my wife's Dual Athlon, these boxes have nothing in their BIOS pertaining to ECC error reporting. (Supermicro 370-DLE) > Our NMI / ECC handling really really sucks in FreeBSD. Consider: > - i686_pagezero - reads before writing in order to minimize cache snooping > traffic in SMP systems. However, if it gets an NMI while trying to check > if the cache line is already zero, it will take the entire machine down > instead of just zeroing the line. > - NFS / VM / bio: when they get an NMI while trying to copy data that is > clean and backed by storage, they take the machine down instead of trying > to recover and re-read the page. > - userland.. If userland gets an NMI, the machine dies instead of killing > the process (or rereading a text page etc if possible) > - our NMI handlers are a festering pile of excretement. They dont have > the code to 'ack' the NMI so it isn't possible to return after recovery. > - and so on. Well, at least we take the machine down, which is a heck of a lot better than ignoring the problem, which is really all that I was hoping for. Thanks again, Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 7: 9:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.numachi.com (numachi.numachi.com [198.175.254.2]) by hub.freebsd.org (Postfix) with SMTP id C0AC137B40F for ; Tue, 25 Sep 2001 07:09:07 -0700 (PDT) Received: (qmail 25894 invoked by uid 3001); 25 Sep 2001 14:09:06 -0000 Received: from natto.numachi.com (198.175.254.216) by numachi.numachi.com with SMTP; 25 Sep 2001 14:09:06 -0000 Received: (qmail 42872 invoked by uid 1001); 25 Sep 2001 14:09:06 -0000 Date: Tue, 25 Sep 2001 10:09:06 -0400 From: Brian Reichert To: Matthew Emmerton Cc: Brian Reichert , freebsd-hackers@FreeBSD.ORG Subject: Re: got bad cookie vp 0xe2e5ef80 bp 0xcf317328 Message-ID: <20010925100906.X49528@numachi.com> References: <20010925093643.W49528@numachi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from matt@gsicomp.on.ca on Tue, Sep 25, 2001 at 09:54:34AM -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Sep 25, 2001 at 09:54:34AM -0400, Matthew Emmerton wrote: > > What OS is running on the NFS client and server? My client is the 4.2-RELEASE box in question. There are several servers, all of which (at this point) are Netapps. > -- > Matthew Emmerton || matt@gsicomp.on.ca > GSI Computer Services || http://www.gsicomp.on.ca -- Brian 'you Bastard' Reichert 37 Crystal Ave. #303 Daytime number: (603) 434-6842 Derry NH 03038-1713 USA Intel architecture: the left-hand path To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 8:22:33 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from r220-1.rz.RWTH-Aachen.DE (r220-1.rz.RWTH-Aachen.DE [134.130.3.31]) by hub.freebsd.org (Postfix) with ESMTP id 03B3337B41C for ; Tue, 25 Sep 2001 08:22:26 -0700 (PDT) Received: from r220-1.rz.RWTH-Aachen.DE (relay2.RWTH-Aachen.DE [134.130.3.1]) by r220-1.rz.RWTH-Aachen.DE (8.10.1/8.11.3-2) with ESMTP id f8PFMPY27619; Tue, 25 Sep 2001 17:22:26 +0200 (MEST) Received: from kawoserv.kawo2.rwth-aachen.de (root@kawoserv.kawo2.RWTH-Aachen.DE [134.130.180.1]) by r220-1.rz.RWTH-Aachen.DE (8.10.1/8.11.3/6) with ESMTP id f8PFMLc27612; Tue, 25 Sep 2001 17:22:23 +0200 (MEST) Received: from fump.kawo2.rwth-aachen.de (root@fump.kawo2.rwth-aachen.de [134.130.181.148]) by kawoserv.kawo2.rwth-aachen.de (8.9.3/8.9.3) with ESMTP id RAA18201; Tue, 25 Sep 2001 17:22:19 +0200 Received: (from alex@localhost) by fump.kawo2.rwth-aachen.de (8.11.3/8.11.3) id f8PFMIC02934; Tue, 25 Sep 2001 17:22:18 +0200 (CEST) (envelope-from alex) Date: Tue, 25 Sep 2001 17:22:17 +0200 From: Alexander Langer To: Peter Wemm Cc: Andrew Gallatin , freebsd-hackers@FreeBSD.ORG Subject: Re: ecc on i386 Message-ID: <20010925172217.C2276@fump.kawo2.rwth-aachen.de> References: <15279.54029.454089.299807@grasshopper.cs.duke.edu> <20010925012041.CC9613808@overcee.netplex.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010925012041.CC9613808@overcee.netplex.com.au>; from peter@wemm.org on Mon, Sep 24, 2001 at 06:20:41PM -0700 X-PGP-Fingerprint: 44 28 CA 4C 46 5B D3 A8 A8 E3 BA F3 4E 60 7D 7F X-PGP-at: finger alex@big.endian.de X-Verwirrung: Dieser Header dient der allgemeinen Verwirrung. Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake Peter Wemm (peter@wemm.org): > Our NMI / ECC handling really really sucks in FreeBSD. Consider: [...] Is there any effort to fix this stuff? Considering FreeBSD is still known as one of the best server platforms, this is more important than a multi-threaded kernel or similar stuff, IMO. Alex To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 8:59:15 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from rover.village.org (rover.bsdimp.com [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id B52BA37B50C for ; Tue, 25 Sep 2001 08:55:30 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.3/8.11.3) with ESMTP id f8PFtTu41547; Tue, 25 Sep 2001 09:55:29 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost [127.0.0.1]) by harmony.village.org (8.11.6/8.11.6) with ESMTP id f8PFtT761298; Tue, 25 Sep 2001 09:55:29 -0600 (MDT) (envelope-from imp@harmony.village.org) Message-Id: <200109251555.f8PFtT761298@harmony.village.org> To: Peter Wemm Subject: Re: ecc on i386 Cc: Andrew Gallatin , freebsd-hackers@FreeBSD.ORG In-reply-to: Your message of "Mon, 24 Sep 2001 18:20:41 PDT." <20010925012041.CC9613808@overcee.netplex.com.au> References: <20010925012041.CC9613808@overcee.netplex.com.au> Date: Tue, 25 Sep 2001 09:55:29 -0600 From: Warner Losh Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20010925012041.CC9613808@overcee.netplex.com.au> Peter Wemm writes: : - our NMI handlers are a festering pile of excretement. They dont have : the code to 'ack' the NMI so it isn't possible to return after recovery. I have code to do the ack, but people have complained in the past that this code is too chipset specific so I've never committed it. :-( Being able to break to debugger with the NMI switch multiple times sure is nice, however. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 9:40:54 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id 4C26737B408 for ; Tue, 25 Sep 2001 09:36:13 -0700 (PDT) Received: from mail.cicely.de (cicely20 [10.1.1.22]) by srv1.cosmo-project.de (8.11.0/8.11.0) with ESMTP id f8PGYK695210; Tue, 25 Sep 2001 18:34:20 +0200 (CEST) Received: (from ticso@localhost) by mail.cicely.de (8.11.0/8.11.0) id f8PGYuo29417; Tue, 25 Sep 2001 18:34:56 +0200 (CEST) Date: Tue, 25 Sep 2001 18:34:55 +0200 From: Bernd Walter To: Bakul Shah Cc: Bernd Walter , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? Message-ID: <20010925183455.A29324@cicely20.cicely.de> References: <20010925095607.B27615@cicely20.cicely.de> <200109251610.MAA18919@warspite.cnchost.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109251610.MAA18919@warspite.cnchost.com>; from bakul@bitblocks.com on Tue, Sep 25, 2001 at 09:10:21AM -0700 X-Operating-System: NetBSD cicely20.cicely.de 1.5 sparc Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Sep 25, 2001 at 09:10:21AM -0700, Bakul Shah wrote: > > > FWIW, in a Unix port we did I remember putting the user > > > struct *above* the kernel stack. The stack grew down so you > > > hit the red zone (the guard pages) without clobbering the > > > user struct. Since struct user _ended_ on a page boundary, > > > its size was needed at locore.s assembly time but that was a > > > small price to pay for the added safety. > > > > I don't think a guard page can help here, because the page fault > > handler needs a working stack. > > You can't continue if you run out of the stack in any case > but the issue is what happens when you run off the end of the > stack. In the FreeBSD case you trash the user struct and > discover this problem in a very indirect way and possibly > after trashing god-knows-what-else. No Doubt about it. But you missed my point completely. Nevertheless PHK already explained why it still help. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 9:59:32 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by hub.freebsd.org (Postfix) with ESMTP id 7D9D437B40E for ; Tue, 25 Sep 2001 09:55:43 -0700 (PDT) Received: from isi.edu (hbo.isi.edu [128.9.160.75]) by boreas.isi.edu (8.11.6/8.11.2) with ESMTP id f8PGshC01015; Tue, 25 Sep 2001 09:54:43 -0700 (PDT) Message-ID: <3BB0B6D3.4050809@isi.edu> Date: Tue, 25 Sep 2001 09:54:43 -0700 From: Lars Eggert User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.4) Gecko/20010924 X-Accept-Language: en, de MIME-Version: 1.0 To: Matthew Emmerton Cc: Brian Reichert , freebsd-hackers@FreeBSD.ORG Subject: NFS append race (was Re: got bad cookie vp 0xe2e5ef80 bp 0xcf317328) References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [Should have included this in my earlier mail, sorry.] In addition to bad cookies, I also see the following messages very frequently: NFS append race @0:954 They're issued in nfs/nfs_bio.c: /* * If dirtyend exceeds file size, chop it down. This should * not normally occur but there is an append race where it * might occur XXX, so we log it. * * If the chopping creates a reverse-indexed or degenerate * situation with dirtyoff/end, we 0 both of them. */ if (bp->b_dirtyend > bcount) { printf("NFS append race @%lx:%d\n", (long)bp->b_blkno * DEV_BSIZE, bp->b_dirtyend - bcount); bp->b_dirtyend = bcount; } Any idea what these are about? Lars -- Lars Eggert Information Sciences Institute http://www.isi.edu/larse/ University of Southern California To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 10: 3:57 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 728E437B412 for ; Tue, 25 Sep 2001 09:59:17 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8PGwMM22481 for ; Tue, 25 Sep 2001 09:58:22 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 157B03809; Tue, 25 Sep 2001 09:58:22 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Andrew Gallatin Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: ecc on i386 In-Reply-To: <15280.36694.786500.622681@grasshopper.cs.duke.edu> Date: Tue, 25 Sep 2001 09:58:22 -0700 From: Peter Wemm Message-Id: <20010925165822.157B03809@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Andrew Gallatin wrote: > > Peter Wemm writes: > > > Thanks for your description of how ECC is reported on PCs. That was > very, very helpful. > > > The Tyan Thunder 2510 BIOS even disables ECC -> NMI routing so you have to > > go to quite a bit of trouble to reprogram the serverworks chipset to > > actually generate NMI's so that you can find out if something got trashed. > > Is that the He-Sl or the LE-3 chipset? Is that code available? > I have some LE-3 based boxes which I'd like be certain DTRT. LE-3 is the one we've been using and the stuff I've tested my hackery with. The main problem is that it currently uses magic bit arithmetic rather than using defined values. I'll clean it up and get it out. I am pretty sure it will work for all the serverworks chips, since the docs for various different chips all describe the same interface. Similarly, the Intel 440BX/GX use the same interface, and I suspect the later ones will as well. We have ECC/NMI drivers for at least the BX/GX as well. > Unlike my wife's Dual Athlon, these boxes have nothing in their > BIOS pertaining to ECC error reporting. (Supermicro 370-DLE) Serverworks say that ECC *must* be turned on in their manuals. However, whether the bioses do this is another thing. > > Our NMI / ECC handling really really sucks in FreeBSD. Consider: > > - i686_pagezero - reads before writing in order to minimize cache snooping > > traffic in SMP systems. However, if it gets an NMI while trying to check > > if the cache line is already zero, it will take the entire machine down > > instead of just zeroing the line. > > - NFS / VM / bio: when they get an NMI while trying to copy data that is > > clean and backed by storage, they take the machine down instead of trying > > to recover and re-read the page. > > - userland.. If userland gets an NMI, the machine dies instead of killing > > the process (or rereading a text page etc if possible) > > - our NMI handlers are a festering pile of excretement. They dont have > > the code to 'ack' the NMI so it isn't possible to return after recovery. > > - and so on. > > Well, at least we take the machine down, which is a heck of a lot > better than ignoring the problem, which is really all that I was > hoping for. I'll email you some code and start doing some cleanup. > Thanks again, > > Drew Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 10: 5:28 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from smtp.noos.fr (racine.noos.net [212.198.2.71]) by hub.freebsd.org (Postfix) with ESMTP id 1646D37B40E for ; Tue, 25 Sep 2001 10:01:44 -0700 (PDT) Received: (qmail 73984857 invoked by uid 0); 25 Sep 2001 16:30:24 -0000 Received: from unknown (HELO gits.dyndns.org) ([212.198.231.187]) (envelope-sender ) by 212.198.2.71 (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 25 Sep 2001 16:30:24 -0000 Received: (from root@localhost) by gits.dyndns.org (8.11.6/8.11.6) id f8PGUNQ43111; Tue, 25 Sep 2001 18:30:23 +0200 (CEST) (envelope-from root) Message-Id: <200109251630.f8PGUNQ43111@gits.dyndns.org> Subject: Re: termcap sources In-Reply-To: <200109250949.f8P9nJT26069@gits.dyndns.org> To: clefevre@citeweb.net Date: Tue, 25 Sep 2001 18:30:23 +0200 (CEST) Cc: Giorgos Keramidas , hackers@freebsd.org Reply-To: clefevre@citeweb.net From: Cyrille Lefevre Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[<97Zd*>^#%Y5Cxv;%Y[PT-LW3;A:fRrJ8+^k"e7@+30g0YD0*^^3jgyShN7o?a]C la*Zv'5NA,=963bM%J^o]C X-Mailer: ELM [version 2.4ME+ PL94b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Cyrille Lefevre wrote: > Giorgos Keramidas wrote: > [snip] > > As you can see there are quite a few terminals that have capabilities defined > > more than once! I don't have THAT many terminals to check, but I'm open to > > suggestions. Should we do something about this? If yes, what? > > this isn't a problem since only the first matching capability is used. > others are ignored. I've a pending termcap update related to > http://www.tuxedo.org/terminfo. for instance, I don't remember if > reorder works w/ termtypes.tc ? done, see PR #30812 : http://www.FreeBSD.org/cgi/query-pr.cgi?pr=30812 Cyrille. -- Cyrille Lefevre mailto:clefevre@citeweb.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 10:13:21 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from boreas.isi.edu (boreas.isi.edu [128.9.160.161]) by hub.freebsd.org (Postfix) with ESMTP id 62B6237B40A for ; Tue, 25 Sep 2001 10:10:34 -0700 (PDT) Received: from isi.edu (hbo.isi.edu [128.9.160.75]) by boreas.isi.edu (8.11.6/8.11.2) with ESMTP id f8PGoNC29706; Tue, 25 Sep 2001 09:50:23 -0700 (PDT) Message-ID: <3BB0B5CE.70801@isi.edu> Date: Tue, 25 Sep 2001 09:50:22 -0700 From: Lars Eggert User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.4) Gecko/20010924 X-Accept-Language: en, de MIME-Version: 1.0 To: Matthew Emmerton Cc: Brian Reichert , freebsd-hackers@FreeBSD.ORG Subject: Re: got bad cookie vp 0xe2e5ef80 bp 0xcf317328 References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Emmerton wrote: > What OS is running on the NFS client and server? I see these too, with a FreeBSD-4.4 client and SunOS 5.5.1 servers. Seen them with FreeBSD-4.2 clients to the same servers as well. Lars -- Lars Eggert Information Sciences Institute http://www.isi.edu/larse/ University of Southern California To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 11:14:38 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from robin.mail.pas.earthlink.net (robin.mail.pas.earthlink.net [207.217.120.65]) by hub.freebsd.org (Postfix) with ESMTP id C180637B415 for ; Tue, 25 Sep 2001 11:11:39 -0700 (PDT) Received: from mindspring.com (dialup-209.245.141.174.Dial1.SanJose1.Level3.net [209.245.141.174]) by robin.mail.pas.earthlink.net (8.11.5/8.9.3) with ESMTP id f8PIBZW21296; Tue, 25 Sep 2001 11:11:35 -0700 (PDT) Message-ID: <3BB0C907.4F6AA199@mindspring.com> Date: Tue, 25 Sep 2001 11:12:23 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Attila Nagy Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Disk based file system cache References: <20010918165302.V17360-100000@scribble.fsn.hu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Attila Nagy wrote: > > Hello, > > I'm just curious: is it possible to set up an NFS server and a client > where the client has very big (28 GB maximum for FreeBSD?) swap area on > multiple disks and caches the NFS exported data on it? > This could save a lot of bandwidth on the NFS server and also redues load > on that. There is a configuration called "dataless", in which you have local swap for an NFS booted system; this has been supported by SunOS since 1991/1992. It can also be used with FreeBSD, with some rc file tweaking (FreeBSD has seemingly resisted rc file changes to make this kind of division easier, as well as diskless). You are also allowed locacl data storage, with the idea that the OS and utilities, etc. all come off the remote server. The major value would be to mark the VFS type as "precache executables to swap"... the main failure mode for diskless and dataless SunOS machines has historically been the fact that, when the machine when to swap in a page in an exectable, the server was down, and therefore, all the engineers sit and twiddle their thumbs while their machine is locked up sitting in the page-in path. To combat this, you could have an attribute flag on the FS that indicated it was remote, and thus triggered local swap caching of the executable file image in its entirety, so that demand paging over the network was held to a minimum. This would permit people to continue to do work during a server outage, since the pages will be there. This is an idea I've suggested before. If, on the other hand, you are asking for caching of data file contents for writable files (unlike executables, which are read-only except for the server, since when you run a program, you do not tend to write to its image), then the answer is "no, not unless you implement NFSv4". The problem is that, prior to NFSv4, there was not a working distributed cache coherency protocol, so locally cached data can become stale; writebacks to the server by one client can therefore overwrite adjacent but unrelated data written back by another client, if such writes are not restricted to page boundaries (or worse). So that answer is that any system that does this, risks the corruption of its data in the common case (even the simplest case, a mail server whose mailbox files are accessed by a single client machine at a time, will get corruption). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 12:27:23 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 1D1CB37B40C for ; Tue, 25 Sep 2001 12:26:16 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8PJOHW03718; Tue, 25 Sep 2001 12:24:17 -0700 (PDT) (envelope-from dillon) Date: Tue, 25 Sep 2001 12:24:17 -0700 (PDT) From: Matt Dillon Message-Id: <200109251924.f8PJOHW03718@earth.backplane.com> To: Mike Silbersack Cc: Ian Dowse , Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <20010924213518.G70783-100000@achilles.silby.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Ok, time to take a good stab at sticking my foot in my mouth here. : :Would it be possible to have a kernel mode where the read-only bit was :turned on for malloc pools which shouldn't currently be accessed? This :could be gated through the spl() calls (or specific mutexes on -current), :ensuring that something like getpid couldn't stomp on the vm structures :w/o first doing a splvm(). Kinda sounds like Multics :-)... no, it would be too messy trying to protect kernel structures in one subsystem from another. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 12:27:37 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 612EC37B40C for ; Tue, 25 Sep 2001 12:27:35 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8PJN3103708; Tue, 25 Sep 2001 12:23:03 -0700 (PDT) (envelope-from dillon) Date: Tue, 25 Sep 2001 12:23:03 -0700 (PDT) From: Matt Dillon Message-Id: <200109251923.f8PJN3103708@earth.backplane.com> To: Peter Wemm Cc: Ian Dowse , Julian Elischer , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <20010925032250.57FF03808@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :I had been contemplating making a fake 'struct user' in userland only in :order to keep the a.out coredump reader code happy. The a.out coredump :code (see cpu_coredump() in */*/vm_machdep.c) can generate this fake :structure in order to keep gdb happy. But then I realized that a.out :coredump debugging was almost totally irrelevant these days. : :Cheers, :-Peter :-- :Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au Hmm. How about this... if we keep the guard field at the end of struct user we could #ifdef _KERNEL it so userland doesn't notice it. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 12:47: 4 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 8DCDA37BA5F for ; Tue, 25 Sep 2001 12:47:01 -0700 (PDT) Received: from elischer.org (InterJet.elischer.org [192.168.1.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA56441; Tue, 25 Sep 2001 13:32:51 -0700 (PDT) Message-ID: <3BB0DD6F.908C6A08@elischer.org> Date: Tue, 25 Sep 2001 12:39:27 -0700 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Matt Dillon Cc: Peter Wemm , Ian Dowse , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <20010925032250.57FF03808@overcee.netplex.com.au> <200109251923.f8PJN3103708@earth.backplane.com> Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matt Dillon wrote: > > :I had been contemplating making a fake 'struct user' in userland only in > :order to keep the a.out coredump reader code happy. The a.out coredump > :code (see cpu_coredump() in */*/vm_machdep.c) can generate this fake > :structure in order to keep gdb happy. But then I realized that a.out > :coredump debugging was almost totally irrelevant these days. > : > :Cheers, > :-Peter > :-- > :Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au > > Hmm. How about this... if we keep the guard field at the end of > struct user we could #ifdef _KERNEL it so userland doesn't notice it. > > -Matt So, Matt, does this solve the original question? (VM Corruption) or is it just a fruitful red-herring? -- +------------------------------------+ ______ _ __ | __--_|\ Julian Elischer | \ U \/ / hard at work in | / \ julian@elischer.org +------>x USA \ a very strange | ( OZ ) \___ ___ | country ! +- X_.---._/ presently in San Francisco \_/ \\ v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 13:16:28 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 7045937B409 for ; Tue, 25 Sep 2001 13:16:25 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8PKFsI04211; Tue, 25 Sep 2001 13:15:54 -0700 (PDT) (envelope-from dillon) Date: Tue, 25 Sep 2001 13:15:54 -0700 (PDT) From: Matt Dillon Message-Id: <200109252015.f8PKFsI04211@earth.backplane.com> To: Julian Elischer Cc: Peter Wemm , Ian Dowse , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <20010925032250.57FF03808@overcee.netplex.com.au> <200109251923.f8PJN3103708@earth.backplane.com> <3BB0DD6F.908C6A08@elischer.org> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :So, Matt, does this solve the original question? (VM Corruption) or :is it just a fruitful red-herring? :-- :+------------------------------------+ ______ _ __ :| __--_|\ Julian Elischer | \ U \/ / hard at work in It seems unlikely to me, but you never know. Certainly this is a problem that has to be fixed now. I've bumped -stable's UPAGES up to 3 but we absolutely have to MFC the fixes for the two devices allocating 2K stacks. Maybe I should have bumped UPAGES up to 4 :-) -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 13:32:25 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.twwells.com (mail.twwells.com [64.38.247.128]) by hub.freebsd.org (Postfix) with ESMTP id 72AF937B401 for ; Tue, 25 Sep 2001 13:32:23 -0700 (PDT) Received: from xfermail (helo=mail.twwells.com) by mail.twwells.com with local-bsmtp (Exim 3.32 #1) id 15lysI-0007a4-00 for freebsd-hackers@freebsd.org; Tue, 25 Sep 2001 13:32:14 -0700 X-Filter-Status: mail.twwells.com ok 19 Received: from twwells.com ( [65.14.140.228] ) by mail.twwells.com via tcp with esmtp id 3bb0e9ba-007172; Tue, 25 Sep 2001 20:31:54 +0000 Received: from root by twwells.com with local (Exim 3.33 #1) id 15lyrt-0002DD-00; Tue, 25 Sep 2001 16:31:49 -0400 To: Peter Wullinger Cc: so@i-clue.de, freebsd-hackers@freebsd.org From: admin@twwells.com Subject: Re: missing words, lots of them References: <20010925130240.A2939@pc04.ipc-kallmuenz.de> Message-Id: Date: Tue, 25 Sep 2001 16:31:49 -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > I know. :) However, there are several lexicons that have > > acceptable copyrights. E.g., the Moby list, though I have some > > reservations about it, is public domain. So there's no good reason > > to live with an archaic list. > > What excactly does `acceptable' mean for you ... The BSD copyright > gives very wide freedom at the use of software, the GPL gives a lot > less ... Well, I had meant "acceptable to the project". However, my personal preference is a BSD-style copyright. Anyway, it's unclear to me who would have to pass on any list I might produce; anyone willing to 'fess up to being responsible for those bits? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 14: 7: 3 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id B8CFE37B410 for ; Tue, 25 Sep 2001 14:06:59 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA56785; Tue, 25 Sep 2001 14:55:18 -0700 (PDT) Date: Tue, 25 Sep 2001 14:55:17 -0700 (PDT) From: Julian Elischer To: Paul Saab Cc: hackers@freebsd.org Subject: librsa and 4.4 In-Reply-To: <20010925134846.A67402@elvis.mu.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In a related problem: we have a set of 4.1.1 binaries we want ot run on 4.4 but they (apache+other stuff) want to find a librsaUSA.so but can't.. I fixed it by copying the one from 4.1.1 into /usr/lib/compat. Is that the right answer? Is it possible we can have a compat librsa? (maybe even empty if the stuff is now in libcrypt or something). On Tue, 25 Sep 2001, Paul Saab wrote: > set COMPAT4X=yes in make.conf > > cd /usr/lib/compat > > make && make install > > Julian Elischer (julian@elischer.org) wrote: > > > > I seem to have cvsupp'd at a bad moment.. > > > > various older ( e.g. Netscape) have the following problem: > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > "__stderrp" > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > "__stderrp" > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > "__stderrp" > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > "__stderrp" > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > "__stderrp" > > /usr/libexec/ld-elf.so.1: /usr/lib/libm.so.2: Undefined symbol "__stderrp" > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > "__stderrp" > > /usr/libexec/ld-elf.so.1: /usr/lib/libm.so.2: Undefined symbol "__stderrp" > > > > > > so I figure: > > "I'll cvsup again.." > > but: > > + /usr/local/bin/cvsup -L1 -P - /tmp/501.supfile > > /usr/libexec/ld-elf.so.1: /usr/lib/libutil.so.3: Undefined symbol > > "__stdoutp" > > > > > > Is here a quick workaround? > > (I still have connectivity and CVS just no netscape, KDE or cvsup..) > > (by workaround I mean a manual patch I should apply somewhere > > to get me up and going again after another buildworld). > > > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-current" in the body of the message > > -- > Paul Saab > Technical Yahoo > ps@mu.org - ps@yahoo-inc.com - ps@freebsd.org > Do You .. uhh .. Yahoo!? > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 14:54:56 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 3A02E37B40D for ; Tue, 25 Sep 2001 14:54:52 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f8PLspM23154 for ; Tue, 25 Sep 2001 14:54:52 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id D21BA3809; Tue, 25 Sep 2001 14:54:51 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Julian Elischer Cc: Paul Saab , hackers@FreeBSD.ORG Subject: Re: librsa and 4.4 In-Reply-To: Date: Tue, 25 Sep 2001 14:54:51 -0700 From: Peter Wemm Message-Id: <20010925215451.D21BA3809@overcee.netplex.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer wrote: > > In a related problem: > > we have a set of 4.1.1 binaries we want ot run on 4.4 > but they (apache+other stuff) want to find a librsaUSA.so > but can't.. I fixed it by copying the one from 4.1.1 into /usr/lib/compat. > Is that the right answer? > Is it possible we can have a compat librsa? > (maybe even empty if the stuff is now in libcrypt or something). libcrypto.so.1.gz.uu and libssl.so.1.gz.uu need to be MFC'ed. This should have been done before 4.4-REL as well. > On Tue, 25 Sep 2001, Paul Saab wrote: > > > set COMPAT4X=yes in make.conf > > > > cd /usr/lib/compat > > > > make && make install > > > > Julian Elischer (julian@elischer.org) wrote: > > > > > > I seem to have cvsupp'd at a bad moment.. > > > > > > various older ( e.g. Netscape) have the following problem: > > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > > "__stderrp" > > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > > "__stderrp" > > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > > "__stderrp" > > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > > "__stderrp" > > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > > "__stderrp" > > > /usr/libexec/ld-elf.so.1: /usr/lib/libm.so.2: Undefined symbol "__stderrp " > > > /usr/libexec/ld-elf.so.1: /usr/lib/libstdc++.so.3: Undefined symbol > > > "__stderrp" > > > /usr/libexec/ld-elf.so.1: /usr/lib/libm.so.2: Undefined symbol "__stderrp " > > > > > > > > > so I figure: > > > "I'll cvsup again.." > > > but: > > > + /usr/local/bin/cvsup -L1 -P - /tmp/501.supfile > > > /usr/libexec/ld-elf.so.1: /usr/lib/libutil.so.3: Undefined symbol > > > "__stdoutp" > > > > > > > > > Is here a quick workaround? > > > (I still have connectivity and CVS just no netscape, KDE or cvsup..) > > > (by workaround I mean a manual patch I should apply somewhere > > > to get me up and going again after another buildworld). > > > > > > > > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > > with "unsubscribe freebsd-current" in the body of the message > > > > -- > > Paul Saab > > Technical Yahoo > > ps@mu.org - ps@yahoo-inc.com - ps@freebsd.org > > Do You .. uhh .. Yahoo!? > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > > Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 15:15:54 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id E488A37B40F for ; Tue, 25 Sep 2001 15:15:50 -0700 (PDT) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_22672)/8.9.3) with ESMTP id IAA14955; Wed, 26 Sep 2001 08:15:41 +1000 (EST) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37645) with ESMTP id <01K8SAYG42QO22XQ9D@cim.alcatel.com.au>; Wed, 26 Sep 2001 08:15:35 +1000 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.1/8.11.1) id f8PMFc153708; Wed, 26 Sep 2001 08:15:38 +1000 (EST envelope-from jeremyp) Content-return: prohibited Date: Wed, 26 Sep 2001 08:15:38 +1000 From: Peter Jeremy Subject: Re: VM Corruption - stumped, anyone have any ideas? In-reply-to: ; from owner-freebsd-hackers-digest@FreeBSD.ORG on Tue, Sep 25, 2001 at 04:47:47AM -0700 To: Matt Dillon Cc: hackers@FreeBSD.ORG Message-id: <20010926081538.L75481@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2001-Sep-25 04:47:47 -0700, freebsd-hackers-digest wrote: On Mon, 24 Sep 2001 14:13:37 -0700 (PDT), Matt Dillon wrote: > This is very similar to the corruption I found on one of Yahoo's > machines. Except on that machine two bits were changed. It's as though > some other subsystem is trying to manipulate a flag in a structure using > a bad structure pointer. I'm not sure how practical this is, and it assumes that the machine can survive with around half the current KVA... How about changing the kernel memory allocation routines to only use every second page, with the other pages unmapped. Obviously large arrays would need to be allocated in a single blob without unmapped pages in the middle of them, but I believe most of the kernel data structures are handled as linked lists, so even some of the large arrays may be amenable to having holes in them. This would increase the chance that a stray off-by-1 index would wind up hitting an unmapped page. Of course, it doesn't help if the problem is that the pointer is valid, it just isn't pointing to the expected object (careful examination of explicit casts, combined with lint/gcc should uncover any of these). What we need is run-time type identification (RTTI) (at least as a debugging option). The only problem is that I don't know of any tools to automatically add RTTI to C and doing it manually would be extremely expensive in developer time. It would be technically feasible to modify GCC to add an RTTI field to every structure and verify it when de-referencing pointers, but that's not a trivial undertaking - the bounds-checking patches to gcc2.7.2 comprised about 160K of diffs and new code, together with 380K of support library code and I suspect RTTI is a similar order of complexity. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 15:48:19 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 1709A37B401 for ; Tue, 25 Sep 2001 15:48:16 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8PMjpd05302; Tue, 25 Sep 2001 15:45:51 -0700 (PDT) (envelope-from dillon) Date: Tue, 25 Sep 2001 15:45:51 -0700 (PDT) From: Matt Dillon Message-Id: <200109252245.f8PMjpd05302@earth.backplane.com> To: Peter Jeremy Cc: hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? References: <20010926081538.L75481@gsmx07.alcatel.com.au> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I really don't think it is necessary to hack up GCC to figure out stack utilization. We have issues with only a few drivers and it is fairly trivial (as my patch shows) to throw a pattern into the kernel stack to determine how much is actually used. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 15:52:42 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from warspite.cnchost.com (warspite.concentric.net [207.155.248.9]) by hub.freebsd.org (Postfix) with ESMTP id 95A1637B401 for ; Tue, 25 Sep 2001 15:52:38 -0700 (PDT) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by warspite.cnchost.com id MAA18919; Tue, 25 Sep 2001 12:10:30 -0400 (EDT) [ConcentricHost SMTP Relay 1.14] Message-ID: <200109251610.MAA18919@warspite.cnchost.com> To: Bernd Walter Cc: hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-reply-to: Your message of "Tue, 25 Sep 2001 09:56:07 +0200." <20010925095607.B27615@cicely20.cicely.de> Date: Tue, 25 Sep 2001 09:10:21 -0700 From: Bakul Shah Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > FWIW, in a Unix port we did I remember putting the user > > struct *above* the kernel stack. The stack grew down so you > > hit the red zone (the guard pages) without clobbering the > > user struct. Since struct user _ended_ on a page boundary, > > its size was needed at locore.s assembly time but that was a > > small price to pay for the added safety. > > I don't think a guard page can help here, because the page fault > handler needs a working stack. You can't continue if you run out of the stack in any case but the issue is what happens when you run off the end of the stack. In the FreeBSD case you trash the user struct and discover this problem in a very indirect way and possibly after trashing god-knows-what-else. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 16: 7: 0 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 6C2AF37B415 for ; Tue, 25 Sep 2001 16:06:58 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id QAA57216; Tue, 25 Sep 2001 16:59:07 -0700 (PDT) Date: Tue, 25 Sep 2001 16:59:06 -0700 (PDT) From: Julian Elischer To: Bakul Shah Cc: Bernd Walter , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109251610.MAA18919@warspite.cnchost.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 25 Sep 2001, Bakul Shah wrote: > > You can't continue if you run out of the stack in any case > but the issue is what happens when you run off the end of the > stack. In the FreeBSD case you trash the user struct and > discover this problem in a very indirect way and possibly > after trashing god-knows-what-else. in -current we have the guard page below, and the uarea above the stack.. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 16: 9:24 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 5EBCE37B407 for ; Tue, 25 Sep 2001 16:09:21 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id QAA57214; Tue, 25 Sep 2001 16:57:02 -0700 (PDT) Date: Tue, 25 Sep 2001 16:57:01 -0700 (PDT) From: Julian Elischer To: Matt Dillon Cc: Peter Jeremy , hackers@FreeBSD.ORG Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109252245.f8PMjpd05302@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The only way to get people to change their code is to make it complain at them when they compile it. If a stack is big, then that's "someone-else's" problem... I was thinking to only turn it on every now and then,.. On Tue, 25 Sep 2001, Matt Dillon wrote: > I really don't think it is necessary to hack up GCC to figure > out stack utilization. We have issues with only a few drivers > and it is fairly trivial (as my patch shows) to throw a pattern > into the kernel stack to determine how much is actually used. > > -Matt > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Sep 25 16:22:35 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from cs.bu.edu (CS.BU.EDU [128.197.10.2]) by hub.freebsd.org (Postfix) with ESMTP id 609A337B40E for ; Tue, 25 Sep 2001 16:22:30 -0700 (PDT) Received: from csa.bu.edu (evms@csa [128.197.12.3]) by cs.bu.edu (8.10.1/8.10.1) with ESMTP id f8PNMSA23298; Tue, 25 Sep 2001 19:22:28 -0400 (EDT) Received: (from evms@localhost) by csa.bu.edu (8.10.1/8.10.1) id f8PNMP814656; Tue, 25 Sep 2001 19:22:25 -0400 (EDT) From: Evan Sarmiento MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15281.4529.188277.263940@csa.bu.edu> Date: Tue, 25 Sep 2001 19:22:25 -0400 (EDT) To: freebsd-hackers@freebsd.org, freebsd-current@freebsdorg.FreeBSD.ORG Subject: IMPORTANT!! Re: panic on mount In-Reply-To: References: <200109250019.RAA08335@windsor.research.att.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, Just to clarify things for everyone who may be having this probme: there is a panic on bootup with current, within the witness* code. You can avoid this by commenting out WITNESS in your kernel configuration and recompiling. It worked for me.. Hope this helps someone. Thanks, Evan John Baldwin writes: > > On 25-Sep-01 Bill Fenner wrote: > > > > I also started getting this error with recent kernels (in the last > > day or so). > > It looks like the mutex is really held since the mtx_assert before > witness_unlock didn't trigger. You can try turning witness off for the time > being as a workaround. I'm not sure why witness would be broken, however. > > > Mounting root from ufs:/dev/ad0s1a > > panic: lock (sleep mutex) vnode interlock not locked @ > > /usr/src/sys/kern/vfs_default.c:460 > > Debugger("panic") > > Stopped at Debugger+0x44: pushl %ebx > > db> t > > Debugger(c03c5bbb) at Debugger+0x44 > > panic(c03c8c40,c03c4b80,c03ccf20,c03cc8a0,1cc) at panic+0x70 > > witness_unlock(c7765f2c,8,c03cc8a0,1cc,c7765f2c,1,c03c4ba0,f6) at > > witness_unlock+0x1d0 > > _mtx_unlock_flags(c7765f2c,0,c03cc8a0,1cc,c0567bd0) at _mtx_unlock_flags+0x59 > > vop_nolock(c0567be8,c0567bf8,c02920c2,c0567be8,c0567d4c) at vop_nolock+0x24 > > vop_defaultop(c0567be8) at vop_defaultop+0x15 > > vn_lock(c7765ec0,20002,c049f7c4,c0567d4c,c1346680) at vn_lock+0xca > > ffs_mountfs(c7765ec0,c1351600,c049f7c4,c0446900,c0567d4c) at ffs_mountfs+0x7e > > ffs_mount(c1351600,0,0,0,c049f7c4) at ffs_mount+0x67 > > vfs_mountroot_try(c05447a8,c03cc48c) at vfs_mountroot_try+0x14e > > vfs_mountroot(0,564c00,564000,0,c012caac) at vfs_mountroot+0x5a > > mi_startup() at mi_startup+0x90 > > begin() at begin+0x43 > > > > I dunno how to get a dump from this point since kern.dumpdev hasn't been > > set.. > > > > Bill > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-current" in the body of the message > > -- > > John Baldwin -- http://www.FreeBSD.org/~jhb/ > PGP Key: http://www.baldwin.cx/~john/pgpkey.asc > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 0: 8:30 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from illuminati.is.co.za (illuminati.is.co.za [196.36.198.65]) by hub.freebsd.org (Postfix) with ESMTP id 99DF937B405 for ; Wed, 26 Sep 2001 00:08:26 -0700 (PDT) Received: by illuminati.is.co.za (Postfix, from userid 1001) id 5BAC5174; Wed, 26 Sep 2001 09:08:20 +0200 (SAST) Date: Wed, 26 Sep 2001 09:08:20 +0200 From: Geoff Rehmet To: freebsd-hackers@freebsd.org Subject: Problem with CMedia 8738 Message-ID: <20010926090820.A41392@illuminati.is.co.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I posted this query FreeBSD-multimedia earlier, in the hope that I might find some pointers there, but to no avail. I seem to have run into a problem with getting a CMedia sound card (CMI8738 chipset) working with 4.3-STABLE or 4.4-RELEASE. For any program that talks to /dev/dsp, only the SPDIF port is active, while the speakers work when using /dev/audio (no SPDIF then). Dmesg output includes the following: pcm0: port 0xd800-0xd8ff irq 11 at device 12.0 on pci0 And cat /dev/sndstat gives: redpoint:~% cat /dev/sndstat FreeBSD Audio Driver (newpcm) Jul 25 2001 21:52:10 Installed devices: pcm0: at io 0xd800 irq 11 (1p/1r channels duplex) I hope I am not missing anything really obvious. Any help on this one would be appreciated. Ciao, Geoff. -- Geoff Rehmet, Internet Solutions tel: +27-11-283-5462, fax: +27-11-283-5401 mobile: +27-83-292-5800 email: geoffr@is.co.za URL: http://www.is.co.za To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 7:20:18 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from musique.teaser.net (musique.teaser.net [213.91.2.11]) by hub.freebsd.org (Postfix) with ESMTP id 1F59D37B42F for ; Wed, 26 Sep 2001 07:20:06 -0700 (PDT) Received: from notbsdems.nantes.kisoft-services.com (chantilly.kisoft-services.com [193.56.60.242]) by musique.teaser.net (Postfix) with ESMTP id C4DF27252B; Wed, 26 Sep 2001 16:20:03 +0200 (CEST) Received: by notbsdems.nantes.kisoft-services.com (Postfix, from userid 1001) id DB31EE6B40; Wed, 26 Sep 2001 16:14:52 +0200 (CEST) To: Mailing List IPFilter Cc: Mailing List FreeBSD Hackers Subject: FreeBSD 4.4-RELEASE & ipf 3.4.20 freeze From: Eric Masson X-Operating-System: FreeBSD 4.4-RC i386 Date: Wed, 26 Sep 2001 16:14:52 +0200 Message-ID: <86g09aqbs3.fsf@notbsdems.nantes.kisoft-services.com> Lines: 323 User-Agent: Gnus/5.090004 (Oort Gnus v0.04) XEmacs/21.1 (Cuyahoga Valley) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, Since I installed my DSL gateway with FreeBSD 4.4-RELEASE, the DSL link freezes every 24 hours more or less, no more traffic possible at this moment. Setup is the following ed0: link to internal lan ed1: link to Alcatel SpeedTouch Home, no ip configuration tun0: link brought up by ppp(8) to the Internet. I've tried with both ppp integrated nat and ipnat, and the problem lasts. Output results given here were made with ppp(8) nat. netstat -i takes a long time (1 minute) to give a result after the line with tun0. Ipfilter and Netgraph are loaded from /boot/loader.conf Here are the relevant files and outputs : ## #kernel config # # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.246.2.20 2000/10/31 23:16:07 n_hibma Exp $ machine i386 cpu I486_CPU ident IBMPS1 maxusers 64 makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols options INET # InterNETworking options FFS # Berkeley Fast Filesystem options FFS_ROOT # FFS usable as root device [keep this!] options NFS # Network Filesystem options NFS_NOSERVER # Network Filesystem server disabled options PROCFS # Process filesystem options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!] options USERCONFIG # boot -c editor options ICMP_BANDLIM # Rate limit bad replies options KBD_INSTALL_CDEV # install a CDEV entry in /dev # Isa bus device isa # Floppy drives device fdc0 at isa? port IO_FD1 irq 6 drq 2 device fd0 at fdc0 drive 0 # ATA and ATAPI devices device ata0 at isa? port IO_WD1 irq 14 device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives # Console control device atkbdc0 at isa? port IO_KBD device atkbd0 at atkbdc? irq 1 flags 0x1 device psm0 at atkbdc? irq 12 device vga0 at isa? device sc0 at isa? flags 0x100 # Floating point support - do not disable. device npx0 at nexus? port IO_NPX irq 13 # Serial (COM) ports device sio0 at isa? port IO_COM1 flags 0x10 irq 4 device sio1 at isa? port IO_COM2 irq 3 # ISA Ethernet NICs device miibus device ed0 at isa? port 0x280 irq 5 device ed1 at isa? port 0x300 irq 10 # Pseudo devices - the number indicates how many units to allocated. pseudo-device loop # Network loopback pseudo-device ether # Ethernet support pseudo-device tun # Packet tunnel. pseudo-device pty # Pseudo-ttys (telnet etc) pseudo-device bpf # Berkeley packet filter pseudo-device vn # Vnode driver ## ppp.conf default: set log Phase tun command set ifaddr 10.0.0.1/0 10.0.0.2/0 teaser: set device PPPoE:ed1 set authname xxxxxxx set authkey yyyyyy set dial set login add default HISADDR ## ppp.linkup # Refresh Ipfilter MYADDR: !bg /sbin/ipf -y ## ipf.rules # Malformed/suspect packets on all interfaces are blocked block in log quick all with opt lsrr block in log quick all with opt ssrr block in log quick all with ipopts block in log quick proto tcp all with short block in log quick proto icmp all with frag # No restrictions on local interface pass in quick on lo0 all pass out quick on lo0 all # External interface block in on tun0 all pass in quick on tun0 proto tcp from any to 193.56.60.242 port = 22 flags S keep state block out on tun0 all pass out quick on tun0 proto tcp from 192.168.1.0/24 to any flags S keep state keep frags pass out quick on tun0 proto udp from 192.168.1.0/24 to any keep state pass out quick on tun0 proto icmp from 192.168.1.0/24 to any keep state pass out quick on tun0 proto tcp from 193.56.60.242/32 to any flags S keep state keep frags pass out quick on tun0 proto udp from 193.56.60.242/32 to any keep state pass out quick on tun0 proto icmp from 193.56.60.242/32 to any keep state # External support interface pass in on ed1 all block in log quick on ed1 proto tcp all block in log quick on ed1 proto udp all block in log quick on ed1 proto icmp all pass out on ed1 all block out log quick on ed1 proto tcp all block out log quick on ed1 proto udp all block out log quick on ed1 proto icmp all # Internal interface pass in on ed0 all block in quick on ed0 proto tcp/udp from any to any port = 137 block in quick on ed0 proto tcp/udp from any to any port = 138 block in quick on ed0 proto tcp/udp from any to any port = 139 block in quick on ed0 proto tcp/udp from any port = 137 to any block in quick on ed0 proto tcp/udp from any port = 138 to any block in quick on ed0 proto tcp/udp from any port = 139 to any pass out on ed0 all ## uname -a FreeBSD rtrbsdnantsr.nantes.kisoft-services.com 4.4-RELEASE FreeBSD 4.4-RELEASE #0: Fri Sep 21 23:03:29 CEST 2001 emss@rtrbsdnantsr.nantes.kisoft-services.com:/usr/src/sys/compile/IBMPS1 i386 ## ifconfig -a ed0: flags=8843 mtu 1500 inet 192.168.1.15 netmask 0xffffff00 broadcast 192.168.1.255 ether 52:54:40:25:96:c8 ed1: flags=8843 mtu 1500 ether 52:54:40:25:a4:72 lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 tun0: flags=8051 mtu 1492 inet 193.56.60.242 --> 194.206.78.3 netmask 0xffffff00 Opened by PID 102 ## netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 194.206.78.3 UGSc 1 161638 tun0 127.0.0.1 127.0.0.1 UH 0 4 lo0 192.168.1 link#1 UC 2 0 ed0 192.168.1.1 0:60:8c:95:c5:e UHLW 0 48 ed0 948 192.168.1.21 0:80:c8:8d:14:b3 UHLW 1 117 ed0 1120 194.206.78.3 193.56.60.242 UH 1 0 tun0 ## netstat -i Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll ed0 1500 52:54:40:25:96:c8 164872 0 120153 0 13 ed0 1500 192.168.1 rtrbsdnantsr 271 - 908 - - ed1 1500 52:54:40:25:a4:72 125802 0 167462 0 5 lo0 16384 4 0 4 0 0 lo0 16384 127 localhost 4 - 4 - - tun0 1492 119257 0 160918 0 0 tun0 1492 193.56.60 193.56.60.242 4 - 2 - - ## netstat -s -p ip ip: 284481 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 476 packets for this host 2 packets for unknown/unsupported protocol 280464 packets forwarded (0 packets fast forwarded) 731 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 1075 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header ## ipf -V ipf: IP Filter: v3.4.20 (264) Kernel: IP Filter: v3.4.20 Running: yes Log Flags: 0 = none set Default: pass all, Logging: available Active list: 0 ## ipfstat input packets: blocked 2822 passed 281883 nomatch 0 counted 0 short 0 output packets: blocked 728 passed 281719 nomatch 0 counted 0 short 0 input packets logged: blocked 0 passed 0 output packets logged: blocked 0 passed 0 packets logged: input 0 output 0 log failures: input 0 output 0 fragment state(in): kept 0 lost 0 fragment state(out): kept 0 lost 0 packet state(in): kept 2 lost 0 packet state(out): kept 968 lost 0 ICMP replies: 0 TCP RSTs sent: 0 Invalid source(in): 0 Result cache hits(in): 2533 (out): 949 IN Pullups succeeded: 0 failed: 0 OUT Pullups succeeded: 0 failed: 0 Fastroute successes: 0 failures: 0 TCP cksum fails(in): 0 (out): 0 Packet log flags set: (0) none ## ipfstat -io pass out quick on lo0 from any to any block out on tun0 from any to any pass out quick on tun0 proto tcp from 192.168.1.0/24 to any flags S/FSRPAU keep state keep frags pass out quick on tun0 proto udp from 192.168.1.0/24 to any keep state pass out quick on tun0 proto icmp from 192.168.1.0/24 to any keep state pass out quick on tun0 proto tcp from 193.56.60.242/32 to any flags S/FSRPAU keep state keep frags pass out quick on tun0 proto udp from 193.56.60.242/32 to any keep state pass out quick on tun0 proto icmp from 193.56.60.242/32 to any keep state pass out on ed1 from any to any block out log quick on ed1 proto tcp from any to any block out log quick on ed1 proto udp from any to any block out log quick on ed1 proto icmp from any to any pass out on ed0 from any to any block in log quick from any to any with opt lsrr block in log quick from any to any with opt ssrr block in log quick from any to any with ipopt block in log quick proto tcp from any to any with short block in log quick proto icmp from any to any with frag pass in quick on lo0 from any to any block in on tun0 from any to any pass in quick on tun0 proto tcp from any to 193.56.60.242/32 port = 22 flags S/FSRPAU keep state pass in on ed1 from any to any block in log quick on ed1 proto tcp from any to any block in log quick on ed1 proto udp from any to any block in log quick on ed1 proto icmp from any to any pass in on ed0 from any to any block in quick on ed0 proto tcp/udp from any to any port = netbios-ns block in quick on ed0 proto tcp/udp from any to any port = netbios-dgm block in quick on ed0 proto tcp/udp from any to any port = netbios-ssn block in quick on ed0 proto tcp/udp from any port = netbios-ns to any block in quick on ed0 proto tcp/udp from any port = netbios-dgm to any block in quick on ed0 proto tcp/udp from any port = netbios-ssn to any ## ipnat -slv mapped in 0 out 0 added 0 expired 0 no memory 0 bad nat 0 inuse 0 rules 0 wilds 0 table 0xbfbffafc list 0x0 List of active MAP/Redirect filters: List of active sessions: List of active host mappings: Regards Eric Masson -- Warning: file "/home/emss/misc/fortune/En_sig.dat" unreadable Warning: file "/home/emss/misc/fortune/Fr_sig.dat" unreadable Faut vraiment que je m'occupe de ce problème de signature :) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 8:41:24 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from shikima.mine.nu (pc2-card3-0-cust61.cdf.cable.ntl.com [213.107.2.61]) by hub.freebsd.org (Postfix) with ESMTP id 9B2DB37B407 for ; Wed, 26 Sep 2001 08:41:20 -0700 (PDT) Received: from rasputin by shikima.mine.nu with local (Exim 3.33 #1) id 15mGoS-000N3M-00; Wed, 26 Sep 2001 16:41:28 +0100 Date: Wed, 26 Sep 2001 16:41:28 +0100 From: Rasputin To: Eric Masson Cc: hackers@freebsd.org Subject: Re: FreeBSD 4.4-RELEASE & ipf 3.4.20 freeze Message-ID: <20010926164128.A88585@shikima.mine.nu> Reply-To: Rasputin References: <86g09aqbs3.fsf@notbsdems.nantes.kisoft-services.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <86g09aqbs3.fsf@notbsdems.nantes.kisoft-services.com>; from e-masson@kisoft-services.com on Wed, Sep 26, 2001 at 04:14:52PM +0200 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Eric Masson [010926 15:25]: > Hello, > > Since I installed my DSL gateway with FreeBSD 4.4-RELEASE, the DSL link > freezes every 24 hours more or less, no more traffic possible at this > moment. Just a thought (if you're using DHCP) Are you seeing a lot of blocked packets in on your DHCPed interface - something like: :29.751341 2x ep0 @0:5 b 10.0.0.2,67 -> 255.255.255.255,68 PR udp len 20 328 IN 26/09/2001 16:36:30.501444 4x ep0 @0:5 b 10.0.0.2,67 -> 255.255.255.255,68 Could be your lease is expiring (especially if it's failing regularly), and your ISPs routers are blocking you? I only mention it because I noticed I have shedloads in my logs, although our ISP is thankfully pretty slack abouit expiring leases. Although /var is slowly filling up... Maybe try the freebsd-net mailing list otherwise. Cheers! -- Armadillo: To provide weapons to a Spanish pickle Rasputin :: Jack of All Trades - Master of Nuns :: To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 8:58:30 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from math.teaser.net (math.teaser.net [213.91.2.4]) by hub.freebsd.org (Postfix) with ESMTP id 129A937B416 for ; Wed, 26 Sep 2001 08:58:27 -0700 (PDT) Received: from notbsdems.nantes.kisoft-services.com (chantilly.kisoft-services.com [193.56.60.242]) by math.teaser.net (Postfix) with ESMTP id A7C916C819; Wed, 26 Sep 2001 17:58:25 +0200 (CEST) Received: by notbsdems.nantes.kisoft-services.com (Postfix, from userid 1001) id 59561E6B42; Wed, 26 Sep 2001 17:52:14 +0200 (CEST) To: Rasputin Cc: hackers@freebsd.org Subject: Re: FreeBSD 4.4-RELEASE & ipf 3.4.20 freeze References: <86g09aqbs3.fsf@notbsdems.nantes.kisoft-services.com> <20010926164128.A88585@shikima.mine.nu> From: Eric Masson In-Reply-To: <20010926164128.A88585@shikima.mine.nu> (Rasputin's message of "Wed, 26 Sep 2001 16:41:28 +0100") X-Operating-System: FreeBSD 4.4-RC i386 Date: Wed, 26 Sep 2001 17:52:13 +0200 Message-ID: <864rpqq79u.fsf@notbsdems.nantes.kisoft-services.com> Lines: 22 User-Agent: Gnus/5.090004 (Oort Gnus v0.04) XEmacs/21.1 (Cuyahoga Valley) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "rasputin" == rasputin writes: rasputin> Just a thought (if you're using DHCP) Are you seeing a lot of rasputin> blocked packets in on your DHCPed interface - something like: Sorry, no DHCP here, fixed ip address. rasputin> I only mention it because I noticed I have shedloads in my rasputin> logs, although our ISP is thankfully pretty slack abouit rasputin> expiring leases. Although /var is slowly filling up... Just think at the moment that I don't log blocked incoming packets on tun0, as the box is only is test at the moment. Thanks for your answer. Eric Masson -- DV: Depuis que j'ai enregistré MacSoup, il arrête pas de geler. GC: Attends le printemps. -+- GV in Guide du Macounet Pervers : faites chauffer MacSoup ! -+- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 9:16:28 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from owa-sj-1.digisle.com (owa-sj-1.digisle.com [167.216.153.124]) by hub.freebsd.org (Postfix) with ESMTP id B8CB037B422 for ; Wed, 26 Sep 2001 09:16:23 -0700 (PDT) Received: from VWALL-SJ-1.digisle.com ([167.216.153.118]) by owa-sj-1.digisle.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 26 Sep 2001 09:17:38 -0700 Received: from 206.220.227.145 by VWALL-SJ-1.digisle.com (InterScan E-Mail VirusWall NT); Wed, 26 Sep 2001 09:16:24 -0700 Message-ID: <3BB1FF55.1460E1E4@digisle.net> Date: Wed, 26 Sep 2001 09:16:21 -0700 From: Maksim Yevmenkin Organization: Digital Island X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.7 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: hackers@freebsd.org Subject: Netgraph feature request/suggestion Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 26 Sep 2001 16:17:38.0400 (UTC) FILETIME=[C2744600:01C146A6] Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hackers, i'm in the middle of the project that uses Netgraph. everything is going pretty good, but there is one small issue. in five words it is "message and data delivery scheduling". here is an example: Node A --> Node B --> Node C Node "A" forwards data/messages to Node "B" and Node "B" in its turn forwards data/messages to Node "C". the issue is that Node "C" can handle only some small amount of data/messages at a time. Node "B" is aware of Node "C"'s limitation and must perform "leaking bucket" type of scheduling. i.e. Node "B" must queue data inside itself and then schedule later delivery to Node "C". but that is not all. sometimes it is required to send chunk of data (several messages) from Node "A" to Node "C". (via Node "B") and until this chunk of data is processed by Node "C", Node "B" is not allowed to send/accept any more data. i know about kernel threads and task queue, but i would really like to stay within Netgraph infrastructure and do not perform any extra synchronization. here is the proposal. every hook has two extra methods "hk_RcvDataShed" and "hk_RcvmMgSched" that performs scheduling. Node can turn on delivery scheduling on one of its hook by setting these methods. before actual data/message delivery Netgraph will call these methods and ask destination node "is that a good time to deliver this data/message". if it is then delivery is performed. otherwise depending on "HK_QUEUE" bit data/message gets queued or dropped. or perhaps turning on delivery scheduling must turn on "HK_QUEUE" bit automatically. thanks, max p.s. i think it should be easy to implement, if people are interested. i can prepare the patches. p.p.s. if anyone knows nice solution to this problem please let me know. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 9:17: 8 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from math.teaser.net (math.teaser.net [213.91.2.4]) by hub.freebsd.org (Postfix) with ESMTP id 6BA3A37B41D for ; Wed, 26 Sep 2001 09:17:03 -0700 (PDT) Received: from notbsdems.nantes.kisoft-services.com (chantilly.kisoft-services.com [193.56.60.242]) by math.teaser.net (Postfix) with ESMTP id 84A9E6C815; Wed, 26 Sep 2001 18:17:02 +0200 (CEST) Received: by notbsdems.nantes.kisoft-services.com (Postfix, from userid 1001) id A95BBE6B42; Wed, 26 Sep 2001 18:16:28 +0200 (CEST) To: Mailing List IPFilter Cc: Mailing List FreeBSD Hackers Subject: Re: FreeBSD 4.4-RELEASE & ipf 3.4.20 freeze References: <86g09aqbs3.fsf@notbsdems.nantes.kisoft-services.com> From: Eric Masson In-Reply-To: <86g09aqbs3.fsf@notbsdems.nantes.kisoft-services.com> (Eric Masson's message of "Wed, 26 Sep 2001 16:14:52 +0200") X-Operating-System: FreeBSD 4.4-RC i386 Date: Wed, 26 Sep 2001 18:16:28 +0200 Message-ID: <86n13iorkz.fsf@notbsdems.nantes.kisoft-services.com> Lines: 16 User-Agent: Gnus/5.090004 (Oort Gnus v0.04) XEmacs/21.1 (Cuyahoga Valley) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "Emss" == Eric Masson writes: Emss> netstat -i takes a long time (1 minute) to give a result after Emss> the line with tun0. Sorry for the followup to myself. Doh, I should sleep sometimes :(, delay for netstat -i comes from the fact that it tries to resolve the external address but can't as the link doesn't work anymore. Eric Masson -- Progrès (nm) : Ce qui a conduit d'un utilisateur intelligent devant un terminal passif à un utilisateur passif devant un terminal intelligent. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 9:21:35 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from raven.ravenbrook.com (raven.ravenbrook.com [193.82.131.18]) by hub.freebsd.org (Postfix) with ESMTP id 5684037B412 for ; Wed, 26 Sep 2001 09:21:31 -0700 (PDT) Received: from thrush.ravenbrook.com (thrush.ravenbrook.com [193.112.141.249]) by raven.ravenbrook.com (8.11.3/8.11.3) with ESMTP id f8QGLMa53462 for ; Wed, 26 Sep 2001 17:21:22 +0100 (BST) (envelope-from nb@ravenbrook.com) Received: from thrush.ravenbrook.com (localhost [127.0.0.1]) by thrush.ravenbrook.com (8.11.4/8.11.2) with ESMTP id f8QGKVx52874 for ; Wed, 26 Sep 2001 17:20:31 +0100 (BST) (envelope-from nb@thrush.ravenbrook.com) From: Nick Barnes To: freebsd-hackers@freebsd.org Subject: distinguising read faults and write faults Date: Wed, 26 Sep 2001 17:20:31 +0100 Message-ID: <52872.1001521231@thrush.ravenbrook.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I am in the process of porting an incremental garbage collector to FreeBSD on x86. This garbage collector uses read barriers and write barriers for incrementality. The details are irrelevant; the key part is that I want to protect parts of memory and handle faults on the protected pages. On FreeBSD, I will be using mmap(MAP_ANON) to obtain memory, and mprotect() to raise the barriers, with a signal handler to handle barrier hits. I have worked out that I need to handle SIGBUS (not SIGSEGV, which surprised me), and that ucontext->mcontext.mc_trapno will be 12 (T_PAGEFLT, not T_PROTFLT, which surprised me). However, I don't see any way of distinguishing between read faults and write faults. This information doesn't appear to be anywhere in the ucontext or the siginfo. Is there any way of distingushing read faults from write faults (other than by decoding the instruction at mc_eip)? Matching code for Linux below: Nick Barnes /* distinguishing read faults from write faults on Linux */ static void sigHandle(int sig, struct sigcontext context) { if(context.trapno == 14) { /* page fault */ int write = ((context.err & 0x2) != 0); /* write */ ... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 9:25:14 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from iguana.aciri.org (iguana.aciri.org [192.150.187.36]) by hub.freebsd.org (Postfix) with ESMTP id 0A1E437B408 for ; Wed, 26 Sep 2001 09:25:07 -0700 (PDT) Received: (from rizzo@localhost) by iguana.aciri.org (8.11.3/8.11.1) id f8QGMAl65454; Wed, 26 Sep 2001 09:22:10 -0700 (PDT) (envelope-from rizzo) From: Luigi Rizzo Message-Id: <200109261622.f8QGMAl65454@iguana.aciri.org> Subject: Re: Netgraph feature request/suggestion In-Reply-To: <3BB1FF55.1460E1E4@digisle.net> from Maksim Yevmenkin at "Sep 26, 2001 9:16:21 am" To: myevmenk@digisle.net (Maksim Yevmenkin) Date: Wed, 26 Sep 2001 09:22:10 -0700 (PDT) Cc: hackers@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL43 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, one problem you have to keep in mind with what you want to do (and i am not talking about the implementation that you suggest, just the "delayed processing" aspects) is that sometimes data passed through modules have a limited lifetime and might have become stale by the time the consumer is scheduled to process them. I had a similar problem with dummynet, related to route entries that are passed around and might disappear. Same applies to interface pointers (which, fortunately, are less dynamic). cheers luigi > i'm in the middle of the project that uses Netgraph. > everything is going pretty good, but there is one small > issue. in five words it is "message and data delivery > scheduling". here is an example: > > Node A --> Node B --> Node C > > Node "A" forwards data/messages to Node "B" and Node "B" > in its turn forwards data/messages to Node "C". the issue > is that Node "C" can handle only some small amount of > data/messages at a time. Node "B" is aware of Node "C"'s > limitation and must perform "leaking bucket" type of > scheduling. i.e. Node "B" must queue data inside itself > and then schedule later delivery to Node "C". > > but that is not all. sometimes it is required to send > chunk of data (several messages) from Node "A" to Node "C". > (via Node "B") and until this chunk of data is processed by > Node "C", Node "B" is not allowed to send/accept any more > data. > > i know about kernel threads and task queue, but i would really > like to stay within Netgraph infrastructure and do not perform > any extra synchronization. > > here is the proposal. every hook has two extra methods > "hk_RcvDataShed" and "hk_RcvmMgSched" that performs scheduling. > Node can turn on delivery scheduling on one of its hook by > setting these methods. before actual data/message delivery > Netgraph will call these methods and ask destination node "is > that a good time to deliver this data/message". if it is then > delivery is performed. otherwise depending on "HK_QUEUE" bit > data/message gets queued or dropped. or perhaps turning on > delivery scheduling must turn on "HK_QUEUE" bit automatically. > > thanks, > max > > p.s. i think it should be easy to implement, if people are > interested. i can prepare the patches. > > p.p.s. if anyone knows nice solution to this problem please > let me know. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 9:33:19 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from wall.polstra.com (wall-gw.polstra.com [206.213.73.130]) by hub.freebsd.org (Postfix) with ESMTP id 97E8E37B42C for ; Wed, 26 Sep 2001 09:33:14 -0700 (PDT) Received: from vashon.polstra.com (vashon.polstra.com [206.213.73.13]) by wall.polstra.com (8.11.3/8.11.3) with ESMTP id f8QGX7812327; Wed, 26 Sep 2001 09:33:07 -0700 (PDT) (envelope-from jdp@wall.polstra.com) Received: (from jdp@localhost) by vashon.polstra.com (8.11.6/8.11.0) id f8QGX7B02240; Wed, 26 Sep 2001 09:33:07 -0700 (PDT) (envelope-from jdp) Date: Wed, 26 Sep 2001 09:33:07 -0700 (PDT) Message-Id: <200109261633.f8QGX7B02240@vashon.polstra.com> To: hackers@freebsd.org From: John Polstra Cc: Nick.Barnes@pobox.com Subject: Re: distinguising read faults and write faults In-Reply-To: <52872.1001521231@thrush.ravenbrook.com> References: <52872.1001521231@thrush.ravenbrook.com> Organization: Polstra & Co., Seattle, WA Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In article <52872.1001521231@thrush.ravenbrook.com>, Nick Barnes wrote: > I am in the process of porting an incremental garbage collector to > FreeBSD on x86. This garbage collector uses read barriers and write > barriers for incrementality. The details are irrelevant; the key part > is that I want to protect parts of memory and handle faults on the > protected pages. > > On FreeBSD, I will be using mmap(MAP_ANON) to obtain memory, and > mprotect() to raise the barriers, with a signal handler to handle > barrier hits. The garbage collector in Modula-3 is constructed this way exactly. It works pretty well. > I have worked out that I need to handle SIGBUS (not SIGSEGV, which > surprised me), and that ucontext->mcontext.mc_trapno will be 12 > (T_PAGEFLT, not T_PROTFLT, which surprised me). However, I don't see > any way of distinguishing between read faults and write faults. This > information doesn't appear to be anywhere in the ucontext or the > siginfo. > > Is there any way of distingushing read faults from write faults (other > than by decoding the instruction at mc_eip)? I don't know of a way. I notice that the collector in Modula-3 protects some regions as read-only and other regions as inaccessible (unreadable and unwritable). There is one thing you need to be aware of. You will need to wrap every system call which takes a pointer as an argument. If you pass a pointer to inaccessible memory to a system call, you don't get a signal. Instead, the call returns EFAULT. So you have to wrap every such system call and make sure its arguments have been made accessible before issuing the call. There is an example of this in the Modula-3 code (look for RTHeapDepC.c). It's a pain, because any little change in the prototype of a system call (e.g., addition of "const") breaks compilation of the corresponding wrapper. As far as I can tell, the whole EFAULT mess is a relic from the days before system calls became restartable in Unix. Today it would be possible to add a new signal SIGFAULT which could be caught to handle this case. Then the wrappers wouldn't be needed. It could still be backward compatible; an application which didn't install a SIGFAULT handler would get EFAULT returns in the traditional way. "One Of These Days" I'm going to add this to FreeBSD. John -- John Polstra jdp@polstra.com John D. Polstra & Co., Inc. Seattle, Washington USA "Disappointment is a good sign of basic intelligence." -- Chögyam Trungpa To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 9:44: 0 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from owa-sj-1.digisle.com (owa-sj-1.digisle.com [167.216.153.124]) by hub.freebsd.org (Postfix) with ESMTP id 0BFAB37B41C for ; Wed, 26 Sep 2001 09:43:57 -0700 (PDT) Received: from VWALL-SJ-1.digisle.com ([167.216.153.118]) by owa-sj-1.digisle.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 26 Sep 2001 09:45:12 -0700 Received: from 206.220.227.145 by VWALL-SJ-1.digisle.com (InterScan E-Mail VirusWall NT); Wed, 26 Sep 2001 09:43:58 -0700 Message-ID: <3BB205CB.257DEF76@digisle.net> Date: Wed, 26 Sep 2001 09:43:55 -0700 From: Maksim Yevmenkin Organization: Digital Island X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.7 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Luigi Rizzo Cc: hackers@FreeBSD.ORG Subject: Re: Netgraph feature request/suggestion References: <200109261622.f8QGMAl65454@iguana.aciri.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 26 Sep 2001 16:45:12.0187 (UTC) FILETIME=[9C3024B0:01C146AA] Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Luigi, [...] > one problem you have to keep in mind with what you want to do > (and i am not talking about the implementation that you suggest, > just the "delayed processing" aspects) is that sometimes data passed > through modules have a limited lifetime and might have become stale > by the time the consumer is scheduled to process them. that is why this feature must be *optional* and turned off by default. the node must decide whenever it wants to do delivery scheduling and on which hook. the node also must be prepared to deal with various nasty side effects. in my case it is not a big problem, but in general case there is always Netgraph "meta data". [....] > > i'm in the middle of the project that uses Netgraph. > > everything is going pretty good, but there is one small > > issue. in five words it is "message and data delivery > > scheduling". here is an example: > > > > Node A --> Node B --> Node C > > > > Node "A" forwards data/messages to Node "B" and Node "B" > > in its turn forwards data/messages to Node "C". the issue > > is that Node "C" can handle only some small amount of > > data/messages at a time. Node "B" is aware of Node "C"'s > > limitation and must perform "leaking bucket" type of > > scheduling. i.e. Node "B" must queue data inside itself > > and then schedule later delivery to Node "C". > > > > but that is not all. sometimes it is required to send > > chunk of data (several messages) from Node "A" to Node "C". > > (via Node "B") and until this chunk of data is processed by > > Node "C", Node "B" is not allowed to send/accept any more > > data. > > > > i know about kernel threads and task queue, but i would really > > like to stay within Netgraph infrastructure and do not perform > > any extra synchronization. > > > > here is the proposal. every hook has two extra methods > > "hk_RcvDataShed" and "hk_RcvmMgSched" that performs scheduling. > > Node can turn on delivery scheduling on one of its hook by > > setting these methods. before actual data/message delivery > > Netgraph will call these methods and ask destination node "is > > that a good time to deliver this data/message". if it is then > > delivery is performed. otherwise depending on "HK_QUEUE" bit > > data/message gets queued or dropped. or perhaps turning on > > delivery scheduling must turn on "HK_QUEUE" bit automatically. thanks, max To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 10: 2:22 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 3839B37B433 for ; Wed, 26 Sep 2001 10:02:19 -0700 (PDT) Received: from mindspring.com (dialup-209.247.137.184.Dial1.SanJose1.Level3.net [209.247.137.184]) by falcon.mail.pas.earthlink.net (8.11.5/8.9.3) with ESMTP id f8QH2GA16385; Wed, 26 Sep 2001 10:02:16 -0700 (PDT) Message-ID: <3BB20A46.474E3C48@mindspring.com> Date: Wed, 26 Sep 2001 10:03:02 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Brian Reichert Cc: freebsd-hackers@freebsd.org Subject: Re: got bad cookie vp 0xe2e5ef80 bp 0xcf317328 References: <20010925093643.W49528@numachi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Brian Reichert wrote: > /* > * Yuck! The directory has been modified on the > * server. The only way to get the block is by > * reading from the beginning to get all the > * offset cookies. > * > * Leave the last bp intact unless there is an error. > * Loop back up to the while if the error is another > * NFSERR_BAD_COOKIE (double yuch!). > */ This is technically incorrect. Unless the directory has been truncated back, the way to do this is to do a mod on the block size for the offset given, and assume a compression of the directory block out from under you. So you do the mod, and then that gives you the directory block (compression does not occur across blocks), and then chain forward in the block until you have an index that exceeds the cookie index, and then go back one. The best way to do that is to remember the index one previous. In the case of a truncation, you're done. The ultimate "bad" result from this will be a duplicate entry from the iteration of the directory. Dealing with a duplicate entry is the job of the NFS client. Note that a directory iteration is just a "snapshot", much like "ps", and is not guaranteed to remain accurate, just as taking a polaroid picture of a crowd will not necessarily remain an accurate indicator of who is there, since it could change in the time between when the picture is taken and it develops. The best way would be to avoid cookies entirely, but FreeBSD is not structured for this (cookies were introduced in the 4.4-Lite release by NetBSD, and adopted by everyone else, including the 4.4-Lite2 release; NetBSD and OpenBSD have a different cookie parameter order than FreeBSD, as well). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 10: 4: 9 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from obsecurity.dyndns.org (adsl-63-207-60-153.dsl.lsan03.pacbell.net [63.207.60.153]) by hub.freebsd.org (Postfix) with ESMTP id 8FD2F37B416 for ; Wed, 26 Sep 2001 10:04:04 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 1CCE166E4F; Wed, 26 Sep 2001 10:04:04 -0700 (PDT) Date: Wed, 26 Sep 2001 10:04:03 -0700 From: Kris Kennaway To: Peter Wemm Cc: Julian Elischer , Paul Saab , hackers@FreeBSD.ORG Subject: Re: librsa and 4.4 Message-ID: <20010926100403.A86509@xor.obsecurity.org> References: <20010925215451.D21BA3809@overcee.netplex.com.au> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="tKW2IUtsqtDRztdT" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010925215451.D21BA3809@overcee.netplex.com.au>; from peter@wemm.org on Tue, Sep 25, 2001 at 02:54:51PM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --tKW2IUtsqtDRztdT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 25, 2001 at 02:54:51PM -0700, Peter Wemm wrote: > Julian Elischer wrote: > >=20 > > In a related problem: > >=20 > > we have a set of 4.1.1 binaries we want ot run on 4.4 > > but they (apache+other stuff) want to find a librsaUSA.so > > but can't.. I fixed it by copying the one from 4.1.1 into /usr/lib/comp= at. > > Is that the right answer? > > Is it possible we can have a compat librsa? > > (maybe even empty if the stuff is now in libcrypt or something). >=20 > libcrypto.so.1.gz.uu and libssl.so.1.gz.uu need to be MFC'ed. This should > have been done before 4.4-REL as well. Um, they were MFCed a long time ago. Julian probably didn't have compat4x enabled. Kris --tKW2IUtsqtDRztdT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7sgqDWry0BWjoQKURAm6RAJ0eOhxPQhcg5nvwfeOSDrTLMiAHUgCdFgJb GPNOf6Nfoir3aZIeZptjoR8= =An9I -----END PGP SIGNATURE----- --tKW2IUtsqtDRztdT-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 10: 4:51 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from tomts9-srv.bellnexxia.net (tomts9.bellnexxia.net [209.226.175.53]) by hub.freebsd.org (Postfix) with ESMTP id 2A11437B410; Wed, 26 Sep 2001 10:04:23 -0700 (PDT) Received: from khan.anarcat.dyndns.org ([65.92.169.79]) by tomts9-srv.bellnexxia.net (InterMail vM.4.01.03.16 201-229-121-116-20010115) with ESMTP id <20010926170412.SGDK25005.tomts9-srv.bellnexxia.net@khan.anarcat.dyndns.org>; Wed, 26 Sep 2001 13:04:12 -0400 Received: from shall.anarcat.dyndns.org (shall.anarcat.dyndns.org [192.168.0.1]) by khan.anarcat.dyndns.org (Postfix) with ESMTP id EE9371AAD; Wed, 26 Sep 2001 13:03:53 -0400 (EDT) Received: by shall.anarcat.dyndns.org (Postfix, from userid 1000) id 02C4E20B36; Wed, 26 Sep 2001 13:03:50 -0400 (EDT) Date: Wed, 26 Sep 2001 13:03:50 -0400 From: The Anarcat To: freebsd-doc@freebsd.org Cc: freebsd-hackers@freebsd.org Subject: libh?disk doc (deficiencies?) Message-ID: <20010926130350.C40172@shall.anarcat.dyndns.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="n2Pv11Ogg/Ox8ay5" Content-Disposition: inline User-Agent: Mutt/1.3.22.1i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --n2Pv11Ogg/Ox8ay5 Content-Type: multipart/mixed; boundary="oTHb8nViIGeoXxdp" Content-Disposition: inline --oTHb8nViIGeoXxdp Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [crossposted to -hackers in a hope to have more information, sorry if this is inappropriate] Hi. I'm currently working a bit on libh, which includes a libhdisk library to interface libdisk(3), newfs, etc, in short, a interface to operate disk partition/slice edition =E0 la sysinstall. The thing is that I'm trying to figure out how all this stuff works. libdisk(3) is not very informative, in fact, it doesn't document anything below the "slice" level (FS_SWAP macro and structure of a Disk/Chunk tree, for example). I made a little graphic (ASCII!) to show how such a tree (Disk/Chunk) is made, along with an algorithm from sysinstall's label.c that extract relevant data out of it. I will make documentation for the Disk/Chunk class hierarchy and functions, and that will imply investigating libdisk's functions.=20 I think libdisk(3) should be split and rewrote. Anyways, isn't it policy to have *functions* instead of *libraries* documented in manpages? :) The thing is that I might not be the best person to do it, and I do not understand everything clearly yet. If anyone has any other source of information (apart from source code), please share. :) I will probably end up posting prs about libdisk's doc... One thing I haven't figured out yet is why there is an intermediate "Chunk" between Disk and slices (see picture, the first chunk on the right of the Disk box). I think the graphic is quite good, and should be included somewhere, for reference because it took me a while to figure it out. A. --oTHb8nViIGeoXxdp Content-Type: text/plain; charset=us-ascii Content-Description: libh/release/labeledit/algo.txt Content-Disposition: attachment; filename="algo.txt" Content-Transfer-Encoding: quoted-printable We have the structures: +--------+ +-------+ | Disk | | Chunk |=20 | | | | This chunk represents the disk in itself | chunks--->| part |=20 +--------+ +--|----+=20 | =20 +--------------|---------------------------+ |Slices (e.g. | freebsd, linux, dos) | | V | | +-----------+ +-----------+ | | | Chunk | | Chunk | | | | | | | | | | type=3Dfbsd | | type=3Dfat | | | | | | | | | | part next----->| part next----> ... | | | | | | | | | | +--|--------+ +---|-------+ | | | ? | +------|-----------------------------------+ | +------|-----------------------------------+ |Partitions (eg. fs, swap) | | V | | +--------------+ +--------+ | | | Chunk | | Chunk | | | | | | | | | | type=3Dpart | | ... | | | | | | | | | | next----->| next----> ... | | | | | | | | | subtype=3Dswap | | ... | | | | | | | | | +--------------+ +--------+ | | | +------------------------------------------+ /* All the chunks currently displayed on the screen */ static struct { struct chunk *c; PartType type; } label_chunk_info[MAX_CHUNKS + 1]; /* this function flattens such a tree in a single array */ static void record_label_chunks(Device **devs, Device *dev) { int i, j, p; struct chunk *c1, *c2; Disk *d; j =3D p =3D 0; /* First buzz through and pick up the FreeBSD slices */ for (i =3D 0; devs[i]; i++) { if ((dev && devs[i] !=3D dev) || !devs[i]->enabled) continue; d =3D (Disk *)devs[i]->private; if (!d->chunks) msgFatal("No chunk list found for %s!", d->name); /* Put the slice entries first */ for (c1 =3D d->chunks->part; c1; c1 =3D c1->next) { if (c1->type =3D=3D freebsd) { label_chunk_info[j].type =3D PART_SLICE; label_chunk_info[j].c =3D c1; ++j; } } } /* Now run through again and get the FreeBSD partition entries */ for (i =3D 0; devs[i]; i++) { if (!devs[i]->enabled) continue; d =3D (Disk *)devs[i]->private; /* Then buzz through and pick up the partitions */ for (c1 =3D d->chunks->part; c1; c1 =3D c1->next) { if (c1->type =3D=3D freebsd) { for (c2 =3D c1->part; c2; c2 =3D c2->next) { if (c2->type =3D=3D part) { if (c2->subtype =3D=3D FS_SWAP) label_chunk_info[j].type =3D PART_SWAP; else label_chunk_info[j].type =3D PART_FILESYSTEM; label_chunk_info[j].c =3D c2; ++j; } } } else if (c1->type =3D=3D fat) { label_chunk_info[j].type =3D PART_FAT; label_chunk_info[j].c =3D c1; ++j; } } } label_chunk_info[j].c =3D NULL; if (here >=3D j) { here =3D j ? j - 1 : 0; } } --oTHb8nViIGeoXxdp-- --n2Pv11Ogg/Ox8ay5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjuyCnUACgkQttcWHAnWiGcIdwCeMGZ0JINGAJ9EAvP6reQdAuFb eCIAni5QtKHblw51OKnEq8lQshr3+Uy7 =7B4T -----END PGP SIGNATURE----- --n2Pv11Ogg/Ox8ay5-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 10:26:37 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id D40D937B429 for ; Wed, 26 Sep 2001 10:26:31 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA60849; Wed, 26 Sep 2001 11:12:18 -0700 (PDT) Date: Wed, 26 Sep 2001 11:12:17 -0700 (PDT) From: Julian Elischer To: Kris Kennaway Cc: Peter Wemm , Paul Saab , hackers@FreeBSD.ORG Subject: Re: librsa and 4.4 In-Reply-To: <20010926100403.A86509@xor.obsecurity.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 26 Sep 2001, Kris Kennaway wrote: > On Tue, Sep 25, 2001 at 02:54:51PM -0700, Peter Wemm wrote: > > Julian Elischer wrote: > > > > > > In a related problem: > > > > > > we have a set of 4.1.1 binaries we want ot run on 4.4 > > > but they (apache+other stuff) want to find a librsaUSA.so > > > but can't.. I fixed it by copying the one from 4.1.1 into /usr/lib/compat. > > > Is that the right answer? > > > Is it possible we can have a compat librsa? > > > (maybe even empty if the stuff is now in libcrypt or something). > > > > libcrypto.so.1.gz.uu and libssl.so.1.gz.uu need to be MFC'ed. This should > > have been done before 4.4-REL as well. > > Um, they were MFCed a long time ago. Julian probably didn't have > compat4x enabled. We installed directly from released images.. there was no option to install "compat4X" in the 4.4 installer. In the old systems we would have had to install librsa in some form or other. now these functions are 'standard' in some way but some programs still want to "see" librsa on the system, so we should have a way of saying "yep.. it's there" (e.g. apache-ssl) > > Kris > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 10:46: 0 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 4AF4B37B428 for ; Wed, 26 Sep 2001 10:45:52 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA60895; Wed, 26 Sep 2001 11:23:17 -0700 (PDT) Date: Wed, 26 Sep 2001 11:23:15 -0700 (PDT) From: Julian Elischer To: Maksim Yevmenkin Cc: hackers@freebsd.org Subject: Re: Netgraph feature request/suggestion In-Reply-To: <3BB1FF55.1460E1E4@digisle.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi! I'd love to hear more about what you are doing.. but to answer your questions.. Note: All this changes in -current where netraph was largely rewritten. On Wed, 26 Sep 2001, Maksim Yevmenkin wrote: > Hackers, > > i'm in the middle of the project that uses Netgraph. > everything is going pretty good, but there is one small > issue. in five words it is "message and data delivery > scheduling". here is an example: > > Node A --> Node B --> Node C > > Node "A" forwards data/messages to Node "B" and Node "B" > in its turn forwards data/messages to Node "C". the issue > is that Node "C" can handle only some small amount of > data/messages at a time. Node "B" is aware of Node "C"'s > limitation and must perform "leaking bucket" type of > scheduling. i.e. Node "B" must queue data inside itself > and then schedule later delivery to Node "C". ok.. > > but that is not all. sometimes it is required to send > chunk of data (several messages) from Node "A" to Node "C". > (via Node "B") and until this chunk of data is processed by > Node "C", Node "B" is not allowed to send/accept any more > data. yes, but I'l not sure what the problem is.... (Do you want C to notify B that it has room?) > > i know about kernel threads and task queue, but i would really > like to stay within Netgraph infrastructure and do not perform > any extra synchronization. > > here is the proposal. every hook has two extra methods > "hk_RcvDataShed" and "hk_RcvmMgSched" that performs scheduling. Hooks have no methods at this time, but, yes we can add them should you be very convincing :-) > Node can turn on delivery scheduling on one of its hook by > setting these methods. before actual data/message delivery > Netgraph will call these methods and ask destination node "is > that a good time to deliver this data/message". if it is then > delivery is performed. otherwise depending on "HK_QUEUE" bit > data/message gets queued or dropped. or perhaps turning on > delivery scheduling must turn on "HK_QUEUE" bit automatically. The node can presently turn on the HK_QUEUE bit itself, but that will only delay the data for a short time.. It seesm to be that "C" needs to do it's own internal queueing. Is "C" an device driver? how does it get to run asynchronously? > > thanks, > max > > p.s. i think it should be easy to implement, if people are > interested. i can prepare the patches. > > p.p.s. if anyone knows nice solution to this problem please > let me know. Well in -current you could do it by locking the node C to serialise all accesses to it. You could also use the "flow control messages" defined in -current netgraph, to disable or re-enable data flow from "A". can you give me more information to help me understand the problem a bit better. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 10:59:30 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from web.cs.ndsu.nodak.edu (web.cs.ndsu.NoDak.edu [134.129.125.7]) by hub.freebsd.org (Postfix) with ESMTP id 57BE437B418 for ; Wed, 26 Sep 2001 10:59:26 -0700 (PDT) Received: (from tinguely@localhost) by web.cs.ndsu.nodak.edu (8.11.4/8.11.4) id f8QHx1q97994; Wed, 26 Sep 2001 12:59:01 -0500 (CDT) (envelope-from tinguely) Date: Wed, 26 Sep 2001 12:59:01 -0500 (CDT) From: mark tinguely Message-Id: <200109261759.f8QHx1q97994@web.cs.ndsu.nodak.edu> To: fjoe@iclub.nsu.ru, freebsd-hackers@FreeBSD.ORG Subject: Re: arcnet support for FreeBSD (request for review) In-Reply-To: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG There is nothing like raising a topic that was last seen several months ago, but ... Has there been any serious consideration to committing the arcnet code that mentioned on 20 Jul 2001 (http://iclub.nsu.ru/~fjoe/arcnet/)? I guess I should ask if anyone else has tried the code. I will most likely have another driver for arcnet in a couple months if the customer goes ahead with the project. --mark tinguely. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 11:32:20 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from owa-sj-1.digisle.com (owa-sj-1.digisle.com [167.216.153.124]) by hub.freebsd.org (Postfix) with ESMTP id B04C937B42B for ; Wed, 26 Sep 2001 11:32:12 -0700 (PDT) Received: from VWALL-SJ-1.digisle.com ([167.216.153.118]) by owa-sj-1.digisle.com with Microsoft SMTPSVC(5.0.2195.2966); Wed, 26 Sep 2001 11:33:27 -0700 Received: from 206.220.227.145 by VWALL-SJ-1.digisle.com (InterScan E-Mail VirusWall NT); Wed, 26 Sep 2001 11:32:14 -0700 Message-ID: <3BB21F2A.F4ABE2D9@digisle.net> Date: Wed, 26 Sep 2001 11:32:10 -0700 From: Maksim Yevmenkin Organization: Digital Island X-Mailer: Mozilla 4.78 [en] (X11; U; SunOS 5.7 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: hackers@freebsd.org Subject: Re: Netgraph feature request/suggestion References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 26 Sep 2001 18:33:27.0833 (UTC) FILETIME=[BBE51090:01C146B9] Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian, [...] > All this changes in -current where netraph was largely rewritten. i know. i'm using current > > i'm in the middle of the project that uses Netgraph. > > everything is going pretty good, but there is one small > > issue. in five words it is "message and data delivery > > scheduling". here is an example: > > > > Node A --> Node B --> Node C > > > > Node "A" forwards data/messages to Node "B" and Node "B" > > in its turn forwards data/messages to Node "C". the issue > > is that Node "C" can handle only some small amount of > > data/messages at a time. Node "B" is aware of Node "C"'s > > limitation and must perform "leaking bucket" type of > > scheduling. i.e. Node "B" must queue data inside itself > > and then schedule later delivery to Node "C". > > ok.. > > > but that is not all. sometimes it is required to send > > chunk of data (several messages) from Node "A" to Node "C". > > (via Node "B") and until this chunk of data is processed by > > Node "C", Node "B" is not allowed to send/accept any more > > data. > > yes, but I'l not sure what the problem is.... > (Do you want C to notify B that it has room?) yes. Node "C" sends periodic notification to Node "B". the easy solution here is to send one data/message at a time and wait until "C" sends notification back or timeout occur, then "B" will schedule another delivery (if any). but it kills the performance. and, like i said before, sometimes i just need to send chunk of data and wait for notification from "C" or for timeout. and in the same time i do not want keep "A" waiting. [....] > > here is the proposal. every hook has two extra methods > > "hk_RcvDataShed" and "hk_RcvmMgSched" that performs scheduling. > > Hooks have no methods at this time, but, yes we can add them should > you be very convincing :-) perhaps i should have used another word. -current Netgraph "ng_hook" structure has "hk_rcvmsg" and "hk_rcvdata" function pointers and node can set different message/data receive function per hook. suggestion was to add two more function pointers. i called them methods. > > Node can turn on delivery scheduling on one of its hook by > > setting these methods. before actual data/message delivery > > Netgraph will call these methods and ask destination node "is > > that a good time to deliver this data/message". if it is then > > delivery is performed. otherwise depending on "HK_QUEUE" bit > > data/message gets queued or dropped. or perhaps turning on > > delivery scheduling must turn on "HK_QUEUE" bit automatically. > > The node can presently turn on the HK_QUEUE bit itself, but that will only > delay the data for a short time.. > > It seesm to be that "C" needs to do it's own internal queueing. there can be multiply instances of "C". i do not want to duplicate this code in every "C" instance. > Is "C" an device driver? how does it get to run asynchronously? it could be a device driver, or it could be another node. it does not really matter (i think). and, yes, it does run asynchronously. that is why i do not want to mess around threads and task queues. extra overhead of locking etc. [...] > Well in -current you could do it by locking the node C to serialise all > accesses to it. > You could also use the "flow control messages" defined in -current > netgraph, to disable or re-enable data flow from "A". hmmm... i should look into it. > can you give me more information to help me understand > the problem a bit better. i hope this helps, thanks, max To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 11:36:14 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.wrs.com (unknown-1-11.windriver.com [147.11.1.11]) by hub.freebsd.org (Postfix) with ESMTP id AEDA037B42B for ; Wed, 26 Sep 2001 11:35:48 -0700 (PDT) Received: from laptop.baldwin.cx ([147.11.46.209]) by mail.wrs.com (8.9.3/8.9.1) with ESMTP id LAA11738; Wed, 26 Sep 2001 11:35:38 -0700 (PDT) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <15281.4529.188277.263940@csa.bu.edu> Date: Wed, 26 Sep 2001 11:35:24 -0700 (PDT) From: John Baldwin To: Evan Sarmiento Subject: RE: IMPORTANT!! Re: panic on mount Cc: freebsd-current@freebsdorg.FreeBSD.ORG, freebsd-hackers@FreeBSD.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 25-Sep-01 Evan Sarmiento wrote: > Hello, > > Just to clarify things for everyone who may be having this probme: > there is a panic on bootup with current, within the witness* code. > You can avoid this by commenting out WITNESS in your kernel configuration > and recompiling. It worked for me.. I've just fixed it. Sorry for the problems. > Hope this helps someone. > > Thanks, > Evan > > John Baldwin writes: > > > > On 25-Sep-01 Bill Fenner wrote: > > > > > > I also started getting this error with recent kernels (in the last > > > day or so). > > > > It looks like the mutex is really held since the mtx_assert before > > witness_unlock didn't trigger. You can try turning witness off for the > time > > being as a workaround. I'm not sure why witness would be broken, however. > > > > > Mounting root from ufs:/dev/ad0s1a > > > panic: lock (sleep mutex) vnode interlock not locked @ > > > /usr/src/sys/kern/vfs_default.c:460 > > > Debugger("panic") > > > Stopped at Debugger+0x44: pushl %ebx > > > db> t > > > Debugger(c03c5bbb) at Debugger+0x44 > > > panic(c03c8c40,c03c4b80,c03ccf20,c03cc8a0,1cc) at panic+0x70 > > > witness_unlock(c7765f2c,8,c03cc8a0,1cc,c7765f2c,1,c03c4ba0,f6) at > > > witness_unlock+0x1d0 > > > _mtx_unlock_flags(c7765f2c,0,c03cc8a0,1cc,c0567bd0) at > _mtx_unlock_flags+0x59 > > > vop_nolock(c0567be8,c0567bf8,c02920c2,c0567be8,c0567d4c) at > vop_nolock+0x24 > > > vop_defaultop(c0567be8) at vop_defaultop+0x15 > > > vn_lock(c7765ec0,20002,c049f7c4,c0567d4c,c1346680) at vn_lock+0xca > > > ffs_mountfs(c7765ec0,c1351600,c049f7c4,c0446900,c0567d4c) at > ffs_mountfs+0x7e > > > ffs_mount(c1351600,0,0,0,c049f7c4) at ffs_mount+0x67 > > > vfs_mountroot_try(c05447a8,c03cc48c) at vfs_mountroot_try+0x14e > > > vfs_mountroot(0,564c00,564000,0,c012caac) at vfs_mountroot+0x5a > > > mi_startup() at mi_startup+0x90 > > > begin() at begin+0x43 > > > > > > I dunno how to get a dump from this point since kern.dumpdev hasn't been > > > set.. > > > > > > Bill > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > > with "unsubscribe freebsd-current" in the body of the message > > > > -- > > > > John Baldwin -- http://www.FreeBSD.org/~jhb/ > > PGP Key: http://www.baldwin.cx/~john/pgpkey.asc > > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 12:26:15 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from freebie.xs4all.nl (freebie.xs4all.nl [213.84.32.253]) by hub.freebsd.org (Postfix) with ESMTP id DE5F037B42A for ; Wed, 26 Sep 2001 12:25:52 -0700 (PDT) Received: (from wkb@localhost) by freebie.xs4all.nl (8.11.6/8.11.6) id f8QJP4H26685; Wed, 26 Sep 2001 21:25:04 +0200 (CEST) (envelope-from wkb) Date: Wed, 26 Sep 2001 21:25:04 +0200 From: Wilko Bulte To: Matt Dillon Cc: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine Message-ID: <20010926212504.B26461@freebie.xs4all.nl> References: <96469.1001237641@critter> <200109231040.f8NAeXw86352@earth.backplane.com> <20010923124702.B9914@freebie.xs4all.nl> <200109232109.f8NL9kU88657@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109232109.f8NL9kU88657@earth.backplane.com>; from dillon@earth.backplane.com on Sun, Sep 23, 2001 at 02:09:46PM -0700 X-OS: FreeBSD 4.4-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Sep 23, 2001 at 02:09:46PM -0700, Matt Dillon wrote: > > :Has the problem of small-memory machines (< 64M IIRC) solved now? As I > :understand it vmiodirenable is counter-productive for these boxes. > :Maybe one could decide on-boot whether the amount of mem is enough to > :make it useful? > : > :Just a thought of course. > : > :| / o / /_ _ email: wilko@FreeBSD.org > > Small memory machines never had a problem. Even though there can be Then I suggest the following to be changed: # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # $FreeBSD: src/etc/sysctl.conf,v 1.5 2001/08/26 02:37:22 dd Exp $ vfs.vmiodirenable=1 # Set to 1 to enable the use of the VM subsystem to # back UFS directory memory requirements. Because of # the amount of wasted memory this causes it's not # advised for machines with less than 64MB of RAM, # on machines with more than 64MB it can provide a # substantial benefit related to directory caching. That was what I read and confused me ;) > considerable memory inefficiencies using vmiodirenable (e.g. a directory > less then 512 bytes eats 512 bytes of physical ram with vmiodirenable > turned off, and 4K with it turned on), there are two compensating factors: > First, the VM Paging cache is a cache, so the cached directory blocks can > be thrown away. Second, the VM Page cache has all of memory to play > with while the buffer cache with vmiodirenable turned off on a 64MB > machine will reserve less then a megabyte to cache directories. #2 is > important because it leads to more I/O which has a far greater effect > on the system then memory waste. Also, if you look at the typical > program's memory footprint and assume that actively accessed directories > eat 4K, the memory used to hold the active directories winds up being > almost nothing compared to the RSS of the program using those directories. > Single-use directories (e.g. make buildworld) are recycled in the VM Page > cache and do not eat as much memory as you might otherwise think. > > So even though the memory inefficiency can be up to 8:1 (4096/512), > this factor is somewhat deceptive in regards to the actual effect on > the system. > > -Matt ---end of quoted text--- -- | / o / /_ _ email: wilko@FreeBSD.org |/|/ / / /( (_) Bulte Arnhem, The Netherlands To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 12:32:45 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 3B3D037B412 for ; Wed, 26 Sep 2001 12:32:40 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8QJWXW14571; Wed, 26 Sep 2001 12:32:33 -0700 (PDT) (envelope-from dillon) Date: Wed, 26 Sep 2001 12:32:33 -0700 (PDT) From: Matt Dillon Message-Id: <200109261932.f8QJWXW14571@earth.backplane.com> To: Wilko Bulte Cc: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <96469.1001237641@critter> <200109231040.f8NAeXw86352@earth.backplane.com> <20010923124702.B9914@freebie.xs4all.nl> <200109232109.f8NL9kU88657@earth.backplane.com> <20010926212504.B26461@freebie.xs4all.nl> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :Then I suggest the following to be changed: : :# :# This file is read when going to multi-user and its contents piped thru :# ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. :# : :# $FreeBSD: src/etc/sysctl.conf,v 1.5 2001/08/26 02:37:22 dd Exp $ : :vfs.vmiodirenable=1 # Set to 1 to enable the use of the VM subsystem to : # back UFS directory memory requirements. Because of : # the amount of wasted memory this causes it's not : # advised for machines with less than 64MB of RAM, : # on machines with more than 64MB it can provide a : # substantial benefit related to directory caching. : :That was what I read and confused me ;) Yah. That isn't correct any more. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 13:57:50 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from freebie.xs4all.nl (freebie.xs4all.nl [213.84.32.253]) by hub.freebsd.org (Postfix) with ESMTP id 8E75237B40E for ; Wed, 26 Sep 2001 13:57:46 -0700 (PDT) Received: (from wkb@localhost) by freebie.xs4all.nl (8.11.6/8.11.6) id f8QKv8N27037; Wed, 26 Sep 2001 22:57:08 +0200 (CEST) (envelope-from wkb) Date: Wed, 26 Sep 2001 22:57:07 +0200 From: Wilko Bulte To: Matt Dillon Cc: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine Message-ID: <20010926225707.A27006@freebie.xs4all.nl> References: <96469.1001237641@critter> <200109231040.f8NAeXw86352@earth.backplane.com> <20010923124702.B9914@freebie.xs4all.nl> <200109232109.f8NL9kU88657@earth.backplane.com> <20010926212504.B26461@freebie.xs4all.nl> <200109261932.f8QJWXW14571@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109261932.f8QJWXW14571@earth.backplane.com>; from dillon@earth.backplane.com on Wed, Sep 26, 2001 at 12:32:33PM -0700 X-OS: FreeBSD 4.4-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Sep 26, 2001 at 12:32:33PM -0700, Matt Dillon wrote: > : > :Then I suggest the following to be changed: > : > :# > :# This file is read when going to multi-user and its contents piped thru > :# ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. > :# > : > :# $FreeBSD: src/etc/sysctl.conf,v 1.5 2001/08/26 02:37:22 dd Exp $ > : > :vfs.vmiodirenable=1 # Set to 1 to enable the use of the VM subsystem to > : # back UFS directory memory requirements. Because of > : # the amount of wasted memory this causes it's not > : # advised for machines with less than 64MB of RAM, > : # on machines with more than 64MB it can provide a > : # substantial benefit related to directory caching. > : > :That was what I read and confused me ;) > > Yah. That isn't correct any more. > > -Matt Do you want me to change the comment section of sysctl.conf, or what is your preference here? Wilko -- | / o / /_ _ email: wilko@FreeBSD.org |/|/ / / /( (_) Bulte Arnhem, The Netherlands To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 14: 0:25 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from freebie.xs4all.nl (freebie.xs4all.nl [213.84.32.253]) by hub.freebsd.org (Postfix) with ESMTP id 9C50C37B42B for ; Wed, 26 Sep 2001 14:00:19 -0700 (PDT) Received: (from wkb@localhost) by freebie.xs4all.nl (8.11.6/8.11.6) id f8QKxg027082; Wed, 26 Sep 2001 22:59:42 +0200 (CEST) (envelope-from wkb) Date: Wed, 26 Sep 2001 22:59:42 +0200 From: Wilko Bulte To: Matt Dillon Cc: Poul-Henning Kamp , David Greenman , Seigo Tanimura , bright@wintelcom.net, hackers@FreeBSD.ORG Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine Message-ID: <20010926225942.A27068@freebie.xs4all.nl> References: <96469.1001237641@critter> <200109231040.f8NAeXw86352@earth.backplane.com> <20010923124702.B9914@freebie.xs4all.nl> <200109232109.f8NL9kU88657@earth.backplane.com> <20010926212504.B26461@freebie.xs4all.nl> <200109261932.f8QJWXW14571@earth.backplane.com> <20010926225707.A27006@freebie.xs4all.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010926225707.A27006@freebie.xs4all.nl>; from wkb@freebie.xs4all.nl on Wed, Sep 26, 2001 at 10:57:07PM +0200 X-OS: FreeBSD 4.4-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Sep 26, 2001 at 10:57:07PM +0200, Wilko Bulte wrote: > On Wed, Sep 26, 2001 at 12:32:33PM -0700, Matt Dillon wrote: > > : > > :Then I suggest the following to be changed: > > : > > :# > > :# This file is read when going to multi-user and its contents piped thru > > :# ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. > > :# > > : > > :# $FreeBSD: src/etc/sysctl.conf,v 1.5 2001/08/26 02:37:22 dd Exp $ > > : > > :vfs.vmiodirenable=1 # Set to 1 to enable the use of the VM subsystem to > > : # back UFS directory memory requirements. Because of > > : # the amount of wasted memory this causes it's not > > : # advised for machines with less than 64MB of RAM, > > : # on machines with more than 64MB it can provide a > > : # substantial benefit related to directory caching. > > : > > :That was what I read and confused me ;) > > > > Yah. That isn't correct any more. > > > > -Matt > > Do you want me to change the comment section of sysctl.conf, or > what is your preference here? Never mind, I just saw the commit message :-/ -- | / o / /_ _ email: wilko@FreeBSD.org |/|/ / / /( (_) Bulte Arnhem, The Netherlands To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 18:20: 9 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from obsecurity.dyndns.org (adsl-63-207-60-153.dsl.lsan03.pacbell.net [63.207.60.153]) by hub.freebsd.org (Postfix) with ESMTP id D26F237B425 for ; Wed, 26 Sep 2001 18:20:04 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 55CFD66D46; Wed, 26 Sep 2001 18:20:04 -0700 (PDT) Date: Wed, 26 Sep 2001 18:20:04 -0700 From: Kris Kennaway To: Julian Elischer Cc: Kris Kennaway , Peter Wemm , Paul Saab , hackers@FreeBSD.ORG Subject: Re: librsa and 4.4 Message-ID: <20010926182003.A90727@xor.obsecurity.org> References: <20010926100403.A86509@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="ReaqsoxgOBHFXBhH" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from julian@elischer.org on Wed, Sep 26, 2001 at 11:12:17AM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --ReaqsoxgOBHFXBhH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 26, 2001 at 11:12:17AM -0700, Julian Elischer wrote: >=20 >=20 > On Wed, 26 Sep 2001, Kris Kennaway wrote: >=20 > > On Tue, Sep 25, 2001 at 02:54:51PM -0700, Peter Wemm wrote: > > > Julian Elischer wrote: > > > >=20 > > > > In a related problem: > > > >=20 > > > > we have a set of 4.1.1 binaries we want ot run on 4.4 > > > > but they (apache+other stuff) want to find a librsaUSA.so > > > > but can't.. I fixed it by copying the one from 4.1.1 into /usr/lib/= compat. > > > > Is that the right answer? > > > > Is it possible we can have a compat librsa? > > > > (maybe even empty if the stuff is now in libcrypt or something). > > >=20 > > > libcrypto.so.1.gz.uu and libssl.so.1.gz.uu need to be MFC'ed. This s= hould > > > have been done before 4.4-REL as well. > >=20 > > Um, they were MFCed a long time ago. Julian probably didn't have > > compat4x enabled. >=20 > We installed directly from released images.. > there was no option to install "compat4X" in the 4.4 installer. I could have sworn there was.. > In the old systems we would have had to install > librsa in some form or other. > now these functions are 'standard' in some way but some programs still > want to "see" librsa on the system, so we should have a way of saying > "yep.. it's there" =20 Yes, it's taken care of by the compat4x distribution. It includes the verions of the openssl libraries which used to be "international" (don't require librsausa) but which can now be distributed to US people as well. The solution to your problem is to install the compat4x libraries. Kris --ReaqsoxgOBHFXBhH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7sn7DWry0BWjoQKURArM7AKCGYa/VRqgKwICtKYCMNup42M231ACg/pvJ +37s55kMpDbCMtnz2LxQtxQ= =bYP0 -----END PGP SIGNATURE----- --ReaqsoxgOBHFXBhH-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 18:43:54 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mw1.texas.net (mw1.texas.net [206.127.30.11]) by hub.freebsd.org (Postfix) with ESMTP id 87E9737B415; Wed, 26 Sep 2001 18:43:39 -0700 (PDT) Received: from staff3.texas.net (staff3.texas.net [207.207.0.40]) by mw1.texas.net (8.11.6/8.11.6) with ESMTP id f8R1hcQ02840; Wed, 26 Sep 2001 20:43:38 -0500 (CDT) Received: (from doug@localhost) by staff3.texas.net (8.11.6/8.11.6) id f8R1hYw15926; Wed, 26 Sep 2001 20:43:34 -0500 (CDT) (envelope-from doug) Date: Wed, 26 Sep 2001 20:43:33 -0500 From: Douglas Swarin To: hackers@freebsd.org Cc: dillon@earth.backplane.com, tmoestl@gmx.net, bp@freebsd.org Subject: Re: more on Re: Please review: bugfix for vinvalbuf() Message-ID: <20010926204333.A15865@staff.texas.net> References: <20010711003926.B8799@crow.dom2ip.de> <200107110643.f6B6hTB24707@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200107110643.f6B6hTB24707@earth.backplane.com>; from dillon@earth.backplane.com on Tue, Jul 10, 2001 at 11:43:29PM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, Jul 10, 2001 at 11:43:29PM -0700, Matt Dillon wrote: > > :Hi, > : > :I've just tripped over an obviously long-standing (since about > :Jan. 1998) bug in vinvalbuf while looking into PR kern/26224. The > :problematic code looks like (on -CURRENT): > : > : /* > : * Destroy the copy in the VM cache, too. > : */ > : mtx_lock(&vp->v_interlock); > : if (VOP_GETVOBJECT(vp, &object) == 0) { > : vm_object_page_remove(object, 0, 0, > : (flags & V_SAVE) ? TRUE : FALSE); > : } > : mtx_unlock(&vp->v_interlock); > : > :The locks seem to be needed for file systems that don't perform real > :locking (on -STABLE, they are simplelocks). > :This, however, is incorrect because vm_object_page_remove may sleep. > :I've attached a patch that passes the interlock to > :vm_object_page_remove, which in turn passes it to a modified version > :of vm_page_sleep, which unlocks it around the sleep. > :I think that this is correct, because the object should be in a valid > :state when we sleep (and all checks are reexecuted in that case). > : > :Since I'm not very experienced with vfs and vm stuff, I'd be glad if > :this patch could get some review. In particular, is the lock/unlock > :pair really still needed, and are there notable differeces in -STABLE > :(because the fix would need the be MFC'ed)? > : > :Thanks, > : - thomas > > Ok, I've looked at this more. What is supposed to happen is that > the 'while (vp->v_numoutput) ...' code just above the section you > cited is supposed to prevent the deadlock. The while code looks like > this: > > while (vp->v_numoutput > 0) { > vp->v_flag |= VBWAIT; > tsleep(&vp->v_numoutput, PVM, "vnvlbv", 0); > } > > However, as Ian points out in his followup, it doesn't work: > > :... > :I've seen a related deadlock here in 4.x with NFS; vm_fault locks > :a page and then calls vput which aquires the v_interlock. This code > :in vinvalbuf locks the v_interlock first, and then vm_object_page_remove() > :locks the page. That's a simple lock-order reversal deadlock which I > :guess would still exist even with this patch. > > It doesn't work for the simple reason that vm_fault isn't busying the > page for writing, it's busying it for reading, so v_numoutput will be > 0. > > I think the best solution is to change vinvalbuf's wait loop to > wait on vm_object->paging_in_progress instead of vp->v_numoutput, > or perhaps to wait for both conditions to be satisfied. > > vm_object->paging_in_progress applies to reads and writes, while > vp->v_numoutput only applies to writes. > > I know this isn't the most correct solution... there is still the > lock reversal if vm_object_remove_pages() were ever to sleep. The > idea is that it won't sleep if there is no paging in progress because > nobody will have any of the object's pages locked. I think it is > the best we can do for the moment. There are several places in > vm/vm_object.c where the same assumption is made (testing against > vm_object->paging_in_progress, though, which works, not vp->v_numoutput). > > - > > Thomas, if you are interested this could be a little project for you. > See if you can code-up another while() loop to also wait for > the object's paging_in_progress count to become 0 in the vinvalbuf() > code. Look in vm/vm_object.c for examples of how to construct such > a loop. I will be happy to review and test whatever you come up with > and I'm sure Ian will too! > > -Matt > > I recently mentioned on freebsd-stable in message <20010923124824.A16392@staff.texas.net> a recurring rslock panic which I believe has been caused by the below mentioned problem in vinvalbuf(). I have worked up a patch for -STABLE (which should also apply to -CURRENT if there have not been major changes to this code). The patch is appended to this message for review. Data from the crash dump revealed the following: SMP 2 cpus IdlePTD 3555328 initial pcb at 2cf280 panicstr: rslock: cpu: 0, addr: 0xd7be66ec, lock: 0x00000001 panic messages: --- panic: rslock: cpu: 0, addr: 0xd7be66ec, lock: 0x00000001 mp_lock = 00000001; cpuid = 0; lapic.id = 01000000 boot() called on cpu#0 --- #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:473 #1 0xc016cf8f in boot (howto=256) at /usr/src/sys/kern/kern_shutdown.c:313 #2 0xc016d3a9 in panic (fmt=0xc025bcce "rslock: cpu: %d, addr: 0x%08x, lock: 0x%08x") at /usr/src/sys/kern/kern_shutdown.c:581 #3 0xc025bcce in bsl1 () #4 0xc021eeac in _unlock_things (fs=0xd7a6dec4, dealloc=0) at /usr/src/sys/vm/vm_fault.c:147 #5 0xc021f8c7 in vm_fault (map=0xd7a6bf40, vaddr=673968128, fault_type=1 '\001', fault_flags=0) at /usr/src/sys/vm/vm_fault.c:826 #6 0xc025d016 in trap_pfault (frame=0xd7a6dfa8, usermode=1, eva=673972223) at /usr/src/sys/i386/i386/trap.c:829 #7 0xc025ca8b in trap (frame={tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 99049, tf_esi = 0, tf_ebp = -1077937884, tf_isp = -676929580, tf_ebx = 48110729, tf_edx = 0, tf_ecx = 1835007, tf_eax = 672137216, tf_trapno = 12, tf_err = 4, tf_eip = 134517190, tf_cs = 31, tf_eflags = 66070, tf_esp = -1077937940, tf_ss = 47}) at /usr/src/sys/i386/i386/trap.c:359 #8 0x80491c6 in ?? () #9 0x8048d9e in ?? () #10 0x804a078 in ?? () #11 0x8048b45 in ?? () --- A quick review of processes revealed a process stuck in vmopar: (kgdb) ps ... 46479 d7ffc560 d806e000 235886 1 46394 004006 3 tail vmopar c09120c8 ... which was sleeping in vm_object_page_remove() in vinvalbuf(): (kgdb) btp 46479 frame 0 at 0xd806fc8c: ebp d806fcb8, eip 0xc0170051 : mov 0x141(%ebx),%al frame 1 at 0xd806fcb8: ebp d806fce0, eip 0xc02262e8 : add $0x10,%esp frame 2 at 0xd806fce0: ebp d806fd2c, eip 0xc019a667 : add $0x10,%esp frame 3 at 0xd806fd2c: ebp d806fd60, eip 0xc01d0aa0 : add $0x18,%esp frame 4 at 0xd806fd60: ebp d806fe2c, eip 0xc01cf5d8 : mov %eax,0xffffff74(%ebp) frame 5 at 0xd806fe2c: ebp d806fe44, eip 0xc01f6842 : jmp 0xc01f6849 frame 6 at 0xd806fe44: ebp d806fe78, eip 0xc01a22cc : mov %eax,0xffffffe8(%ebp) frame 7 at 0xd806fe78: ebp d806fef4, eip 0xc017b690 : mov %eax,%esi frame 8 at 0xd806fef4: ebp d806ff28, eip 0xc017b556 : mov %eax,%esi frame 9 at 0xd806ff28: ebp d806ffa0, eip 0xc025d745 : mov %eax,0xffffffb8(%ebp) The patch is below, against vfs_subr.c 1.249.2.11 2001/09/11 --- vfs_subr.c Tue Sep 11 04:49:53 2001 +++ vfs_subr.c.new Wed Sep 26 15:23:09 2001 @@ -721,9 +721,9 @@ } } - while (vp->v_numoutput > 0) { - vp->v_flag |= VBWAIT; - tsleep(&vp->v_numoutput, PVM, "vnvlbv", 0); + if (VOP_GETVOBJECT(vp, &object) == 0) { + while (object->paging_in_progress) + vm_object_pip_sleep(object, "vnvlbv"); } splx(s); To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 18:52:12 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from hotmail.com (f147.law3.hotmail.com [209.185.241.147]) by hub.freebsd.org (Postfix) with ESMTP id ADC9037B42C for ; Wed, 26 Sep 2001 18:52:09 -0700 (PDT) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 26 Sep 2001 18:40:39 -0700 Received: from 205.158.104.176 by lw3fd.law3.hotmail.msn.com with HTTP; Thu, 27 Sep 2001 01:40:38 GMT X-Originating-IP: [205.158.104.176] From: "Srinivas Dharmasanam" To: freebsd-hackers@freebsd.org Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 Date: Wed, 26 Sep 2001 18:40:38 -0700 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 27 Sep 2001 01:40:39.0325 (UTC) FILETIME=[69745CD0:01C146F5] Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, I'm trying to use the TCP&IP checksum offload capability of the Netgear GA620 NIC from a SMP FreeBSD 4.2R system running on a typical PIII SBC. I did enable TCP&IP cksum offload for receive operations by setting the if_hwassist flag in the driver /sys/pci/if_ti.c and verified that it is working. However I'm unable to offload TCP&IP checksum for the send operations. For the send operation, I see that the kernel is still doing the TCP and IP checksum calculations although the driver has the hw_assist flag set. Can you please let me know what else needs to be done to offload TCP and IP checksum for both rcv/send operations on the Tigon2 NIC? Thanks, -Srinivas _________________________________________________________________ Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 18:59:56 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from prism.flugsvamp.com (cb58709-a.mdsn1.wi.home.com [24.17.241.9]) by hub.freebsd.org (Postfix) with ESMTP id 6E22737B411 for ; Wed, 26 Sep 2001 18:59:53 -0700 (PDT) Received: (from jlemon@localhost) by prism.flugsvamp.com (8.11.0/8.11.0) id f8R1vZ546863; Wed, 26 Sep 2001 20:57:35 -0500 (CDT) (envelope-from jlemon) Date: Wed, 26 Sep 2001 20:57:35 -0500 (CDT) From: Jonathan Lemon Message-Id: <200109270157.f8R1vZ546863@prism.flugsvamp.com> To: the_srinivas@hotmail.com, hackers@freebsd.org Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 X-Newsgroups: local.mail.freebsd-hackers In-Reply-To: Organization: Cc: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In article you write: >Hello, >I'm trying to use the TCP&IP checksum offload capability of the Netgear >GA620 NIC from a SMP FreeBSD 4.2R system running on a typical PIII SBC. >I did enable TCP&IP cksum offload for receive operations by setting the >if_hwassist flag in the driver /sys/pci/if_ti.c and verified that it is >working. However I'm unable to offload TCP&IP checksum for the send >operations. All you need to do is uncomment the TI_CSUM_FEATURES flag at the top of the file, and recompile. -- Jonatha To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Sep 26 21:36: 6 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from ms10.hinet.net (ms10.hinet.net [168.95.4.100]) by hub.freebsd.org (Postfix) with ESMTP id 63E8C37B43D; Wed, 26 Sep 2001 21:35:52 -0700 (PDT) Received: from 168.95.192.1 (61-216-51-248.HINET-IP.hinet.net [61.216.51.248]) by ms10.hinet.net (8.8.8/8.8.8) with SMTP id MAA08686; Thu, 27 Sep 2001 12:35:45 +0800 (CST) Message-Id: <200109270435.MAA08686@ms10.hinet.net> From: "=?BIG5?Q?12345=A2I12345?=" To: "" <> Subject: =?BIG5?Q?2=A4p=AE=C9=A5=DF=A7Y=C0=B0=B1z=BA=F4=B8=F4=B6}=A9=B1=C0=E7=B7~!!?= Date: Thu, 27 Sep 2001 10:42:42 +0800 Content-Type: multipart/mixed; boundary="---RisingEdge.798C1A93.2001.09.27.10.42.42" Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multipart message in mime format. -----RisingEdge.798C1A93.2001.09.27.10.42.42 Content-Type: multipart/alternative; boundary="---RisingEdge.14FEBFB6.2001.09.27.10.42.42" Content-Transfer-Encoding: 8bit -----RisingEdge.14FEBFB6.2001.09.27.10.42.42 Content-Type: text/plain; charset="BIG5" Content-Transfer-Encoding: quoted-printable 2=A4p=AE=C9=A7=D6=B3t=BA=F4=AD=B6=BBs=A7@=A4=CE=AD=D7=A7=EF. =BA=F4=A7= }=A7K=B6O!! =BA=F4=AD=B6=BBs=A7@=B6W=A7C=BB=F9=A5u=ADnNT800=A4=B8. =A7K=B6O=BA=F4= =A7}=B1=BE=A4W. =A5=DF=A7Y=C0=B0=B1z=BA=F4=B8=F4=B6}=A9=B1=C0=E7=B7~=B0= =B5=A5=CD=B7N!! (=BA=F4=AF=B8=A4=BA=AEe=A5i=A4=C0: =A4=BD=A5q=C2=B2=A4=B6. =B2=A3=AB= ~=A4=B6=B2=D0. =B2=A3=AB~=AC=DB=A4=F9. =AB=C8=A4=E1=AFd=A8=A5=AAO. =A9= w=C1=CA=AA=ED=B3=E6. =B1m=A6=E2=B0=CA=B5e=B5=A5) =A6=B3=B7N=AA=CC=C5w=AA=EF=AC=A2=BD=CD!! http://in.members.tripodasia.com/gffff/ =BA=F4=AD=B6=B3]=ADp=AFd=A8=A5: http://in.members.tripodasia.com/gffff/k.htm =BD=D0=A4=C5=AA=BD=B1=B5=A6^=ABH!! -----RisingEdge.14FEBFB6.2001.09.27.10.42.42-- -----RisingEdge.798C1A93.2001.09.27.10.42.42-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 0: 7:17 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from sasami.jurai.net (sasami.jurai.net [64.0.106.45]) by hub.freebsd.org (Postfix) with ESMTP id AE8F537B425 for ; Thu, 27 Sep 2001 00:07:14 -0700 (PDT) Received: from localhost (winter@localhost) by sasami.jurai.net (8.9.3/8.8.7) with ESMTP id DAA57841; Thu, 27 Sep 2001 03:06:33 -0400 (EDT) Date: Thu, 27 Sep 2001 03:06:33 -0400 (EDT) From: "Matthew N. Dodd" To: mark tinguely Cc: fjoe@iclub.nsu.ru, freebsd-hackers@FreeBSD.ORG Subject: Re: arcnet support for FreeBSD (request for review) In-Reply-To: <200109261759.f8QHx1q97994@web.cs.ndsu.nodak.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 26 Sep 2001, mark tinguely wrote: > There is nothing like raising a topic that was last seen several months > ago, but ... > > Has there been any serious consideration to committing the arcnet code > that mentioned on 20 Jul 2001 (http://iclub.nsu.ru/~fjoe/arcnet/)? > > I guess I should ask if anyone else has tried the code. I will most > likely have another driver for arcnet in a couple months if the customer > goes ahead with the project. I keep meaning to take a look at it but haven't yet had the chance. I wanna see IPv6 over ARCNET; 2 great tastes that go great together! -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 0:57:29 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.nsu.ru (b.ns.ssc.nsu.ru [193.124.215.221]) by hub.freebsd.org (Postfix) with ESMTP id 0987337B425 for ; Thu, 27 Sep 2001 00:57:16 -0700 (PDT) Received: from iclub.nsu.ru ([193.124.222.66] ident=root) by mail.nsu.ru with esmtp (Exim 3.20 #1) id 15mW2d-0000bX-00; Thu, 27 Sep 2001 14:57:07 +0700 Received: (from fjoe@localhost) by iclub.nsu.ru (8.11.4/8.11.4) id f8R7v5R16988; Thu, 27 Sep 2001 14:57:05 +0700 (NSS) (envelope-from fjoe) Date: Thu, 27 Sep 2001 14:57:04 +0700 From: Max Khon To: mark tinguely Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: arcnet support for FreeBSD (request for review) Message-ID: <20010927145704.B16895@iclub.nsu.ru> References: <200109261759.f8QHx1q97994@web.cs.ndsu.nodak.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109261759.f8QHx1q97994@web.cs.ndsu.nodak.edu>; from tinguely@web.cs.ndsu.nodak.edu on Wed, Sep 26, 2001 at 12:59:01PM -0500 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG hi, there! On Wed, Sep 26, 2001 at 12:59:01PM -0500, mark tinguely wrote: > There is nothing like raising a topic that was last seen several months > ago, but ... > > Has there been any serious consideration to committing the arcnet code > that mentioned on 20 Jul 2001 (http://iclub.nsu.ru/~fjoe/arcnet/)? yes, I am working on this /fjoe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 6:25: 4 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 8C6F337B427 for ; Thu, 27 Sep 2001 06:25:00 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id JAA09390; Thu, 27 Sep 2001 09:24:52 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8RDOQp83675; Thu, 27 Sep 2001 09:24:26 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15283.10378.718858.212593@grasshopper.cs.duke.edu> Date: Thu, 27 Sep 2001 09:24:26 -0400 (EDT) To: Jonathan Lemon Cc: hackers@freebsd.org Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <200109270157.f8R1vZ546863@prism.flugsvamp.com> References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jonathan, I just wanted to say that you did a hell of a job with the csum offload stuff in FreeBSD. FreeBSD is the only OS that I'm aware of which allows a driver to choose not to handle csum'ing IP frags on transmit. Having the option to not handle frags is very, very handy. I wish other platforms had it. Thanks! Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 6:41: 6 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from acl.lanl.gov (acl.lanl.gov [128.165.147.1]) by hub.freebsd.org (Postfix) with SMTP id CDBAA37B439 for ; Thu, 27 Sep 2001 06:41:02 -0700 (PDT) Received: (qmail 90694 invoked from network); 27 Sep 2001 07:41:02 -0600 Received: from snaresland.acl.lanl.gov (128.165.147.113) by acl.lanl.gov with SMTP; 27 Sep 2001 07:41:02 -0600 Received: (qmail 26576 invoked by uid 3499); 27 Sep 2001 07:40:53 -0600 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 27 Sep 2001 07:40:53 -0600 Date: Thu, 27 Sep 2001 07:40:53 -0600 (MDT) From: Ronald G Minnich X-X-Sender: To: Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <15283.10378.718858.212593@grasshopper.cs.duke.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 27 Sep 2001, Andrew Gallatin wrote: > I just wanted to say that you did a hell of a job with the csum > offload stuff in FreeBSD. FreeBSD is the only OS that I'm aware of > which allows a driver to choose not to handle csum'ing IP frags on > transmit. Having the option to not handle frags is very, very handy. > I wish other platforms had it. I have a question on the checksum offloading. Has anyone measured any incidence of data corruption between the PCI card and memory. In other words, when you offload checksums the end-to-end checking becomes card-to-card checking, and the possibility exists that what goes in memory at the destination end is not what was sent at the source. Very remote possibility, of course, but ... It's not that the data gets corrupted (usually). It's that once-in-a-100-trillion errors could result in the occasional dropped half-packet or missed word (i.e. overflow). The missed word problem is usual a miscommunication between card and PCI chipset about how a PCI ABORT is supposed to work ... which we've seen on some very recent just-released chipset/network card combinations,. Does this happen? Yes. We've seen it on, to name just two, HIPPI800 and Myrinet cards. In each case it was not actual data corruption, it was "can't happen" DMA scenarios that once in a very long while (1 in 10^14 or so) resulted in bits of packets getting corrupted. Each of these cards has a very high-quality end-end CRC for the data, and Myrinet has flow control. We're not the only place that has seen this problem, and I've been told that many commerical Myrinet clients run IP over Myrinet because of these types of problems (of course FreeBSD has the fastest IP over Myrinet anyway, so it's not like that's a huge problem). Is it likely? Well, on one cluster here, with 48 machines and 12 interfaces per machine, it's not only likely, it's a given. Without software checksums you are going to get data corruption. What I don't know is whether offloaded checksums on commodity ethernet cards have seen anything similar. I assume that checksums across all the frags are done by the kernel (i.e. NFS would checksum the full UDP packet)? Has anyone measured to see if there is corruption occuring on the frags, ever? Of course it would probably take a while ... Thanks in advance for any information you might have. ron To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 7: 4:27 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from web.cs.ndsu.nodak.edu (web.cs.ndsu.NoDak.edu [134.129.125.7]) by hub.freebsd.org (Postfix) with ESMTP id E4F4037B43A for ; Thu, 27 Sep 2001 07:04:24 -0700 (PDT) Received: (from tinguely@localhost) by web.cs.ndsu.nodak.edu (8.11.4/8.11.4) id f8RE4Cp03337; Thu, 27 Sep 2001 09:04:12 -0500 (CDT) (envelope-from tinguely) Date: Thu, 27 Sep 2001 09:04:12 -0500 (CDT) From: mark tinguely Message-Id: <200109271404.f8RE4Cp03337@web.cs.ndsu.nodak.edu> To: fjoe@iclub.nsu.ru, tinguely@web.cs.ndsu.nodak.edu Subject: Re: arcnet support for FreeBSD (request for review) Cc: freebsd-hackers@FreeBSD.ORG In-Reply-To: <20010927145704.B16895@iclub.nsu.ru> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > On Wed, Sep 26, 2001 at 12:59:01PM -0500, mark tinguely wrote: > > > There is nothing like raising a topic that was last seen several months > > ago, but ... > > > > Has there been any serious consideration to committing the arcnet code > > that mentioned on 20 Jul 2001 (http://iclub.nsu.ru/~fjoe/arcnet/)? > > yes, I am working on this I noticed that the hooks for the netgraph and BRIDGE code is missing. I guess this is why I would like to see it committed so that there is not a constant catchup of new features. I also noticed that the outputed packet get bpf_mtap in two places, once for the complete packet arc_output(), and again in the outputting of the fragments (in the device driver). I wonder if the bpf_mtap in the arc_output() should be removed. I am not able to test yet, but I am willing to help. --mark tinguely To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 7:15:54 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mongrel.pacific.net.au (mongrel.pacific.net.au [61.8.0.107]) by hub.freebsd.org (Postfix) with ESMTP id 32F1E37B42A for ; Thu, 27 Sep 2001 07:15:50 -0700 (PDT) Received: from dungeon.home (ppp223.dyn249.pacific.net.au [203.143.249.223]) by mongrel.pacific.net.au (8.9.3/8.9.3/Debian 8.9.3-21) with ESMTP id AAA00993; Fri, 28 Sep 2001 00:15:10 +1000 X-Authentication-Warning: mongrel.pacific.net.au: Host ppp223.dyn249.pacific.net.au [203.143.249.223] claimed to be dungeon.home Received: from dungeon.home (localhost [127.0.0.1]) by dungeon.home (8.11.3/8.11.1) with ESMTP id f8REJp116422; Fri, 28 Sep 2001 00:19:51 +1000 (EST) (envelope-from mckay) Message-Id: <200109271419.f8REJp116422@dungeon.home> To: Poul-Henning Kamp Cc: hackers@freebsd.org, Matt Dillon , mckay@thehub.com.au Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <200109231040.f8NAeXw86352@earth.backplane.com> <98331.1001250396@critter> In-Reply-To: <98331.1001250396@critter> from Poul-Henning Kamp at "Sun, 23 Sep 2001 13:06:36 +0000" Date: Fri, 28 Sep 2001 00:19:51 +1000 From: Stephen McKay Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sunday, 23rd September 2001, Poul-Henning Kamp wrote: >Things to look out for: > >1. !ufs filesystems I am irredeemably slack for not testing this a lot but... I believe I saw bad interactions between vmiodirenable and isofs on 4.3-R. I mounted a CD, looked at stuff on it, did a lot of other work, went back to the CD and files were screwy (files contained the contents of other files, files were zero size). I unmounted and remounted the CD and everything was fine. The machine is a reliable old workhorse, and has no hardware errors. Since then, I've not had a chance to go back and check. It's only because you are making vmiodirenable the default that I'm mentioning it. Sorry for not making a proper bug report containing actual facts. :-( Stephen. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 7:16:40 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from whizzo.transsys.com (whizzo.TransSys.COM [144.202.42.10]) by hub.freebsd.org (Postfix) with ESMTP id B6C3937B405 for ; Thu, 27 Sep 2001 07:16:38 -0700 (PDT) Received: from whizzo.transsys.com (#6@localhost.transsys.com [127.0.0.1]) by whizzo.transsys.com (8.11.4/8.11.4) with ESMTP id f8REGaZ64624; Thu, 27 Sep 2001 10:16:36 -0400 (EDT) (envelope-from louie@whizzo.transsys.com) Message-Id: <200109271416.f8REGaZ64624@whizzo.transsys.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Ronald G Minnich Cc: hackers@FreeBSD.ORG X-Image-URL: http://www.transsys.com/louie/images/louie-mail.jpg From: "Louis A. Mamakos" Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: In-reply-to: Your message of "Thu, 27 Sep 2001 07:40:53 MDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 27 Sep 2001 10:16:36 -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The other type of failure you might not catch are software errors; that is, where a packet is produced by the network stack and then is subsequently stomped on by a random store from some other code. Or a mis-programmed I/O card with scatter/gather capability doesn't pick up what was intended, etc. The Internet checksum is useful for detecting this class of error. Louis Mamakos To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 7:22:21 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 2DBB837B401 for ; Thu, 27 Sep 2001 07:22:16 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id KAA10853; Thu, 27 Sep 2001 10:22:04 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8RELcY83736; Thu, 27 Sep 2001 10:21:38 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15283.13810.632100.547964@grasshopper.cs.duke.edu> Date: Thu, 27 Sep 2001 10:21:38 -0400 (EDT) To: Cc: Ronald G Minnich Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: References: <15283.10378.718858.212593@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ronald G Minnich writes: > I have a question on the checksum offloading. Has anyone measured any > incidence of data corruption between the PCI card and memory. In other > words, when you offload checksums the end-to-end checking becomes > card-to-card checking, and the possibility exists that what goes in memory > at the destination end is not what was sent at the source. Very remote > possibility, of course, but ... We used to see occasional data corruption at Duke with 440BX based motherboards with non-ecc ram. We never saw it on higher-quality hosts (alphas or serverworks based pc motherboards) with ecc memory. It would manifest itself as bad TCP checksums (no csum offload at the time). > of these types of problems (of course FreeBSD has the fastest IP over > Myrinet anyway, so it's not like that's a huge problem). > Not any more. A 2.4 linux kernel will do a bit better than FreeBSD on an SMP box because it is able to use both processors. Speaking of which -- who is working on making the network stack SMP capable in -current? Anything I can do to help? Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 7:36:12 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 132F237B434 for ; Thu, 27 Sep 2001 07:36:08 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id KAA11167; Thu, 27 Sep 2001 10:36:02 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8REZaU83753; Thu, 27 Sep 2001 10:35:36 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15283.14648.430630.163513@grasshopper.cs.duke.edu> Date: Thu, 27 Sep 2001 10:35:36 -0400 (EDT) To: "Louis A. Mamakos" Cc: Ronald G Minnich , hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <200109271416.f8REGaZ64624@whizzo.transsys.com> References: <200109271416.f8REGaZ64624@whizzo.transsys.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Louis A. Mamakos writes: > The other type of failure you might not catch are software errors; that > is, where a packet is produced by the network stack and then is > subsequently stomped on by a random store from some other code. Or > a mis-programmed I/O card with scatter/gather capability doesn't pick > up what was intended, etc. The Internet checksum is useful for > detecting this class of error. > No, you're missing the point almost entirely. The checksum is not skipped. It is calculated by the DMA engine based on the data that's transferred across the I/O bus on the receiver (and / or the sender). If the data is incorrect as seen by the receiving nic, the checksum will be wrong and the packet will be dropped. If the packet lands in the wrong place, you have much worse problems. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 7:42:55 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from bubo.vslib.cz (bubo.vslib.cz [147.230.16.1]) by hub.freebsd.org (Postfix) with ESMTP id 4DE5F37B406 for ; Thu, 27 Sep 2001 07:42:52 -0700 (PDT) Received: from A410A (a410a.kolej.vslib.cz [147.230.152.17]) by bubo.vslib.cz (Postfix) with SMTP id 3D77F8361 for ; Thu, 27 Sep 2001 16:42:51 +0200 (CEST) Message-ID: <003101c14762$eaf24e80$1198e693@kolej.vslib.cz> From: "Martin Vana" To: Subject: Planet Date: Thu, 27 Sep 2001 16:44:31 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_002E_01C14773.AE506500" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. ------=_NextPart_000_002E_01C14773.AE506500 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Hi, I tried to rebuilt my kernel with device rl0 (NIC PLANET ENW-9503A = based on REALTEK 8139 chip) it has collapsed during make phase. It = cannot find files starting with mii* but they are present. Thank you Im runnign FreeBSD 4.4(Amnesiac) ------=_NextPart_000_002E_01C14773.AE506500 Content-Type: text/html; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable
Hi, I tried to rebuilt my kernel with = device rl0=20 (NIC PLANET ENW-9503A based on REALTEK 8139 chip) it has collapsed = during make=20 phase. It cannot find files starting with mii* but they are=20 present.
Thank you
 
Im runnign FreeBSD=20 4.4(Amnesiac)
------=_NextPart_000_002E_01C14773.AE506500-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 7:56:23 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from bubo.vslib.cz (bubo.vslib.cz [147.230.16.1]) by hub.freebsd.org (Postfix) with ESMTP id DC7FC37B405 for ; Thu, 27 Sep 2001 07:56:20 -0700 (PDT) Received: from A410A (a410a.kolej.vslib.cz [147.230.152.17]) by bubo.vslib.cz (Postfix) with SMTP id EC85180BE for ; Thu, 27 Sep 2001 16:56:19 +0200 (CEST) Message-ID: <00df01c14764$ccd07600$1198e693@kolej.vslib.cz> From: "Martin Vana" To: Subject: Planet Date: Thu, 27 Sep 2001 16:58:00 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_00DC_01C14775.9040DC00" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. ------=_NextPart_000_00DC_01C14775.9040DC00 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Hi, I tried to rebuilt my kernel with device rl0 (NIC PLANET ENW-9503A = based on REALTEK 8139 chip) it has collapsed during make phase. It = cannot find files starting with mii* but they are present. Thank you Im runnign FreeBSD 4.4(Amnesiac) ------=_NextPart_000_00DC_01C14775.9040DC00 Content-Type: text/html; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable
Hi, I tried to rebuilt my kernel with = device rl0=20 (NIC PLANET ENW-9503A based on REALTEK 8139 chip) it has collapsed = during make=20 phase. It cannot find files starting with mii* but they are=20 present.
Thank you
 
Im runnign FreeBSD=20 4.4(Amnesiac)
------=_NextPart_000_00DC_01C14775.9040DC00-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 7:56:46 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from ringworld.nanolink.com (straylight.ringlet.net [217.75.134.254]) by hub.freebsd.org (Postfix) with SMTP id 730EB37B40B for ; Thu, 27 Sep 2001 07:56:41 -0700 (PDT) Received: (qmail 13537 invoked by uid 1000); 27 Sep 2001 14:49:54 -0000 Date: Thu, 27 Sep 2001 17:49:54 +0300 From: Peter Pentchev To: Martin Vana Cc: freebsd-hackers@freebsd.org Subject: Re: Planet Message-ID: <20010927174954.B658@ringworld.oblivion.bg> Mail-Followup-To: Martin Vana , freebsd-hackers@freebsd.org References: <003101c14762$eaf24e80$1198e693@kolej.vslib.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <003101c14762$eaf24e80$1198e693@kolej.vslib.cz>; from martin.vana@vslib.cz on Thu, Sep 27, 2001 at 04:44:31PM +0200 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Sep 27, 2001 at 04:44:31PM +0200, Martin Vana wrote: > Hi, I tried to rebuilt my kernel with device rl0 (NIC PLANET ENW-9503A based on REALTEK 8139 chip) it has collapsed during make phase. It cannot find files starting with mii* but they are present. Is it that it cannot find the files, or functions from those files? If it's functions that it cannot find, try adding 'device miibus' to your kernel, like the comments in LINT say (rl is listed under 'PCI Ethernet NICs that use the common MII bus controller code'). If it is the files themselves - are you sure you have re-run config(8) after changing your kernel configuration file? To be absolutely certain, run config with the -r flag to remove any old directories. To be *really* certain, use the buildworld/buildkernel way. G'luck, Peter -- I've heard that this sentence is a rumor. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 8:31:36 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from acl.lanl.gov (acl.lanl.gov [128.165.147.1]) by hub.freebsd.org (Postfix) with SMTP id 1D95C37B41C for ; Thu, 27 Sep 2001 08:31:33 -0700 (PDT) Received: (qmail 93681 invoked from network); 27 Sep 2001 09:31:32 -0600 Received: from snaresland.acl.lanl.gov (128.165.147.113) by acl.lanl.gov with SMTP; 27 Sep 2001 09:31:32 -0600 Received: (qmail 27153 invoked by uid 3499); 27 Sep 2001 09:31:32 -0600 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 27 Sep 2001 09:31:32 -0600 Date: Thu, 27 Sep 2001 09:31:32 -0600 (MDT) From: Ronald G Minnich X-X-Sender: To: Andrew Gallatin Cc: "Louis A. Mamakos" , Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <15283.14648.430630.163513@grasshopper.cs.duke.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 27 Sep 2001, Andrew Gallatin wrote: > No, you're missing the point almost entirely. The checksum is not > skipped. It is calculated by the DMA engine based on the data that's > transferred across the I/O bus on the receiver (and / or the sender). > If the data is incorrect as seen by the receiving nic, the checksum > will be wrong and the packet will be dropped. you still have a potential problem here with variance in chipsets, namely the case of broken ABORT or other unusual PCI cycle handling (missed word problem). I agree it's a low probability. But we've seen it, just a week or two ago on a brand new box. But then we tend to see things here nobody else sees due to our scale. ron To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 8:42:37 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from beppo.feral.com (beppo.feral.com [192.67.166.79]) by hub.freebsd.org (Postfix) with ESMTP id 0EF6C37B426 for ; Thu, 27 Sep 2001 08:42:33 -0700 (PDT) Received: from mailhost.feral.com (mjacob@mailhost.feral.com [192.67.166.1]) by beppo.feral.com (8.11.3/8.11.3) with ESMTP id f8RFgPH64720; Thu, 27 Sep 2001 08:42:25 -0700 (PDT) (envelope-from mjacob@feral.com) Date: Thu, 27 Sep 2001 08:42:25 -0700 (PDT) From: Matthew Jacob X-Sender: mjacob@beppo Reply-To: mjacob@feral.com To: Ronald G Minnich Cc: hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG It certainly occurs at a rate to worry one. Alan Poston found definite cases of corruption when doing heavy IDE testing. It varies, motherboard to motherboard. On Thu, 27 Sep 2001, Ronald G Minnich wrote: > On Thu, 27 Sep 2001, Andrew Gallatin wrote: > > > I just wanted to say that you did a hell of a job with the csum > > offload stuff in FreeBSD. FreeBSD is the only OS that I'm aware of > > which allows a driver to choose not to handle csum'ing IP frags on > > transmit. Having the option to not handle frags is very, very handy. > > I wish other platforms had it. > > I have a question on the checksum offloading. Has anyone measured any > incidence of data corruption between the PCI card and memory. In other > words, when you offload checksums the end-to-end checking becomes > card-to-card checking, and the possibility exists that what goes in memory > at the destination end is not what was sent at the source. Very remote > possibility, of course, but ... > > It's not that the data gets corrupted (usually). It's that > once-in-a-100-trillion errors could result in the occasional dropped > half-packet or missed word (i.e. overflow). The missed word problem is > usual a miscommunication between card and PCI chipset about how a PCI > ABORT is supposed to work ... which we've seen on some very recent > just-released chipset/network card combinations,. > > Does this happen? Yes. We've seen it on, to name just two, HIPPI800 and > Myrinet cards. In each case it was not actual data corruption, it was > "can't happen" DMA scenarios that once in a very long while (1 in 10^14 or > so) resulted in bits of packets getting corrupted. Each of these cards > has a very high-quality end-end CRC for the data, and Myrinet has flow > control. We're not the only place that has seen this problem, and I've > been told that many commerical Myrinet clients run IP over Myrinet because > of these types of problems (of course FreeBSD has the fastest IP over > Myrinet anyway, so it's not like that's a huge problem). > > Is it likely? Well, on one cluster here, with 48 machines and 12 > interfaces per machine, it's not only likely, it's a given. Without > software checksums you are going to get data corruption. > > What I don't know is whether offloaded checksums on commodity ethernet > cards have seen anything similar. > > I assume that checksums across all the frags are done by the kernel (i.e. > NFS would checksum the full UDP packet)? Has anyone measured to see if > there is corruption occuring on the frags, ever? Of course it would > probably take a while ... > > Thanks in advance for any information you might have. > > ron > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 8:58:19 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 6E94837B42C for ; Thu, 27 Sep 2001 08:58:11 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id LAA13398; Thu, 27 Sep 2001 11:57:30 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8RFv4o83861; Thu, 27 Sep 2001 11:57:04 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15283.19536.410864.339942@grasshopper.cs.duke.edu> Date: Thu, 27 Sep 2001 11:57:04 -0400 (EDT) To: Ronald G Minnich Cc: Andrew Gallatin , "Louis A. Mamakos" , Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: References: <15283.14648.430630.163513@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ronald G Minnich writes: > > you still have a potential problem here with variance in chipsets, namely > the case of broken ABORT or other unusual PCI cycle handling (missed word > problem). I agree it's a low probability. But we've seen it, just a week > or two ago on a brand new box. > > But then we tend to see things here nobody else sees due to our scale. > > ron At this level, you're basically screwed. A sofware checksum isn't even an option on other PCI users, like disk controllers. If you don't trust your PCI chipset, what do you do about things like that? I'm rather curious -- what was the problematic hardware combination? Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 9: 3:43 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from dirty.research.bell-labs.com (dirty.research.bell-labs.com [204.178.16.6]) by hub.freebsd.org (Postfix) with SMTP id 527AE37B412 for ; Thu, 27 Sep 2001 09:03:39 -0700 (PDT) Received: from scummy.research.bell-labs.com ([135.104.2.10]) by dirty; Thu Sep 27 12:03:59 EDT 2001 Received: from aura.research.bell-labs.com (aura.research.bell-labs.com [135.104.46.10]) by scummy.research.bell-labs.com (8.11.4/8.11.4) with ESMTP id f8RG3Vl93949 for ; Thu, 27 Sep 2001 12:03:31 -0400 (EDT) Received: from research.bell-labs.com (IDENT:sandeepj@sandeepj-pcmh.research.bell-labs.com [135.104.47.90]) by aura.research.bell-labs.com (8.9.1/8.9.1) with ESMTP id MAA24113 for ; Thu, 27 Sep 2001 12:03:31 -0400 (EDT) Message-ID: <3BB34DD2.FC65196A@research.bell-labs.com> Date: Thu, 27 Sep 2001 12:03:30 -0400 From: Sandeep Joshi X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.16-3 i686) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ron, This may be of interest... http://citeseer.nj.nec.com/stone00when.html When The CRC and TCP Checksum Disagree Jonathan Stone, Craig Partridge SIGCOMM -Sandeep On Thu, 27 Sep 2001, Ronald G Minnich wrote: > > I have a question on the checksum offloading. Has anyone measured any > incidence of data corruption between the PCI card and memory. In other > words, when you offload checksums the end-to-end checking becomes > card-to-card checking, and the possibility exists that what goes in memory > at the destination end is not what was sent at the source. Very remote > possibility, of course, but ... > > It's not that the data gets corrupted (usually). It's that > once-in-a-100-trillion errors could result in the occasional dropped > half-packet or missed word (i.e. overflow). The missed word problem is > usual a miscommunication between card and PCI chipset about how a PCI > ABORT is supposed to work ... which we've seen on some very recent > just-released chipset/network card combinations,. > > Does this happen? Yes. We've seen it on, to name just two, HIPPI800 and > Myrinet cards. In each case it was not actual data corruption, it was > "can't happen" DMA scenarios that once in a very long while (1 in 10^14 or > so) resulted in bits of packets getting corrupted. Each of these cards > has a very high-quality end-end CRC for the data, and Myrinet has flow > control. We're not the only place that has seen this problem, and I've > been told that many commerical Myrinet clients run IP over Myrinet because > of these types of problems (of course FreeBSD has the fastest IP over > Myrinet anyway, so it's not like that's a huge problem). > > Is it likely? Well, on one cluster here, with 48 machines and 12 > interfaces per machine, it's not only likely, it's a given. Without > software checksums you are going to get data corruption. > > What I don't know is whether offloaded checksums on commodity ethernet > cards have seen anything similar. > > I assume that checksums across all the frags are done by the kernel (i.e. > NFS would checksum the full UDP packet)? Has anyone measured to see if > there is corruption occuring on the frags, ever? Of course it would > probably take a while ... > > Thanks in advance for any information you might have. > > ron > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 9:11:37 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from acl.lanl.gov (acl.lanl.gov [128.165.147.1]) by hub.freebsd.org (Postfix) with SMTP id EBBDB37B618 for ; Thu, 27 Sep 2001 09:11:34 -0700 (PDT) Received: (qmail 96475 invoked from network); 27 Sep 2001 10:11:34 -0600 Received: from snaresland.acl.lanl.gov (128.165.147.113) by acl.lanl.gov with SMTP; 27 Sep 2001 10:11:34 -0600 Received: (qmail 27330 invoked by uid 3499); 27 Sep 2001 10:11:33 -0600 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 27 Sep 2001 10:11:33 -0600 Date: Thu, 27 Sep 2001 10:11:33 -0600 (MDT) From: Ronald G Minnich X-X-Sender: To: Andrew Gallatin Cc: "Louis A. Mamakos" , Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <15283.19536.410864.339942@grasshopper.cs.duke.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 27 Sep 2001, Andrew Gallatin wrote: > At this level, you're basically screwed. A sofware checksum isn't > even an option on other PCI users, like disk controllers. If you > don't trust your PCI chipset, what do you do about things like that? > > I'm rather curious -- what was the problematic hardware combination? Can't say yet :-( But it is one of the fancy network interfaces that essentially runs an RTOS on the NIC so it can "help you". Actually fancy $5000 network interfaces are in general less reliable than your average garden-variety $2 IDE chip. Partly because they have so much capability. So we don't worry a lot about lossage with IDE. But it's a big problem on expensive, high end, high performance network interfaces. ron To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 9:22:14 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from beppo.feral.com (beppo.feral.com [192.67.166.79]) by hub.freebsd.org (Postfix) with ESMTP id 7B63937B61D for ; Thu, 27 Sep 2001 09:22:09 -0700 (PDT) Received: from wonky.feral.com (wonky.feral.com [192.67.166.7]) by beppo.feral.com (8.11.3/8.11.3) with ESMTP id f8RGLxH64966; Thu, 27 Sep 2001 09:21:59 -0700 (PDT) (envelope-from mjacob@feral.com) Date: Thu, 27 Sep 2001 09:21:29 -0700 (PDT) From: Matthew Jacob Reply-To: To: Sandeep Joshi Cc: Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <3BB34DD2.FC65196A@research.bell-labs.com> Message-ID: <20010927092116.B50870-100000@wonky.feral.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Oh, yeah- I forgot about this. Jonathon is a pretty good NetBSD hacker.. On Thu, 27 Sep 2001, Sandeep Joshi wrote: > > Ron, > > This may be of interest... > > http://citeseer.nj.nec.com/stone00when.html > > When The CRC and TCP Checksum Disagree > Jonathan Stone, Craig Partridge SIGCOMM > > -Sandeep > > On Thu, 27 Sep 2001, Ronald G Minnich wrote: > > > > I have a question on the checksum offloading. Has anyone measured any > > incidence of data corruption between the PCI card and memory. In other > > words, when you offload checksums the end-to-end checking becomes > > card-to-card checking, and the possibility exists that what goes in memory > > at the destination end is not what was sent at the source. Very remote > > possibility, of course, but ... > > > > It's not that the data gets corrupted (usually). It's that > > once-in-a-100-trillion errors could result in the occasional dropped > > half-packet or missed word (i.e. overflow). The missed word problem is > > usual a miscommunication between card and PCI chipset about how a PCI > > ABORT is supposed to work ... which we've seen on some very recent > > just-released chipset/network card combinations,. > > > > Does this happen? Yes. We've seen it on, to name just two, HIPPI800 and > > Myrinet cards. In each case it was not actual data corruption, it was > > "can't happen" DMA scenarios that once in a very long while (1 in 10^14 or > > so) resulted in bits of packets getting corrupted. Each of these cards > > has a very high-quality end-end CRC for the data, and Myrinet has flow > > control. We're not the only place that has seen this problem, and I've > > been told that many commerical Myrinet clients run IP over Myrinet because > > of these types of problems (of course FreeBSD has the fastest IP over > > Myrinet anyway, so it's not like that's a huge problem). > > > > Is it likely? Well, on one cluster here, with 48 machines and 12 > > interfaces per machine, it's not only likely, it's a given. Without > > software checksums you are going to get data corruption. > > > > What I don't know is whether offloaded checksums on commodity ethernet > > cards have seen anything similar. > > > > I assume that checksums across all the frags are done by the kernel (i.e. > > NFS would checksum the full UDP packet)? Has anyone measured to see if > > there is corruption occuring on the frags, ever? Of course it would > > probably take a while ... > > > > Thanks in advance for any information you might have. > > > > ron > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 9:24:50 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 41BB337B426 for ; Thu, 27 Sep 2001 09:24:45 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id MAA14130; Thu, 27 Sep 2001 12:24:36 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8RGOAI83899; Thu, 27 Sep 2001 12:24:10 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15283.21162.173735.343154@grasshopper.cs.duke.edu> Date: Thu, 27 Sep 2001 12:24:10 -0400 (EDT) To: Ronald G Minnich Cc: "Louis A. Mamakos" , Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: References: <15283.19536.410864.339942@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ronald G Minnich writes: > On Thu, 27 Sep 2001, Andrew Gallatin wrote: > > > At this level, you're basically screwed. A sofware checksum isn't > > even an option on other PCI users, like disk controllers. If you > > don't trust your PCI chipset, what do you do about things like that? > > > > I'm rather curious -- what was the problematic hardware combination? > > Can't say yet :-( > > But it is one of the fancy network interfaces that essentially runs an > RTOS on the NIC so it can "help you". Actually fancy $5000 network > interfaces are in general less reliable than your average garden-variety > $2 IDE chip. Partly because they have so much capability. > > So we don't worry a lot about lossage with IDE. But it's a big problem on > expensive, high end, high performance network interfaces. But SCSI isn't immune either. We had some data corruption problems with early adaptec Ultra-2 scsi controllers too, before Justin fixed it by working around it in the driver. Basically, anything that uses a PCI chipset harder or in different ways than its designers expected can end up being a problem. Low volume hardware is somtimes worse, but not always... Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 9:31:27 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from whizzo.transsys.com (whizzo.TransSys.COM [144.202.42.10]) by hub.freebsd.org (Postfix) with ESMTP id 7B93137B42A for ; Thu, 27 Sep 2001 09:31:22 -0700 (PDT) Received: from whizzo.transsys.com (#6@localhost.transsys.com [127.0.0.1]) by whizzo.transsys.com (8.11.4/8.11.4) with ESMTP id f8RGVCZ65964; Thu, 27 Sep 2001 12:31:12 -0400 (EDT) (envelope-from louie@whizzo.transsys.com) Message-Id: <200109271631.f8RGVCZ65964@whizzo.transsys.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Andrew Gallatin Cc: Ronald G Minnich , hackers@FreeBSD.ORG X-Image-URL: http://www.transsys.com/louie/images/louie-mail.jpg From: "Louis A. Mamakos" Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: <200109271416.f8REGaZ64624@whizzo.transsys.com> <15283.14648.430630.163513@grasshopper.cs.duke.edu> In-reply-to: Your message of "Thu, 27 Sep 2001 10:35:36 EDT." <15283.14648.430630.163513@grasshopper.cs.duke.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 27 Sep 2001 12:31:12 -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > Louis A. Mamakos writes: > > The other type of failure you might not catch are software errors; that > > is, where a packet is produced by the network stack and then is > > subsequently stomped on by a random store from some other code. Or > > a mis-programmed I/O card with scatter/gather capability doesn't pick > > up what was intended, etc. The Internet checksum is useful for > > detecting this class of error. > > > > No, you're missing the point almost entirely. The checksum is not > skipped. It is calculated by the DMA engine based on the data that's > transferred across the I/O bus on the receiver (and / or the sender). > If the data is incorrect as seen by the receiving nic, the checksum > will be wrong and the packet will be dropped. I was referring to the case on the transmit side where the wrong data get's gathered up by the DMA engine because of software related errors. You get a valid checksum, but for the wrong data. You might have the wrong data because a drive screwed up setting the DMA descriptors, or some other I/O transfer splatted over the buffer waiting in a transmit queue. > If the packet lands in the wrong place, you have much worse problems. Sure you do; a software checksum at least gives you an opportunity to detect these failures. While these are improbable failures, I know that I've experienced the class I described in my years of hacking on network platforms, and Ron has experienced the one he described. These are not impossible occurances; it's a matter of weighing the additional cost of the software checksum vs. the likelyhood (and cost/impact) of undetected corruption of the data. If you're running IPSEC or SSL up above all this, there's another mechanism to detect these types of failures. I just think back 10 or 15 years at the impact of Sun's decision to not compute UDP checksums on their NFS traffic, because the network adapter had a much stronger Ethernet CRC. It was a trade-off not worth making even then, with CPUs in the single-digit MIPS performance. We ought to at least consider the previous experience. louie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 9:55:30 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id C06F437B50E for ; Thu, 27 Sep 2001 09:55:25 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id MAA14922; Thu, 27 Sep 2001 12:55:21 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8RGste83955; Thu, 27 Sep 2001 12:54:55 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15283.23007.137091.883110@grasshopper.cs.duke.edu> Date: Thu, 27 Sep 2001 12:54:55 -0400 (EDT) To: "Louis A. Mamakos" Cc: hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <200109271631.f8RGVCZ65964@whizzo.transsys.com> References: <200109271416.f8REGaZ64624@whizzo.transsys.com> <15283.14648.430630.163513@grasshopper.cs.duke.edu> <200109271631.f8RGVCZ65964@whizzo.transsys.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Louis A. Mamakos writes: > I was referring to the case on the transmit side where the wrong > data get's gathered up by the DMA engine because of software related > errors. You get a valid checksum, but for the wrong data. You might > have the wrong data because a drive screwed up setting the DMA descriptors, > or some other I/O transfer splatted over the buffer waiting in a > transmit queue. What happens if that same i/o transfer splatted over the buffer waiting in user space prior to the copyin, or sitting in the socket buffer prior to a software checksum being done? Software checksums are not quite the panacea you make them out to be. And they're very expensive. Geez. All I wanted to do was pat Jonathan on the back for coming up with what is apparently the most flexible and well though out mechanism out there. These issues have been argued to death; I don't feel like arguing with you. I'm satisified that I'm not going to convince you & you're not going to convince me. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 10:17:17 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from whizzo.transsys.com (whizzo.TransSys.COM [144.202.42.10]) by hub.freebsd.org (Postfix) with ESMTP id 0155C37B418 for ; Thu, 27 Sep 2001 10:17:11 -0700 (PDT) Received: from whizzo.transsys.com (#6@localhost.transsys.com [127.0.0.1]) by whizzo.transsys.com (8.11.4/8.11.4) with ESMTP id f8RHHBZ66485; Thu, 27 Sep 2001 13:17:11 -0400 (EDT) (envelope-from louie@whizzo.transsys.com) Message-Id: <200109271717.f8RHHBZ66485@whizzo.transsys.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Andrew Gallatin Cc: hackers@FreeBSD.ORG X-Image-URL: http://www.transsys.com/louie/images/louie-mail.jpg From: "Louis A. Mamakos" Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: <200109271416.f8REGaZ64624@whizzo.transsys.com> <15283.14648.430630.163513@grasshopper.cs.duke.edu> <200109271631.f8RGVCZ65964@whizzo.transsys.com> <15283.23007.137091.883110@grasshopper.cs.duke.edu> In-reply-to: Your message of "Thu, 27 Sep 2001 12:54:55 EDT." <15283.23007.137091.883110@grasshopper.cs.duke.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 27 Sep 2001 13:17:11 -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > Louis A. Mamakos writes: > > > I was referring to the case on the transmit side where the wrong > > data get's gathered up by the DMA engine because of software related > > errors. You get a valid checksum, but for the wrong data. You might > > have the wrong data because a drive screwed up setting the DMA descriptors, > > or some other I/O transfer splatted over the buffer waiting in a > > transmit queue. > > What happens if that same i/o transfer splatted over the buffer > waiting in user space prior to the copyin, or sitting in > the socket buffer prior to a software checksum being done? > Software checksums are not quite the panacea you make them out to be. > And they're very expensive. > > Geez. All I wanted to do was pat Jonathan on the back for coming up > with what is apparently the most flexible and well though out > mechanism out there. And I don't disagree with you, it's wonderful work. What I guess I'm trying to get across is that like any tool, it ought to be used properly and in an informed way. For instance, you can mount a file system async or with soft updates, and each of these choices have their own trade-offs. Folks ought to consider the likelyhood of this class of data corruption, unlikely as it is, and weigh it along with the impact on your application, and the differences in performance and loading. louie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 10:32: 2 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 8740237B509 for ; Thu, 27 Sep 2001 10:31:52 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id NAA16015; Thu, 27 Sep 2001 13:31:48 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8RHVLN83997; Thu, 27 Sep 2001 13:31:21 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15283.25193.787035.280848@grasshopper.cs.duke.edu> Date: Thu, 27 Sep 2001 13:31:21 -0400 (EDT) To: "Louis A. Mamakos" Cc: hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <200109271717.f8RHHBZ66485@whizzo.transsys.com> References: <200109271416.f8REGaZ64624@whizzo.transsys.com> <15283.14648.430630.163513@grasshopper.cs.duke.edu> <200109271631.f8RGVCZ65964@whizzo.transsys.com> <15283.23007.137091.883110@grasshopper.cs.duke.edu> <200109271717.f8RHHBZ66485@whizzo.transsys.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Louis A. Mamakos writes: > > Folks ought to consider the likelyhood of this class of data > corruption, unlikely as it is, and weigh it along with the impact on > your application, and the differences in performance and loading. > Agreed. Very well said, by the way.. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 11:54:18 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from acl.lanl.gov (acl.lanl.gov [128.165.147.1]) by hub.freebsd.org (Postfix) with SMTP id C8FB537B40C for ; Thu, 27 Sep 2001 11:54:13 -0700 (PDT) Received: (qmail 110054 invoked from network); 27 Sep 2001 12:54:13 -0600 Received: from snaresland.acl.lanl.gov (128.165.147.113) by acl.lanl.gov with SMTP; 27 Sep 2001 12:54:13 -0600 Received: (qmail 27855 invoked by uid 3499); 27 Sep 2001 12:54:13 -0600 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 27 Sep 2001 12:54:13 -0600 Date: Thu, 27 Sep 2001 12:54:13 -0600 (MDT) From: Ronald G Minnich X-X-Sender: To: Andrew Gallatin Cc: "Louis A. Mamakos" , Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <15283.23007.137091.883110@grasshopper.cs.duke.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 27 Sep 2001, Andrew Gallatin wrote: > Geez. All I wanted to do was pat Jonathan on the back for coming up > with what is apparently the most flexible and well though out > mechanism out there. it's great work. I was mainly curious to see if anyone had measured this kind of problem. Thanks ron To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 11:55:38 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from roaming.cacheboy.net (orb60.arach.net.au [203.30.47.126]) by hub.freebsd.org (Postfix) with ESMTP id 52C2037B40C; Thu, 27 Sep 2001 11:55:28 -0700 (PDT) Received: (from adrian@localhost) by roaming.cacheboy.net (8.11.6/8.11.1) id f8RIxDY04325; Thu, 27 Sep 2001 20:59:13 +0200 (CEST) (envelope-from adrian) Date: Thu, 27 Sep 2001 20:59:12 +0200 From: Adrian Chadd To: ports@freebsd.org Cc: hackers@freebsd.org Subject: closing down the squid22/23 ports? Message-ID: <20010927205912.E4232@roaming.cacheboy.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi all, Pardon the cross-posting. :-) I'd like to look at closing down / making inactive the squid22 and squid23 ports. The squid-2.2 and squid-2.3 codebases have been inactive and largely unsupported by the squid developers (read: myself inclusive here) for some time now, and I'd like to point users at the actively developed/maintained squid branch. Squid-2.5 is also in the pipeline for release soon, and I don't think there is a point in having 4 squid ports. What do people think? (please CC me, I'm currently not on the ports/hackers list for various time-related reasons..) Thanks! Adrian -- Adrian Chadd "Programming is like sex: One mistake and you have to support for a lifetime." -- rec.humor.funny To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 12:54:47 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from welshfantasyfootball.com (host48.tns.co.uk [194.152.91.114]) by hub.freebsd.org (Postfix) with SMTP id 86D6737B40E for ; Thu, 27 Sep 2001 12:54:42 -0700 (PDT) From: "Claire" To: Subject: Cash Prizes Win! Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Date: Thu, 27 Sep 2001 20:52:51 +0100 Content-Transfer-Encoding: 8bit Message-Id: <20010927195442.86D6737B40E@hub.freebsd.org> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG To all our fans!! Welsh Fantasy Football has paid your entrance fee to the WELSH FANTASY FOOTBALL GAME 2001 Go to www.welshfantasyfootball.com you have to be in it to WIN it!! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 12:55: 0 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from welshfantasyfootball.com (host48.tns.co.uk [194.152.91.114]) by hub.freebsd.org (Postfix) with SMTP id AD00537B409 for ; Thu, 27 Sep 2001 12:54:46 -0700 (PDT) From: "Claire" To: Subject: Cash Prizes Win! Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Date: Thu, 27 Sep 2001 20:52:55 +0100 Content-Transfer-Encoding: 8bit Message-Id: <20010927195446.AD00537B409@hub.freebsd.org> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG To all our fans!! Welsh Fantasy Football has paid your entrance fee to the WELSH FANTASY FOOTBALL GAME 2001 Go to www.welshfantasyfootball.com you have to be in it to WIN it!! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 13:12:41 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id AB9D437B411 for ; Thu, 27 Sep 2001 13:12:37 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8RKC2d22173; Thu, 27 Sep 2001 13:12:02 -0700 (PDT) (envelope-from dillon) Date: Thu, 27 Sep 2001 13:12:02 -0700 (PDT) From: Matt Dillon Message-Id: <200109272012.f8RKC2d22173@earth.backplane.com> To: Stephen McKay Cc: Poul-Henning Kamp , hackers@freebsd.org, mckay@thehub.com.au Subject: Re: Conclusions on... was Re: More on the cache_purgeleafdirs() routine References: <200109231040.f8NAeXw86352@earth.backplane.com> <98331.1001250396@critter> <200109271419.f8REJp116422@dungeon.home> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :On Sunday, 23rd September 2001, Poul-Henning Kamp wrote: : :>Things to look out for: :> :>1. !ufs filesystems : :I am irredeemably slack for not testing this a lot but... : :I believe I saw bad interactions between vmiodirenable and isofs on 4.3-R. : :I mounted a CD, looked at stuff on it, did a lot of other work, went back :to the CD and files were screwy (files contained the contents of other :files, files were zero size). I unmounted and remounted the CD and :everything was fine. The machine is a reliable old workhorse, and has :no hardware errors. : :Since then, I've not had a chance to go back and check. It's only because :you are making vmiodirenable the default that I'm mentioning it. Sorry :for not making a proper bug report containing actual facts. :-( : :Stephen. Hmm. Well, if someone can reproduce the problem it sounds like it ought to be easy to track down. I am somewhat skeptical that vmiodirenable could cause that but I suppose it's possible. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 13:22:37 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 51EE637B40B; Thu, 27 Sep 2001 13:22:29 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8RKMMo22244; Thu, 27 Sep 2001 13:22:22 -0700 (PDT) (envelope-from dillon) Date: Thu, 27 Sep 2001 13:22:22 -0700 (PDT) From: Matt Dillon Message-Id: <200109272022.f8RKMMo22244@earth.backplane.com> To: Douglas Swarin Cc: hackers@freebsd.org, tmoestl@gmx.net, bp@freebsd.org Subject: Re: more on Re: Please review: bugfix for vinvalbuf() References: <20010711003926.B8799@crow.dom2ip.de> <200107110643.f6B6hTB24707@earth.backplane.com> <20010926204333.A15865@staff.texas.net> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I totally forgot about that one. Your fix looks good, I'll start testing it. The bigger picture here is that vinvalbuf() is not typically called while a vnode is still active. NFS calls it on active vnodes in order to invalidate the cache when the client detects that the file mtime has been changed by someone else (ugly ugly ugly). So this sort of crash can occur if one client is mmap()ing a file another another client (or the server) writes to the file. -Matt :I recently mentioned on freebsd-stable in message : : <20010923124824.A16392@staff.texas.net> : :a recurring rslock panic which I believe has been caused by the below :mentioned problem in vinvalbuf(). I have worked up a patch for -STABLE :(which should also apply to -CURRENT if there have not been major changes :to this code). The patch is appended to this message for review. : :Data from the crash dump revealed the following: : :SMP 2 cpus :IdlePTD 3555328 :initial pcb at 2cf280 :panicstr: rslock: cpu: 0, addr: 0xd7be66ec, lock: 0x00000001 :panic messages: :--- :panic: rslock: cpu: 0, addr: 0xd7be66ec, lock: 0x00000001 :mp_lock = 00000001; cpuid = 0; lapic.id = 01000000 :boot() called on cpu#0 : :--- : :#0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:473 :#1 0xc016cf8f in boot (howto=256) at /usr/src/sys/kern/kern_shutdown.c:313 :#2 0xc016d3a9 in panic (fmt=0xc025bcce "rslock: cpu: %d, addr: 0x%08x, lock: 0x%08x") : at /usr/src/sys/kern/kern_shutdown.c:581 :#3 0xc025bcce in bsl1 () :#4 0xc021eeac in _unlock_things (fs=0xd7a6dec4, dealloc=0) : at /usr/src/sys/vm/vm_fault.c:147 :#5 0xc021f8c7 in vm_fault (map=0xd7a6bf40, vaddr=673968128, fault_type=1 '\001', : fault_flags=0) at /usr/src/sys/vm/vm_fault.c:826 :#6 0xc025d016 in trap_pfault (frame=0xd7a6dfa8, usermode=1, eva=673972223) : at /usr/src/sys/i386/i386/trap.c:829 :#7 0xc025ca8b in trap (frame={tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 99049, : tf_esi = 0, tf_ebp = -1077937884, tf_isp = -676929580, tf_ebx = 48110729, : tf_edx = 0, tf_ecx = 1835007, tf_eax = 672137216, tf_trapno = 12, tf_err = 4, : tf_eip = 134517190, tf_cs = 31, tf_eflags = 66070, tf_esp = -1077937940, : tf_ss = 47}) : at /usr/src/sys/i386/i386/trap.c:359 :#8 0x80491c6 in ?? () :#9 0x8048d9e in ?? () :#10 0x804a078 in ?? () :#11 0x8048b45 in ?? () : :--- : :A quick review of processes revealed a process stuck in vmopar: : :(kgdb) ps :... :46479 d7ffc560 d806e000 235886 1 46394 004006 3 tail vmopar c09120c8 :... : :which was sleeping in vm_object_page_remove() in vinvalbuf(): : :(kgdb) btp 46479 : frame 0 at 0xd806fc8c: ebp d806fcb8, eip 0xc0170051 : mov 0x141(%ebx),%al : frame 1 at 0xd806fcb8: ebp d806fce0, eip 0xc02262e8 : add $0x10,%esp : frame 2 at 0xd806fce0: ebp d806fd2c, eip 0xc019a667 : add $0x10,%esp : frame 3 at 0xd806fd2c: ebp d806fd60, eip 0xc01d0aa0 : add $0x18,%esp : frame 4 at 0xd806fd60: ebp d806fe2c, eip 0xc01cf5d8 : mov %eax,0xffffff74(%ebp) : frame 5 at 0xd806fe2c: ebp d806fe44, eip 0xc01f6842 : jmp 0xc01f6849 : frame 6 at 0xd806fe44: ebp d806fe78, eip 0xc01a22cc : mov %eax,0xffffffe8(%ebp) : frame 7 at 0xd806fe78: ebp d806fef4, eip 0xc017b690 : mov %eax,%esi : frame 8 at 0xd806fef4: ebp d806ff28, eip 0xc017b556 : mov %eax,%esi : frame 9 at 0xd806ff28: ebp d806ffa0, eip 0xc025d745 : mov %eax,0xffffffb8(%ebp) : : :The patch is below, against vfs_subr.c 1.249.2.11 2001/09/11 : :--- vfs_subr.c Tue Sep 11 04:49:53 2001 :+++ vfs_subr.c.new Wed Sep 26 15:23:09 2001 :@@ -721,9 +721,9 @@ : } : } : :- while (vp->v_numoutput > 0) { :- vp->v_flag |= VBWAIT; :- tsleep(&vp->v_numoutput, PVM, "vnvlbv", 0); :+ if (VOP_GETVOBJECT(vp, &object) == 0) { :+ while (object->paging_in_progress) :+ vm_object_pip_sleep(object, "vnvlbv"); : } : : splx(s); To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 13:53:38 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from tsunami.acidpit.org (tsunami.acidpit.org [206.190.163.234]) by hub.freebsd.org (Postfix) with ESMTP id B7FAA37B40A for ; Thu, 27 Sep 2001 13:53:35 -0700 (PDT) Received: (from rch@localhost) by tsunami.acidpit.org (8.11.3/8.11.3) id f8RKrYd23767 for freebsd-hackers@freebsd.org; Thu, 27 Sep 2001 16:53:34 -0400 (EDT) (envelope-from rch@acidpit.org) Date: Thu, 27 Sep 2001 16:53:34 -0400 From: Robert Hough To: freebsd-hackers@freebsd.org Subject: ng_bridge Message-ID: <20010927165304.C23689@acidpit.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Yesterday, I used a script on my bridge that pretty much brought down the entire network segment it was connected to. The script in question, was found in /usr/share/examples/netgraph -- called 'ether.bridge'. I modified the script, and ran it -- boom. Problems galore! At the time, the bridge did *not* function correctly. I was using a couple of cards that didn't support bridging. I have since corrected this, and on a smaller network repeated the same steps. This time, everything is working correctly. My question, is really this. Does anyone have an idea as to why this caused so much trouble to begin with, and how can I protect my network from allowing this to happen again? -- Robert Hough (rch@acidpit.org) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 14:44:49 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 1BE3537B407 for ; Thu, 27 Sep 2001 14:44:43 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA67004; Thu, 27 Sep 2001 15:39:45 -0700 (PDT) Date: Thu, 27 Sep 2001 15:39:45 -0700 (PDT) From: Julian Elischer To: Robert Hough Cc: freebsd-hackers@freebsd.org Subject: Re: ng_bridge In-Reply-To: <20010927165304.C23689@acidpit.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG well, maybe if you told us what you modified, and what happenned..... On Thu, 27 Sep 2001, Robert Hough wrote: > Yesterday, I used a script on my bridge that pretty much brought down > the entire network segment it was connected to. The script in question, > was found in /usr/share/examples/netgraph -- called 'ether.bridge'. I > modified the script, and ran it -- boom. Problems galore! > > At the time, the bridge did *not* function correctly. I was using a > couple of cards that didn't support bridging. I have since corrected > this, and on a smaller network repeated the same steps. This time, > everything is working correctly. > > My question, is really this. Does anyone have an idea as to why this > caused so much trouble to begin with, and how can I protect my network > from allowing this to happen again? > > -- > Robert Hough (rch@acidpit.org) > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 15: 2:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from tsunami.acidpit.org (tsunami.acidpit.org [206.190.163.234]) by hub.freebsd.org (Postfix) with ESMTP id B909B37B401 for ; Thu, 27 Sep 2001 15:02:06 -0700 (PDT) Received: (from rch@localhost) by tsunami.acidpit.org (8.11.3/8.11.3) id f8RM23E23975; Thu, 27 Sep 2001 18:02:03 -0400 (EDT) (envelope-from rch@acidpit.org) Date: Thu, 27 Sep 2001 18:02:03 -0400 From: Robert Hough To: Julian Elischer Cc: freebsd-hackers@freebsd.org Subject: Re: ng_bridge Message-ID: <20010927180203.A23934@acidpit.org> References: <20010927165304.C23689@acidpit.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from julian@elischer.org on Thu, Sep 27, 2001 at 15:39:45 -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Sep 27, 2001, Julian Elischer wrote: > well, maybe if you told us what you modified, and what happenned..... $ diff /usr/share/examples/netgraph/ether.bridge ~/eth_bridge.sh 41,42c41,42 < BRIDGE_IFACES="ed0 fxp0 fxp1" < LOCAL_IFACE="fxp0" --- > BRIDGE_IFACES="vx0 vx1" > LOCAL_IFACE="" As far as what happened, it basically made everything connected to that hub unreachable by everything else. Other settings are to follow: # kernel config options BRIDGE options DUMMYNET options IPFIREWALL options IPFIREWALL_DEFAULT_TO_ACCEPT # /etc/sysctl.conf net.link.ether.bridge=1 net.link.ether.bridge_ipfw=1 No interface was configured with an IP address on the box at the time. The only ipfirewall rule in use was the default_accept. Thanks. -- Robert Hough (rch@acidpit.org) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 15:24:34 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id E2BB737B405 for ; Thu, 27 Sep 2001 15:24:29 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id QAA67194; Thu, 27 Sep 2001 16:07:20 -0700 (PDT) Date: Thu, 27 Sep 2001 16:07:19 -0700 (PDT) From: Julian Elischer To: Robert Hough Cc: freebsd-hackers@freebsd.org Subject: Re: ng_bridge In-Reply-To: <20010927180203.A23934@acidpit.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG BRIDGE/DUMMYNET/net.link.ether.bridge=1/net.link.ether.bridge_ipfw=1 is one set of bridging code ng_bridge is a completely separate (in my opinion, better, but I'm biased) setr of code. they might interract if you turn them both on at the same time.... On Thu, 27 Sep 2001, Robert Hough wrote: > On Thu, Sep 27, 2001, Julian Elischer wrote: > > > well, maybe if you told us what you modified, and what happenned..... > > $ diff /usr/share/examples/netgraph/ether.bridge ~/eth_bridge.sh > 41,42c41,42 > < BRIDGE_IFACES="ed0 fxp0 fxp1" > < LOCAL_IFACE="fxp0" > --- > > BRIDGE_IFACES="vx0 vx1" > > LOCAL_IFACE="" > > As far as what happened, it basically made everything connected to that > hub unreachable by everything else. Other settings are to follow: > > > # kernel config > options BRIDGE > options DUMMYNET > options IPFIREWALL > options IPFIREWALL_DEFAULT_TO_ACCEPT > > # /etc/sysctl.conf > net.link.ether.bridge=1 > net.link.ether.bridge_ipfw=1 > > No interface was configured with an IP address on the box at the time. > The only ipfirewall rule in use was the default_accept. Thanks. > > -- > Robert Hough (rch@acidpit.org) > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 18:33:56 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from gandalf.vi.bravenet.com (gandalf.bravenet.com [139.142.105.50]) by hub.freebsd.org (Postfix) with SMTP id 8D06537B40B for ; Thu, 27 Sep 2001 18:33:50 -0700 (PDT) Received: (qmail 10148 invoked by uid 1001); 28 Sep 2001 01:44:58 -0000 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 28 Sep 2001 01:44:58 -0000 Date: Thu, 27 Sep 2001 18:44:58 -0700 (PDT) From: Dan To: Subject: power supplies Message-ID: <20010927184223.J1885-100000@gandalf.bravenet.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I had the stangest situation today where a new nic card was put into a machine and then the machine did not start up. Placed the old nic card back in the box and it still did not start up. Switched power supplies with an exactly equal box and both machine booted up fine. This has happened twice since we started replacing nic cards today with ones with more buffer space available on them out of about 8 machines now. Does this make any sense to anyone? -- Dan +------------------------------------------------------+ | BRAVENET WEB SERVICES | | dan@bravenet.com | | screen;cd /usr/src;make buildworld;cd ~ | | cp MYKERNEL /sys/i386/conf;cd /usr/src | | make buildkernel KERNCONF=MYKERNEL | |make installkernel KERNCONF=MYKERNEL;make installworld| +______________________________________________________+ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 18:46:30 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from cain.gsoft.com.au (genesi.lnk.telstra.net [139.130.136.161]) by hub.freebsd.org (Postfix) with ESMTP id 8744637B40A for ; Thu, 27 Sep 2001 18:46:19 -0700 (PDT) Received: from cain.gsoft.com.au (root@spare0.gsoft.com.au [203.38.152.114]) by cain.gsoft.com.au (8.8.8/8.8.8) with ESMTP id LAA07358; Fri, 28 Sep 2001 11:16:13 +0930 (CST) (envelope-from doconnor@gsoft.com.au) Message-ID: X-Mailer: XFMail 1.5.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20010927184223.J1885-100000@gandalf.bravenet.com> Date: Fri, 28 Sep 2001 11:16:12 +0930 (CST) From: "Daniel O'Connor" To: Dan Subject: RE: power supplies Cc: freebsd-hackers@freebsd.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 28-Sep-2001 Dan wrote: > > I had the stangest situation today where a new nic card was put into a > machine and then the machine did not start up. Placed the old nic card > back in the box and it still did not start up. Switched power supplies > with an exactly equal box and both machine booted up fine. This has > happened twice since we started replacing nic cards today with ones with > more buffer space available on them out of about 8 machines now. > > Does this make any sense to anyone? The PSU might have just been on the border of delivering enough current.. I have found slightly over spec'ing the PSU is a good idea when using a UPS, because if another device powered by the UPS draws a big load the computer tends to reset :( (250w vs 300w) --- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 19:39: 6 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from rutger.owt.com (rutger.owt.com [204.118.6.16]) by hub.freebsd.org (Postfix) with ESMTP id 3F82C37B406 for ; Thu, 27 Sep 2001 19:39:02 -0700 (PDT) Received: from owt.com (owt-207-41-94-232.owt.com [207.41.94.232]) by rutger.owt.com (8.9.3/8.9.3) with ESMTP id TAA03666; Thu, 27 Sep 2001 19:38:58 -0700 Message-ID: <3BB3E2BF.648840A7@owt.com> Date: Thu, 27 Sep 2001 19:38:55 -0700 From: Kent Stewart Reply-To: kbstew99@hotmail.com Organization: One World Telecommunications X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Dan Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: power supplies References: <20010927184223.J1885-100000@gandalf.bravenet.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan wrote: > > I had the stangest situation today where a new nic card was put into a > machine and then the machine did not start up. Placed the old nic card > back in the box and it still did not start up. Switched power supplies > with an exactly equal box and both machine booted up fine. This has > happened twice since we started replacing nic cards today with ones with > more buffer space available on them out of about 8 machines now. > > Does this make any sense to anyone? There are problems with PSes when you use NICs with wake up capability. The NIC may exceed the capability of one of your low amperage voltages. Kent > > -- > Dan > > +------------------------------------------------------+ > | BRAVENET WEB SERVICES | > | dan@bravenet.com | > | screen;cd /usr/src;make buildworld;cd ~ | > | cp MYKERNEL /sys/i386/conf;cd /usr/src | > | make buildkernel KERNCONF=MYKERNEL | > |make installkernel KERNCONF=MYKERNEL;make installworld| > +______________________________________________________+ > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message -- Kent Stewart Richland, WA mailto:kbstew99@hotmail.com http://kstewart.urx.com/kstewart/index.html FreeBSD News http://daily.daemonnews.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 19:53:41 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from gandalf.vi.bravenet.com (gandalf.bravenet.com [139.142.105.50]) by hub.freebsd.org (Postfix) with SMTP id C719737B409 for ; Thu, 27 Sep 2001 19:53:32 -0700 (PDT) Received: (qmail 12317 invoked by uid 1001); 28 Sep 2001 03:04:40 -0000 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 28 Sep 2001 03:04:40 -0000 Date: Thu, 27 Sep 2001 20:04:40 -0700 (PDT) From: Dan To: Cc: Subject: Re: power supplies In-Reply-To: <3BB3E2BF.648840A7@owt.com> Message-ID: <20010927200213.U1885-100000@gandalf.bravenet.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ya but even putting the old nic back in the machine does not still boot up. I don't think this has to do with the nic but you never know. fxp1: On Thu, 27 Sep 2001, Kent Stewart wrote: > Date: Thu, 27 Sep 2001 19:38:55 -0700 > From: Kent Stewart > Reply-To: kbstew99@hotmail.com > To: Dan > Cc: freebsd-hackers@FreeBSD.ORG > Subject: Re: power supplies > > > > Dan wrote: > > > > I had the stangest situation today where a new nic card was put into a > > machine and then the machine did not start up. Placed the old nic card > > back in the box and it still did not start up. Switched power supplies > > with an exactly equal box and both machine booted up fine. This has > > happened twice since we started replacing nic cards today with ones with > > more buffer space available on them out of about 8 machines now. > > > > Does this make any sense to anyone? > > There are problems with PSes when you use NICs with wake up > capability. The NIC may exceed the capability of one of your low > amperage voltages. > > Kent > > > > > -- > > Dan > > > > +------------------------------------------------------+ > > | BRAVENET WEB SERVICES | > > | dan@bravenet.com | > > | screen;cd /usr/src;make buildworld;cd ~ | > > | cp MYKERNEL /sys/i386/conf;cd /usr/src | > > | make buildkernel KERNCONF=MYKERNEL | > > |make installkernel KERNCONF=MYKERNEL;make installworld| > > +______________________________________________________+ > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-hackers" in the body of the message > > -- > Kent Stewart > Richland, WA > > mailto:kbstew99@hotmail.com > http://kstewart.urx.com/kstewart/index.html > FreeBSD News http://daily.daemonnews.org/ > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 19:59:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from obsecurity.dyndns.org (adsl-63-207-60-153.dsl.lsan03.pacbell.net [63.207.60.153]) by hub.freebsd.org (Postfix) with ESMTP id 52A9237B40C for ; Thu, 27 Sep 2001 19:59:06 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id D1DF466D46; Thu, 27 Sep 2001 19:59:05 -0700 (PDT) Date: Thu, 27 Sep 2001 19:59:05 -0700 From: Kris Kennaway To: Dan Cc: kbstew99@hotmail.com, freebsd-hackers@FreeBSD.ORG Subject: Re: power supplies Message-ID: <20010927195905.C4627@xor.obsecurity.org> References: <3BB3E2BF.648840A7@owt.com> <20010927200213.U1885-100000@gandalf.bravenet.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="c3bfwLpm8qysLVxt" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010927200213.U1885-100000@gandalf.bravenet.com>; from dphoenix@bravenet.com on Thu, Sep 27, 2001 at 08:04:40PM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --c3bfwLpm8qysLVxt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Sep 27, 2001 at 08:04:40PM -0700, Dan wrote: >=20 > ya but even putting the old nic back in the machine does not still boot > up. I don't think this has to do with the nic but you never know. > fxp1: You overloaded and burned out the power supply? Kris --c3bfwLpm8qysLVxt Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7s+d5Wry0BWjoQKURAqmwAKC/ydUmvvNIEtcN4l8CEpkpOzHtXgCfSr2A 6ivtqN81CCJZXsicW6VN8ik= =0tSA -----END PGP SIGNATURE----- --c3bfwLpm8qysLVxt-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 20:23: 8 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from gandalf.vi.bravenet.com (gandalf.bravenet.com [139.142.105.50]) by hub.freebsd.org (Postfix) with SMTP id C703237B40D for ; Thu, 27 Sep 2001 20:23:05 -0700 (PDT) Received: (qmail 13361 invoked by uid 1001); 28 Sep 2001 03:34:13 -0000 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 28 Sep 2001 03:34:13 -0000 Date: Thu, 27 Sep 2001 20:34:13 -0700 (PDT) From: Dan To: Kris Kennaway Cc: , Subject: Re: power supplies In-Reply-To: <20010927195905.C4627@xor.obsecurity.org> Message-ID: <20010927203346.E1885-100000@gandalf.bravenet.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG no, it worked when i put it in a different machine that was exactly the same. On Thu, 27 Sep 2001, Kris Kennaway wrote: > Date: Thu, 27 Sep 2001 19:59:05 -0700 > From: Kris Kennaway > To: Dan > Cc: kbstew99@hotmail.com, freebsd-hackers@FreeBSD.ORG > Subject: Re: power supplies > > On Thu, Sep 27, 2001 at 08:04:40PM -0700, Dan wrote: > > > > ya but even putting the old nic back in the machine does not still boot > > up. I don't think this has to do with the nic but you never know. > > fxp1: > > You overloaded and burned out the power supply? > > Kris > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 20:40:57 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from whizzo.transsys.com (whizzo.TransSys.COM [144.202.42.10]) by hub.freebsd.org (Postfix) with ESMTP id 8092237B401 for ; Thu, 27 Sep 2001 20:40:55 -0700 (PDT) Received: from whizzo.transsys.com (#6@localhost.transsys.com [127.0.0.1]) by whizzo.transsys.com (8.11.4/8.11.4) with ESMTP id f8S3esZ71606; Thu, 27 Sep 2001 23:40:54 -0400 (EDT) (envelope-from louie@whizzo.transsys.com) Message-Id: <200109280340.f8S3esZ71606@whizzo.transsys.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Ronald G Minnich Cc: Andrew Gallatin , hackers@FreeBSD.ORG X-Image-URL: http://www.transsys.com/louie/images/louie-mail.jpg From: "Louis A. Mamakos" Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: In-reply-to: Your message of "Thu, 27 Sep 2001 12:54:13 MDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 27 Sep 2001 23:40:54 -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > On Thu, 27 Sep 2001, Andrew Gallatin wrote: > > > Geez. All I wanted to do was pat Jonathan on the back for coming up > > with what is apparently the most flexible and well though out > > mechanism out there. > > it's great work. I was mainly curious to see if anyone had measured this > kind of problem. > > Thanks The paper that someone mentioned earlier in this thread had some statistics on various classes of errors. In a nutshell, they put packet sniffers on 4 different networks, and collected traffic. For each back packet (where the checksum and ethernet CRC differed), they then looked for retransmissions of the same data, and tried to characterize the different failure modes they observed. It's very interesting reading. louie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 22: 7:12 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by hub.freebsd.org (Postfix) with ESMTP id 94E4C37B40E for ; Thu, 27 Sep 2001 22:07:07 -0700 (PDT) Received: (from dan@localhost) by dan.emsphone.com (8.11.4/8.11.4) id f8S56qo15073; Fri, 28 Sep 2001 00:06:52 -0500 (CDT) (envelope-from dan) Date: Fri, 28 Sep 2001 00:06:52 -0500 From: Dan Nelson To: "Louis A. Mamakos" Cc: Andrew Gallatin , hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 Message-ID: <20010928000651.C6178@dan.emsphone.com> References: <200109271416.f8REGaZ64624@whizzo.transsys.com> <15283.14648.430630.163513@grasshopper.cs.duke.edu> <200109271631.f8RGVCZ65964@whizzo.transsys.com> <15283.23007.137091.883110@grasshopper.cs.duke.edu> <200109271717.f8RHHBZ66485@whizzo.transsys.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200109271717.f8RHHBZ66485@whizzo.transsys.com> User-Agent: Mutt/1.3.22.1i X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Sep 27), Louis A. Mamakos said: > And I don't disagree with you, it's wonderful work. > > What I guess I'm trying to get across is that like any tool, it ought to > be used properly and in an informed way. For instance, you can mount a > file system async or with soft updates, and each of these choices have > their own trade-offs. > > Folks ought to consider the likelyhood of this class of data > corruption, unlikely as it is, and weigh it along with the impact on > your application, and the differences in performance and loading. Something to do would be to enable hardware checksumming on 1/2 your machines, and compare the bad packet counts at reported by netstat on the unchanged machines for (say) a 1-month period before and after the change. That should tell you whether you're gaining or losing reliability. It'll be really easy for me, as my current (software cksum) stats show no errors at all: 11:41PM up 11 days, 11:17, 18 users, load averages: 0.08, 0.04, 0.01 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll ti0 1500 00:02:e3:00:17:00 510130554 0 624290928 0 0 tcp: 300848337 packets received 0 discarded for bad checksums udp: 127972686 datagrams received 0 with bad checksum ip: 555526765 total packets received 0 bad header checksums Each counter has probably rolled over at least 5 times (I have to ask, why aren't these 64 bit counters?) -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 22:24:28 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 09D0637B40C for ; Thu, 27 Sep 2001 22:24:25 -0700 (PDT) Received: from elischer.org (InterJet.elischer.org [192.168.1.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id XAA68820; Thu, 27 Sep 2001 23:16:32 -0700 (PDT) Message-ID: <3BB4088E.7B65FDB5@elischer.org> Date: Thu, 27 Sep 2001 22:20:17 -0700 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Dan Nelson Cc: "Louis A. Mamakos" , Andrew Gallatin , hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: <200109271416.f8REGaZ64624@whizzo.transsys.com> <15283.14648.430630.163513@grasshopper.cs.duke.edu> <200109271631.f8RGVCZ65964@whizzo.transsys.com> <15283.23007.137091.883110@grasshopper.cs.duke.edu> <200109271717.f8RHHBZ66485@whizzo.transsys.com> <20010928000651.C6178@dan.emsphone.com> Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Nelson wrote: > > Something to do would be to enable hardware checksumming on 1/2 your > machines, and compare the bad packet counts at reported by netstat on > the unchanged machines for (say) a 1-month period before and after the > change. That should tell you whether you're gaining or losing > reliability. It'll be really easy for me, as my current (software > cksum) stats show no errors at all: the trouble is that if you are trying to find errors between RAM and WIRE then the checksums will be correct regardless of whether there is an error.. so the counts you are checking would be no use.. you need to know what the correct packet contents should be.. > -- +------------------------------------+ ______ _ __ | __--_|\ Julian Elischer | \ U \/ / hard at work in | / \ julian@elischer.org +------>x USA \ a very strange | ( OZ ) \___ ___ | country ! +- X_.---._/ presently in San Francisco \_/ \\ v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 22:39:27 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from smtp016.mail.yahoo.com (smtp016.mail.yahoo.com [216.136.174.113]) by hub.freebsd.org (Postfix) with SMTP id 216D737B410 for ; Thu, 27 Sep 2001 22:39:24 -0700 (PDT) Received: from mkc-65-30-96-67.kc.rr.com (HELO yahoo.com) (65.30.96.67) by smtp.mail.vip.sc5.yahoo.com with SMTP; 28 Sep 2001 05:39:23 -0000 X-Apparently-From: Message-ID: <3BB40D0B.7000008@yahoo.com> Date: Fri, 28 Sep 2001 00:39:23 -0500 From: Jim Bryant Reply-To: kc5vdj@yahoo.com User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1 X-Accept-Language: en-us MIME-Version: 1.0 To: kbstew99@hotmail.com Cc: Dan , freebsd-hackers@FreeBSD.ORG Subject: Re: power supplies References: <20010927184223.J1885-100000@gandalf.bravenet.com> <3BB3E2BF.648840A7@owt.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Kent Stewart wrote: > There are problems with PSes when you use NICs with wake up > capability. The NIC may exceed the capability of one of your low > amperage voltages. > > Kent How much current can wake-on-LAN take? I wouldn't think it would be enough to overload a power supply unless it overloaded the -12V line which is usually only rated in the mils. What is the average 12V rating? 5-8 amps? and even more for 5V? I forget what the average -5V line is rated. jim -- ET has one helluva sense of humor! He's always anal-probing right-wing schizos! ----------------------------------------------------- POWER TO THE PEOPLE! ----------------------------------------------------- "Religious fundamentalism is the biggest threat to international security that exists today." United Nations Secretary General B.B.Ghali _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 23:21:48 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from milliways.chance.ru (milliways.chance.ru [195.190.107.35]) by hub.freebsd.org (Postfix) with ESMTP id 6ECC037B40C for ; Thu, 27 Sep 2001 23:21:44 -0700 (PDT) Received: from do-labs.spb.ru (ppp-5.chance.ru [195.190.107.8]) by milliways.chance.ru (8.9.0/8.9.0) with SMTP id KAA08474 for ; Fri, 28 Sep 2001 10:21:35 +0400 (MSD) Received: (qmail 550 invoked by uid 1000); 27 Sep 2001 23:46:24 -0000 Date: Thu, 27 Sep 2001 23:46:24 +0000 From: Vladimir Dozen To: hackers@FreeBSD.org Subject: calling open() from inside kernel Message-ID: <20010927234624.A403@eix.do-labs.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ehlo. I'm creating a patch to kernel that requires to create a set of files; names of files are generated inside kernel, i.e., strings belong to kernel address space. Initially, I tried to use open(), but failed with EFAULT: open() expects filename string is in userspace, and passes UIO_USERSPACE to NDINIT. Well, I copied a portion of code from kern/vfs_syscalls, and it works fine. But, the length and complexity of the code is too far beyond I could expect from such a basic operation as file opening, and all this just because single string is in wrong space. So, is there any way to call open() in simple way? Something like remapping string into curproc space, or telling open() that string is not in userspace, or smth else? Or, may be, I do something completely wrong? I'm new in kernel programming. -- dozen @ home To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 23:28:42 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from rutger.owt.com (rutger.owt.com [204.118.6.16]) by hub.freebsd.org (Postfix) with ESMTP id 1F42837B401 for ; Thu, 27 Sep 2001 23:28:39 -0700 (PDT) Received: from oneworld.owt.com (oneworld.owt.com [204.118.6.2]) by rutger.owt.com (8.9.3/8.9.3) with ESMTP id XAA14515; Thu, 27 Sep 2001 23:28:37 -0700 Received: from owt.com (owt-207-41-94-232.owt.com [207.41.94.232]) by oneworld.owt.com (8.11.4/8.11.4) with ESMTP id f8S6Sat26904; Thu, 27 Sep 2001 23:28:36 -0700 Message-ID: <3BB41892.7718B5E4@owt.com> Date: Thu, 27 Sep 2001 23:28:34 -0700 From: Kent Stewart Reply-To: kbstew99@hotmail.com X-Mailer: Mozilla 4.77 [en] (Windows NT 5.0; U) X-Accept-Language: en,pdf MIME-Version: 1.0 To: kc5vdj@yahoo.com Cc: kbstew99@hotmail.com, Dan , freebsd-hackers@FreeBSD.ORG Subject: Re: power supplies References: <20010927184223.J1885-100000@gandalf.bravenet.com> <3BB3E2BF.648840A7@owt.com> <3BB40D0B.7000008@yahoo.com> Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jim Bryant wrote: > > Kent Stewart wrote: > > > There are problems with PSes when you use NICs with wake up > > capability. The NIC may exceed the capability of one of your low > > amperage voltages. > > > > Kent > > How much current can wake-on-LAN take? I wouldn't think it would be enough to overload a power supply unless it overloaded the -12V > line which is usually only rated in the mils. What is the average 12V rating? 5-8 amps? and even more for 5V? I forget what the > average -5V line is rated. I don't remember which one it is. The -12v sounds likely. It was the one in the milliamp range. The NICs came with a warning that the typical AT power supplies were insufficient. I haven't bought one in a box for quite a while and that was where I saw the warning. Kent > > jim > -- > ET has one helluva sense of humor! > He's always anal-probing right-wing schizos! > ----------------------------------------------------- > POWER TO THE PEOPLE! > ----------------------------------------------------- > "Religious fundamentalism is the biggest threat to > international security that exists today." > United Nations Secretary General B.B.Ghali > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message -- Kent Stewart Richland, WA mailto:kbstew99@hotmail.com Carl Sagan quote on Seti@home http://setiathome.ssl.berkeley.edu/pale_blue_dot.html It is hard to believe you are soaring with Eagles (las águilas) when you accept SPAM like a mouse (el ratón). To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 23:32:45 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id F27ED37B412 for ; Thu, 27 Sep 2001 23:32:32 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.4/8.11.4) with ESMTP id f8S6WIv69919; Fri, 28 Sep 2001 08:32:19 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: "Louis A. Mamakos" Cc: Ronald G Minnich , Andrew Gallatin , hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: Your message of "Thu, 27 Sep 2001 23:40:54 EDT." <200109280340.f8S3esZ71606@whizzo.transsys.com> Date: Fri, 28 Sep 2001 08:32:18 +0200 Message-ID: <69917.1001658738@critter> From: Poul-Henning Kamp Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200109280340.f8S3esZ71606@whizzo.transsys.com>, "Louis A. Mamakos" writes: >The paper that someone mentioned earlier in this thread had some >statistics on various classes of errors. In a nutshell, they put >packet sniffers on 4 different networks, and collected traffic. For >each back packet (where the checksum and ethernet CRC differed), they >then looked for retransmissions of the same data, and tried to characterize >the different failure modes they observed. > >It's very interesting reading. Absolutely. We have a pretty big FreeBSD concentration at one of my customers and I was actually considering running their collector (if I can get my hands on it) just to see what the error rate is... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Sep 27 23:38:32 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from freebsd.dk (fw-rl0.freebsd.dk [212.242.86.114]) by hub.freebsd.org (Postfix) with ESMTP id B8D0F37B40A for ; Thu, 27 Sep 2001 23:38:28 -0700 (PDT) Received: (from sos@localhost) by freebsd.dk (8.11.3/8.11.3) id f8S6cKI40211; Fri, 28 Sep 2001 08:38:20 +0200 (CEST) (envelope-from sos) From: Søren Schmidt Message-Id: <200109280638.f8S6cKI40211@freebsd.dk> Subject: Re: power supplies In-Reply-To: <3BB40D0B.7000008@yahoo.com> "from Jim Bryant at Sep 28, 2001 00:39:23 am" To: kc5vdj@yahoo.com Date: Fri, 28 Sep 2001 08:38:20 +0200 (CEST) Cc: kbstew99@hotmail.com, Dan , freebsd-hackers@FreeBSD.ORG Reply-To: sos@freebsd.dk X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG It seems Jim Bryant wrote: > Kent Stewart wrote: > > > There are problems with PSes when you use NICs with wake up > > capability. The NIC may exceed the capability of one of your low > > amperage voltages. > > How much current can wake-on-LAN take? I wouldn't think it would be enough to overload a power supply unless it overloaded the -12V > line which is usually only rated in the mils. What is the average 12V rating? 5-8 amps? and even more for 5V? I forget what the > average -5V line is rated. The problem is the +5 standby supply. In a modern ATX evironment lots of stuff pull power on that rail when the machine is switched off, and most ATX supplies only can delive .5-.7 amps here which can easily be overwhelmed. BTW you all know that you ATX machine use about 25W even when switched off right ? This gets into real money if you have a handfull of machines, and you get absolutly nothing for it :) -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 1: 1: 0 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from snipe.mail.pas.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by hub.freebsd.org (Postfix) with ESMTP id 1728337B403 for ; Fri, 28 Sep 2001 01:00:57 -0700 (PDT) Received: from mindspring.com (dialup-209.245.138.226.Dial1.SanJose1.Level3.net [209.245.138.226]) by snipe.mail.pas.earthlink.net (8.11.5/8.9.3) with ESMTP id f8S80VC07214; Fri, 28 Sep 2001 01:00:35 -0700 (PDT) Message-ID: <3BB42E50.B32F5E95@mindspring.com> Date: Fri, 28 Sep 2001 01:01:20 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Lemon Cc: the_srinivas@hotmail.com, hackers@freebsd.org Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jonathan Lemon wrote: > >I'm trying to use the TCP&IP checksum offload capability of the Netgear > >GA620 NIC from a SMP FreeBSD 4.2R system running on a typical PIII SBC. > >I did enable TCP&IP cksum offload for receive operations by setting the > >if_hwassist flag in the driver /sys/pci/if_ti.c and verified that it is > >working. However I'm unable to offload TCP&IP checksum for the send > >operations. > > All you need to do is uncomment the TI_CSUM_FEATURES flag at the top > of the file, and recompile. He didn't say his packet size, either. To the original poster: if you are sending jumbograms, the buffer size on these cards is limited, so the entire packet can't be in the card buffer at the same time, which means that you can not offload the send checksum for jumbograms, only for regular sized packets. Also note that this card is not known to be the fastest one ot there, so you may be better off doing the checksum on your 1GHz+ host processor instead. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 1:52:42 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id DC83437B408; Fri, 28 Sep 2001 01:52:37 -0700 (PDT) Received: from mindspring.com (dialup-209.245.138.226.Dial1.SanJose1.Level3.net [209.245.138.226]) by falcon.mail.pas.earthlink.net (8.11.5/8.9.3) with ESMTP id f8S8qa122087; Fri, 28 Sep 2001 01:52:36 -0700 (PDT) Message-ID: <3BB43A84.34F8E732@mindspring.com> Date: Fri, 28 Sep 2001 01:53:24 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Adrian Chadd Cc: ports@freebsd.org, hackers@freebsd.org Subject: Re: closing down the squid22/23 ports? References: <20010927205912.E4232@roaming.cacheboy.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Adrian Chadd wrote: > I'd like to look at closing down / making inactive the squid22 and > squid23 ports. The squid-2.2 and squid-2.3 codebases have been > inactive and largely unsupported by the squid developers (read: myself > inclusive here) for some time now, and I'd like to point users > at the actively developed/maintained squid branch. > > Squid-2.5 is also in the pipeline for release soon, and I don't think > there is a point in having 4 squid ports. > > What do people think? You might want to check out which versions infringe which patents... I think the oldest port doesn't infringe any, but the newer ports infringe about 8 of them from IBM. IBM made us rip squid out of the InterJet before we ever shipped with it because of this. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 2: 9:35 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from scaup.mail.pas.earthlink.net (scaup.mail.pas.earthlink.net [207.217.121.49]) by hub.freebsd.org (Postfix) with ESMTP id 247EC37B40A for ; Fri, 28 Sep 2001 02:09:32 -0700 (PDT) Received: from mindspring.com (dialup-209.245.138.226.Dial1.SanJose1.Level3.net [209.245.138.226]) by scaup.mail.pas.earthlink.net (EL-8_9_3_3/8.9.3) with ESMTP id CAA20639; Fri, 28 Sep 2001 02:09:20 -0700 (PDT) Message-ID: <3BB43E70.C198F4BB@mindspring.com> Date: Fri, 28 Sep 2001 02:10:08 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: Dan Nelson , "Louis A. Mamakos" , Andrew Gallatin , hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: <200109271416.f8REGaZ64624@whizzo.transsys.com> <15283.14648.430630.163513@grasshopper.cs.duke.edu> <200109271631.f8RGVCZ65964@whizzo.transsys.com> <15283.23007.137091.883110@grasshopper.cs.duke.edu> <200109271717.f8RHHBZ66485@whizzo.transsys.com> <20010928000651.C6178@dan.emsphone.com> <3BB4088E.7B65FDB5@elischer.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer wrote: > > Something to do would be to enable hardware checksumming on 1/2 your > > machines, and compare the bad packet counts at reported by netstat on > > the unchanged machines for (say) a 1-month period before and after the > > change. That should tell you whether you're gaining or losing > > reliability. It'll be really easy for me, as my current (software > > cksum) stats show no errors at all: > > the trouble is that if you are trying to find errors between RAM and WIRE > then the checksums will be correct regardless of whether there is an error.. > so the counts you are checking would be no use.. > you need to know what the correct packet contents should be.. I have a bad idea! I have a bad idea! (timeout for the "bad idea" dance...) Let's checksum all our RAM, so we can know when it corrupts data, too! ...at some point, you are going to have to trust your hardware. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 2:43:40 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from math.teaser.net (math.teaser.net [213.91.2.4]) by hub.freebsd.org (Postfix) with ESMTP id 9B1B737B407 for ; Fri, 28 Sep 2001 02:43:37 -0700 (PDT) Received: from notbsdems.nantes.kisoft-services.com (chantilly.kisoft-services.com [193.56.60.242]) by math.teaser.net (Postfix) with ESMTP id 104076C808; Fri, 28 Sep 2001 11:43:36 +0200 (CEST) Received: by notbsdems.nantes.kisoft-services.com (Postfix, from userid 1001) id 8610EE6BB5; Fri, 28 Sep 2001 11:43:24 +0200 (CEST) To: Mailing List IPFilter Cc: Mailing List FreeBSD Hackers Subject: Re: FreeBSD 4.4-RELEASE & ipf 3.4.20 freeze References: <86g09aqbs3.fsf@notbsdems.nantes.kisoft-services.com> From: Eric Masson In-Reply-To: <86g09aqbs3.fsf@notbsdems.nantes.kisoft-services.com> (Eric Masson's message of "Wed, 26 Sep 2001 16:14:52 +0200") X-Operating-System: FreeBSD 4.4-RC i386 Date: Fri, 28 Sep 2001 11:43:23 +0200 Message-ID: <863d57y7k4.fsf@notbsdems.nantes.kisoft-services.com> Lines: 16 User-Agent: Gnus/5.090004 (Oort Gnus v0.04) XEmacs/21.1 (Cuyahoga Valley) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "Emss" == Eric Masson writes: Emss> I've tried with both ppp integrated nat and ipnat, and the Emss> problem lasts. I let the box working without nat since (1 Mb download every 10 minutes for 40 hours), and the freeze doesn't seem to happen anymore. Any idea ? Eric Masson -- je suis sur qu'on peut faire encore pire, imagine un mec plus colereux que Jaco, plus chiant que Fleury et plus emmerdeur que moi, non mais, imagine un instant. -+- ST in Guide du linuxien pervers - "Le bon, la brute et le truand." -+- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 5:29:55 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id D3D5F37B40C for ; Fri, 28 Sep 2001 05:29:46 -0700 (PDT) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id f8SCTAB32080; Fri, 28 Sep 2001 08:29:11 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 28 Sep 2001 08:29:10 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Vladimir Dozen Cc: hackers@FreeBSD.org Subject: Re: calling open() from inside kernel In-Reply-To: <20010927234624.A403@eix.do-labs.spb.ru> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Generally speaking, you don't want to invoke system call functionss from within the kernel due to address space expectations, you want to invoke the supporting service calls. Probably what that maps into in your case is using NDINIT()/namei() on a string in UIO_SYSSPACE, and then using vn_open(). You'll want to look carefully at the open() code to see if there are other things to do/watch out for. In the Linux emulation code, when the kernel wants to change arguments around and keep them in userspace, it uses space allocated out of the stack gap, a section of VM I assume is otherwise unused in userspace (and presumably is per-stack, or there would be problems with linux threading). Similar tricks could probably be played in the FreeBSD vm space, in a worst case scenario, by mmap'ing some space for the process. However, I would generally advise using the UIO_SYSPACE/vn_open() appraoch above. One final thing to note: name lookups occur with respects to a process's current working directory, and root directory--this means you need to think carefully about what process is present when invoking namei(), especially if there's likely to be lots of chroot()ing going on. Hope that helps, Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services On Thu, 27 Sep 2001, Vladimir Dozen wrote: > ehlo. > > I'm creating a patch to kernel that requires to create a set > of files; names of files are generated inside kernel, i.e., > strings belong to kernel address space. > > Initially, I tried to use open(), but failed with EFAULT: open() > expects filename string is in userspace, and passes UIO_USERSPACE > to NDINIT. > > Well, I copied a portion of code from kern/vfs_syscalls, and it works > fine. But, the length and complexity of the code is too far beyond > I could expect from such a basic operation as file opening, and all > this just because single string is in wrong space. > > So, is there any way to call open() in simple way? Something like > remapping string into curproc space, or telling open() that string > is not in userspace, or smth else? Or, may be, I do something > completely wrong? I'm new in kernel programming. > > -- > dozen @ home > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 5:48:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from ipcard.iptcom.net (ipcard.iptcom.net [212.9.224.5]) by hub.freebsd.org (Postfix) with ESMTP id BE80F37B40A; Fri, 28 Sep 2001 05:47:56 -0700 (PDT) Received: from vega.vega.com (h72.229.dialup.iptcom.net [212.9.229.72]) by ipcard.iptcom.net (8.9.3/8.9.3) with ESMTP id PAA71209; Fri, 28 Sep 2001 15:47:51 +0300 (EEST) (envelope-from sobomax@FreeBSD.org) Received: from FreeBSD.org (big_brother.vega.com [192.168.1.1]) by vega.vega.com (8.11.6/8.11.3) with ESMTP id f8SClJA58842; Fri, 28 Sep 2001 15:47:19 +0300 (EEST) (envelope-from sobomax@FreeBSD.org) Message-ID: <3BB471BD.976B65B6@FreeBSD.org> Date: Fri, 28 Sep 2001 15:49:01 +0300 From: Maxim Sobolev Organization: Vega International Capital X-Mailer: Mozilla 4.78 [en] (Windows NT 5.0; U) X-Accept-Language: en,uk,ru MIME-Version: 1.0 To: Adrian Chadd Cc: ports@FreeBSD.org, hackers@FreeBSD.org Subject: Re: closing down the squid22/23 ports? References: <20010927205912.E4232@roaming.cacheboy.net> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Adrian Chadd wrote: > Hi all, > > Pardon the cross-posting. :-) > > I'd like to look at closing down / making inactive the squid22 and > squid23 ports. The squid-2.2 and squid-2.3 codebases have been > inactive and largely unsupported by the squid developers (read: myself > inclusive here) for some time now, and I'd like to point users > at the actively developed/maintained squid branch. > > Squid-2.5 is also in the pipeline for release soon, and I don't think > there is a point in having 4 squid ports. > > What do people think? > > (please CC me, I'm currently not on the ports/hackers list > for various time-related reasons..) I'm pretty sure that squid22 could be safely killed, but perhaps it would have a sense to keep squid23 around for some more time, because many production systems out there still use it. -Maxim To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 6:22:15 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from prism.flugsvamp.com (cb58709-a.mdsn1.wi.home.com [24.17.241.9]) by hub.freebsd.org (Postfix) with ESMTP id 4C6B137B40E for ; Fri, 28 Sep 2001 06:22:12 -0700 (PDT) Received: (from jlemon@localhost) by prism.flugsvamp.com (8.11.0/8.11.0) id f8SDJev62735; Fri, 28 Sep 2001 08:19:40 -0500 (CDT) (envelope-from jlemon) Date: Fri, 28 Sep 2001 08:19:40 -0500 From: Jonathan Lemon To: Terry Lambert Cc: Jonathan Lemon , the_srinivas@hotmail.com, hackers@freebsd.org Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 Message-ID: <20010928081940.S9056@prism.flugsvamp.com> References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: <3BB42E50.B32F5E95@mindspring.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Sep 28, 2001 at 01:01:20AM -0700, Terry Lambert wrote: > Jonathan Lemon wrote: > > >I'm trying to use the TCP&IP checksum offload capability of the Netgear > > >GA620 NIC from a SMP FreeBSD 4.2R system running on a typical PIII SBC. > > >I did enable TCP&IP cksum offload for receive operations by setting the > > >if_hwassist flag in the driver /sys/pci/if_ti.c and verified that it is > > >working. However I'm unable to offload TCP&IP checksum for the send > > >operations. > > > > All you need to do is uncomment the TI_CSUM_FEATURES flag at the top > > of the file, and recompile. > > He didn't say his packet size, either. > > To the original poster: if you are sending jumbograms, the > buffer size on these cards is limited, so the entire packet > can't be in the card buffer at the same time, which means > that you can not offload the send checksum for jumbograms, > only for regular sized packets. This is true for the nge driver. Does it apply to the ti driver as well? -- Jonathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 6:34:46 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from ip.eth.net (mail.ip.eth.net [202.9.128.18]) by hub.freebsd.org (Postfix) with ESMTP id 29DD837B40A for ; Fri, 28 Sep 2001 06:34:32 -0700 (PDT) Received: (apparently) from anjali ([61.11.16.239]) by ip.eth.net with Microsoft SMTPSVC(5.5.1877.197.19); Fri, 28 Sep 2001 19:04:25 +0530 Message-ID: <008601c14822$3b21bfc0$0a00a8c0@indranet> From: "Anjali Kulkarni" To: Subject: setjmp/longjmp Date: Fri, 28 Sep 2001 19:03:51 +0530 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0083_01C14850.4FF1E140" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. ------=_NextPart_000_0083_01C14850.4FF1E140 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable hi, Does anyone know whether it is advisable or not to use setjmp/longjmp = within kernel code? I could not see any setjmp/longjmp in kernel source = code. Is there a good reason for this or can it be used? Thanks, Anjali ------=_NextPart_000_0083_01C14850.4FF1E140 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
hi,
Does anyone know whether it is = advisable or not to=20 use setjmp/longjmp within kernel code? I could not see any = setjmp/longjmp in=20 kernel source code. Is there a good reason for this or can it be=20 used?
 
Thanks,
Anjali
 
------=_NextPart_000_0083_01C14850.4FF1E140-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 6:44: 5 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from panzer.kdm.org (panzer.kdm.org [216.160.178.169]) by hub.freebsd.org (Postfix) with ESMTP id D998D37B40D for ; Fri, 28 Sep 2001 06:44:01 -0700 (PDT) Received: (from ken@localhost) by panzer.kdm.org (8.11.6/8.9.1) id f8SDhls36993; Fri, 28 Sep 2001 07:43:47 -0600 (MDT) (envelope-from ken) Date: Fri, 28 Sep 2001 07:43:47 -0600 From: "Kenneth D. Merry" To: Terry Lambert Cc: Jonathan Lemon , the_srinivas@hotmail.com, hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 Message-ID: <20010928074347.A36960@panzer.kdm.org> References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <3BB42E50.B32F5E95@mindspring.com>; from tlambert2@mindspring.com on Fri, Sep 28, 2001 at 01:01:20AM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Sep 28, 2001 at 01:01:20 -0700, Terry Lambert wrote: > Jonathan Lemon wrote: > > >I'm trying to use the TCP&IP checksum offload capability of the Netgear > > >GA620 NIC from a SMP FreeBSD 4.2R system running on a typical PIII SBC. > > >I did enable TCP&IP cksum offload for receive operations by setting the > > >if_hwassist flag in the driver /sys/pci/if_ti.c and verified that it is > > >working. However I'm unable to offload TCP&IP checksum for the send > > >operations. > > > > All you need to do is uncomment the TI_CSUM_FEATURES flag at the top > > of the file, and recompile. > > He didn't say his packet size, either. > > To the original poster: if you are sending jumbograms, the > buffer size on these cards is limited, so the entire packet > can't be in the card buffer at the same time, which means > that you can not offload the send checksum for jumbograms, > only for regular sized packets. You've got things confused. I think that may be a limitation of some SysKonnect boards, but certainly isn't a Tigon limitation. Tigon boards come with 512KB, 1MB, or 2MB (never seen one of these) of SRAM on board. The firmware takes a couple hundred KB, so you generally have at least 200KB, and possibly a good bit more for packet storage space on the card. So there is buffer space for a number of packets on both the send and receive sides, and checksumming works in both directions. > Also note that this card is not known to be the fastest > one ot there, so you may be better off doing the checksum > on your 1GHz+ host processor instead. You can get wire rate with a Tigon 2 board, with jumbo frames at least. (That's with checksum offloading enabled.) Ken -- Kenneth Merry ken@kdm.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 6:47: 0 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from imo-r08.mx.aol.com (imo-r08.mx.aol.com [152.163.225.104]) by hub.freebsd.org (Postfix) with ESMTP id DCDBA37B40A for ; Fri, 28 Sep 2001 06:46:56 -0700 (PDT) Received: from Bsdguru@aol.com by imo-r08.mx.aol.com (mail_out_v31_r1.7.) id e.a7.147ebf08 (4417); Fri, 28 Sep 2001 09:46:19 -0400 (EDT) From: Bsdguru@aol.com Message-ID: Date: Fri, 28 Sep 2001 09:46:19 EDT Subject: Re: ecc on i386 To: peter@wemm.org Cc: hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: AOL 5.0 for Windows sub 138 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In a message dated 9/25/01 1:05:21 PM Eastern Daylight Time, peter@wemm.org writes: > > Well, at least we take the machine down, which is a heck of a lot > > better than ignoring the problem, which is really all that I was > > hoping for. I dont think this is "good". Back in the XT days we used to get a false parity error every once on a while on an ISA card...taking the machine down on a bit error (which XTs used to do) was completly wrong and unnecessary. If you are using the box as a router, you dont want the machine to do down because of a memory error, or in this case, a non-error. It should certainly be optional. If you are running a R/O or flash system there is no harm in keeping the machine running if possible. B To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 6:53: 4 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id 44ABC37B405 for ; Fri, 28 Sep 2001 06:53:01 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id JAA11236; Fri, 28 Sep 2001 09:52:45 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8SDqIL88765; Fri, 28 Sep 2001 09:52:18 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15284.32914.874966.260722@grasshopper.cs.duke.edu> Date: Fri, 28 Sep 2001 09:52:18 -0400 (EDT) To: tlambert2@mindspring.com Cc: Jonathan Lemon , the_srinivas@hotmail.com, hackers@freebsd.org Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <3BB42E50.B32F5E95@mindspring.com> References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert writes: > Jonathan Lemon wrote: > > >I'm trying to use the TCP&IP checksum offload capability of the Netgear > > >GA620 NIC from a SMP FreeBSD 4.2R system running on a typical PIII SBC. <..> > > He didn't say his packet size, either. > > To the original poster: if you are sending jumbograms, the > buffer size on these cards is limited, so the entire packet > can't be in the card buffer at the same time, which means > that you can not offload the send checksum for jumbograms, > only for regular sized packets. This is an Alteon Tigon-2 (ti driver) based card with 512K of sram on board. It has plenty of space for offloading transmit checksums on jumbo frames. Perhaps you're thinking of the DP83820/DP83821 (nge driver), which cannot compute the checksum on an outgoing frame unless it fits in the 8K tx fifo. I think NetGear sells a card with a similar name (GA622T) based around this chip. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 7: 7: 0 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id 7794337B409 for ; Fri, 28 Sep 2001 07:06:57 -0700 (PDT) Received: from mail.cicely.de (cicely20 [10.1.1.22]) by srv1.cosmo-project.de (8.11.0/8.11.0) with ESMTP id f8SE6n617995; Fri, 28 Sep 2001 16:06:50 +0200 (CEST) Received: (from ticso@localhost) by mail.cicely.de (8.11.0/8.11.0) id f8SE7Qm12179; Fri, 28 Sep 2001 16:07:26 +0200 (CEST) Date: Fri, 28 Sep 2001 16:07:25 +0200 From: Bernd Walter To: Anjali Kulkarni Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: setjmp/longjmp Message-ID: <20010928160725.D11634@cicely20.cicely.de> References: <008601c14822$3b21bfc0$0a00a8c0@indranet> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <008601c14822$3b21bfc0$0a00a8c0@indranet>; from anjali@indranetworks.com on Fri, Sep 28, 2001 at 07:03:51PM +0530 X-Operating-System: NetBSD cicely20.cicely.de 1.5 sparc Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Sep 28, 2001 at 07:03:51PM +0530, Anjali Kulkarni wrote: > hi, > Does anyone know whether it is advisable or not to use setjmp/longjmp within kernel code? I could not see any setjmp/longjmp in kernel source code. Is there a good reason for this or can it be used? You need to look again, it's used in several places in the kernel. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 7:21:26 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id 5335D37B408 for ; Fri, 28 Sep 2001 07:21:23 -0700 (PDT) Received: from mail.cicely.de (cicely20 [10.1.1.22]) by srv1.cosmo-project.de (8.11.0/8.11.0) with ESMTP id f8SELF618091; Fri, 28 Sep 2001 16:21:17 +0200 (CEST) Received: (from ticso@localhost) by mail.cicely.de (8.11.0/8.11.0) id f8SELre12194; Fri, 28 Sep 2001 16:21:53 +0200 (CEST) Date: Fri, 28 Sep 2001 16:21:53 +0200 From: Bernd Walter To: Bsdguru@aol.com Cc: peter@wemm.org, hackers@FreeBSD.ORG Subject: Re: ecc on i386 Message-ID: <20010928162153.E11634@cicely20.cicely.de> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from Bsdguru@aol.com on Fri, Sep 28, 2001 at 09:46:19AM -0400 X-Operating-System: NetBSD cicely20.cicely.de 1.5 sparc Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Sep 28, 2001 at 09:46:19AM -0400, Bsdguru@aol.com wrote: > In a message dated 9/25/01 1:05:21 PM Eastern Daylight Time, peter@wemm.org > writes: > > > > Well, at least we take the machine down, which is a heck of a lot > > > better than ignoring the problem, which is really all that I was > > > hoping for. > > I dont think this is "good". Back in the XT days we used to get a false > parity error every once on a while on an ISA card...taking the machine down > on a bit error (which XTs used to do) was completly wrong and unnecessary. If Haeh - if your memory content has been changed behind you can only hope that it doesn't trashed some important metadata and won't trash the whole system. Well it's much better if you check the use of the memory region and do some inteligent handling. But ignoring is definately a very dangerous thing. I never understood why computers are build wihtout at least parity. DRAMs have a so called soft-error-rate and may toggle a bit no matter how good the memory is - it's only changing how likely that is. Thus DRAM usage implies using ECC if you don't want any surprises. > you are using the box as a router, you dont want the machine to do down > because of a memory error, or in this case, a non-error. It should certainly A memory corruption is not a "non-error". > be optional. If you are running a R/O or flash system there is no harm in > keeping the machine running if possible. Even in an R/O case it can trust corrupted data and may even distribute. Broken Hardware needs to be exchanged. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 8:19:50 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from tabby.sonn.com (tabby.sonn.com [206.79.239.6]) by hub.freebsd.org (Postfix) with ESMTP id 37F2C37B40F for ; Fri, 28 Sep 2001 08:19:48 -0700 (PDT) Received: from localhost (gersh@localhost) by tabby.sonn.com (8.9.3/8.9.3) with ESMTP id IAA80668; Fri, 28 Sep 2001 08:35:18 -0700 (PDT) Date: Fri, 28 Sep 2001 08:35:17 -0700 (PDT) From: Gersh To: Bernd Walter Cc: Anjali Kulkarni , freebsd-hackers@FreeBSD.ORG Subject: Re: setjmp/longjmp In-Reply-To: <20010928160725.D11634@cicely20.cicely.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Look at sys/i386/i386/db_interface.c On Fri, 28 Sep 2001, Bernd Walter wrote: > On Fri, Sep 28, 2001 at 07:03:51PM +0530, Anjali Kulkarni wrote: > > hi, > > Does anyone know whether it is advisable or not to use setjmp/longjmp within kernel code? I could not see any setjmp/longjmp in kernel source code. Is there a good reason for this or can it be used? > > You need to look again, it's used in several places in the kernel. > > -- > B.Walter COSMO-Project http://www.cosmo-project.de > ticso@cicely.de Usergroup info@cosmo-project.de > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 9: 4: 5 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 7FDB937B40D; Fri, 28 Sep 2001 09:03:52 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id JAA71149; Fri, 28 Sep 2001 09:51:37 -0700 (PDT) Date: Fri, 28 Sep 2001 09:51:35 -0700 (PDT) From: Julian Elischer To: Robert Watson Cc: Vladimir Dozen , hackers@FreeBSD.org Subject: Re: calling open() from inside kernel In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The stack-gap is a bug waiting to happen (we discovered while doing KSE stuff) linux-threads programs that open files in 2 threads at the same time will over-write each other's filenames.. On Fri, 28 Sep 2001, Robert Watson wrote: > > Generally speaking, you don't want to invoke system call functionss from > within the kernel due to address space expectations, you want to invoke > the supporting service calls. Probably what that maps into in your case is > using NDINIT()/namei() on a string in UIO_SYSSPACE, and then using > vn_open(). You'll want to look carefully at the open() code to see if > there are other things to do/watch out for. > > In the Linux emulation code, when the kernel wants to change arguments > around and keep them in userspace, it uses space allocated out of the > stack gap, a section of VM I assume is otherwise unused in userspace (and > presumably is per-stack, or there would be problems with linux threading). > Similar tricks could probably be played in the FreeBSD vm space, in a > worst case scenario, by mmap'ing some space for the process. > > However, I would generally advise using the UIO_SYSPACE/vn_open() appraoch > above. One final thing to note: name lookups occur with respects to a > process's current working directory, and root directory--this means you > need to think carefully about what process is present when invoking > namei(), especially if there's likely to be lots of chroot()ing going on. > > Hope that helps, > > Robert N M Watson FreeBSD Core Team, TrustedBSD Project > robert@fledge.watson.org NAI Labs, Safeport Network Services > > On Thu, 27 Sep 2001, Vladimir Dozen wrote: > > > ehlo. > > > > I'm creating a patch to kernel that requires to create a set > > of files; names of files are generated inside kernel, i.e., > > strings belong to kernel address space. > > > > Initially, I tried to use open(), but failed with EFAULT: open() > > expects filename string is in userspace, and passes UIO_USERSPACE > > to NDINIT. > > > > Well, I copied a portion of code from kern/vfs_syscalls, and it works > > fine. But, the length and complexity of the code is too far beyond > > I could expect from such a basic operation as file opening, and all > > this just because single string is in wrong space. > > > > So, is there any way to call open() in simple way? Something like > > remapping string into curproc space, or telling open() that string > > is not in userspace, or smth else? Or, may be, I do something > > completely wrong? I'm new in kernel programming. > > > > -- > > dozen @ home > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-hackers" in the body of the message > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 9:24: 9 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id EDB3B37B409 for ; Fri, 28 Sep 2001 09:24:00 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id KAA71256; Fri, 28 Sep 2001 10:12:15 -0700 (PDT) Date: Fri, 28 Sep 2001 10:12:14 -0700 (PDT) From: Julian Elischer To: Gersh Cc: Bernd Walter , Anjali Kulkarni , freebsd-hackers@FreeBSD.ORG Subject: Re: setjmp/longjmp In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Yeah but it would probably be a pretty bad idea to use it without very careful thought. Especialy with the kernel becoming pre-emptable in the future.. On Fri, 28 Sep 2001, Gersh wrote: > Look at sys/i386/i386/db_interface.c > > On Fri, 28 Sep 2001, Bernd Walter wrote: > > > On Fri, Sep 28, 2001 at 07:03:51PM +0530, Anjali Kulkarni wrote: > > > hi, > > > Does anyone know whether it is advisable or not to use setjmp/longjmp within kernel code? I could not see any setjmp/longjmp in kernel source code. Is there a good reason for this or can it be used? > > > > You need to look again, it's used in several places in the kernel. > > > > -- > > B.Walter COSMO-Project http://www.cosmo-project.de > > ticso@cicely.de Usergroup info@cosmo-project.de > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-hackers" in the body of the message > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 9:55:33 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mailgw3a.lmco.com (mailgw3a.lmco.com [192.35.35.24]) by hub.freebsd.org (Postfix) with ESMTP id 50D2B37B406 for ; Fri, 28 Sep 2001 09:55:30 -0700 (PDT) Received: from emss01g01.ems.lmco.com ([129.197.181.54]) by mailgw3a.lmco.com (8.8.8/8.8.8) with ESMTP id MAA06106 for ; Fri, 28 Sep 2001 12:55:29 -0400 (EDT) Received: from CONVERSION-DAEMON by lmco.com (PMDF V5.2-32 #38886) id <0GKD00001TNLBO@lmco.com> for freebsd-hackers@freebsd.org; Fri, 28 Sep 2001 09:55:23 -0700 (PDT) Received: from cui1.lmms.lmco.com ([129.197.1.64]) by lmco.com (PMDF V5.2-32 #38886) with ESMTP id <0GKD00AHQTNJDE@lmco.com> for freebsd-hackers@freebsd.org; Fri, 28 Sep 2001 09:54:55 -0700 (PDT) Received: from lmco.com (CONNECTICUT1.lmms.lmco.com [129.197.23.84]) by cui1.lmms.lmco.com (8.11.0/8.9.2) with ESMTP id f8SGss606394 for ; Fri, 28 Sep 2001 09:54:55 -0700 (PDT) Date: Fri, 28 Sep 2001 09:55:03 -0700 From: rick norman Subject: bind : address already inuse To: freebsd-hackers@freebsd.org Message-id: <3BB4AB67.61217AA4@lmco.com> MIME-version: 1.0 X-Mailer: Mozilla 4.77 [en] (WinNT; U) Content-type: MULTIPART/MIXED; BOUNDARY="Boundary_(ID_mQ1p0DshiB00ke/hmU0cHA)" X-Accept-Language: en Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. --Boundary_(ID_mQ1p0DshiB00ke/hmU0cHA) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7BIT When an app binds an address and port to a listen socket, what variables can I adjust so the address may be reused immediately after the app exits. My understanding was that int on = 1; setsockopt(s,SOL_SOCKET,SO_REUSEADDR,&on,sizeof(on)); would do it but there still seems to be a significant amount of time between the exit and bind allowing a new app to use the address, even though there are no inbound connections pending in the listen queue when the exit occurs. I am debugging a server and the process requires restarting often. Thanks, Rick --Boundary_(ID_mQ1p0DshiB00ke/hmU0cHA) Content-type: text/x-vcard; name=rick.norman.vcf; charset=us-ascii Content-description: Card for rick norman Content-disposition: attachment; filename=rick.norman.vcf Content-transfer-encoding: 7BIT begin:vcard n:norman;rick tel;cell:650 303 3877 tel;work:408 742 1619 x-mozilla-html:FALSE adr:;;;;;; version:2.1 email;internet:rick.norman@lmco.com note:Logically speaking, logic is innappropriate. x-mozilla-cpt:;0 fn:rick norman end:vcard --Boundary_(ID_mQ1p0DshiB00ke/hmU0cHA)-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 9:59:46 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from freebie.xs4all.nl (freebie.xs4all.nl [213.84.32.253]) by hub.freebsd.org (Postfix) with ESMTP id B908F37B406 for ; Fri, 28 Sep 2001 09:59:39 -0700 (PDT) Received: (from wkb@localhost) by freebie.xs4all.nl (8.11.6/8.11.6) id f8SGxG901090; Fri, 28 Sep 2001 18:59:16 +0200 (CEST) (envelope-from wkb) Date: Fri, 28 Sep 2001 18:59:16 +0200 From: Wilko Bulte To: =?iso-8859-1?Q?S=F8ren_Schmidt?= Cc: kc5vdj@yahoo.com, kbstew99@hotmail.com, Dan , freebsd-hackers@FreeBSD.ORG Subject: Re: power supplies Message-ID: <20010928185916.A1032@freebie.xs4all.nl> References: <3BB40D0B.7000008@yahoo.com> <200109280638.f8S6cKI40211@freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.2.5i In-Reply-To: <200109280638.f8S6cKI40211@freebsd.dk>; from sos@freebsd.dk on Fri, Sep 28, 2001 at 08:38:20AM +0200 X-OS: FreeBSD 4.4-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Sep 28, 2001 at 08:38:20AM +0200, Søren Schmidt wrote: > It seems Jim Bryant wrote: > > Kent Stewart wrote: > > > > > There are problems with PSes when you use NICs with wake up > > > capability. The NIC may exceed the capability of one of your low > > > amperage voltages. > > > > How much current can wake-on-LAN take? I wouldn't think it would be enough to overload a power supply unless it overloaded the -12V > > line which is usually only rated in the mils. What is the average 12V rating? 5-8 amps? and even more for 5V? I forget what the > > average -5V line is rated. > > The problem is the +5 standby supply. In a modern ATX evironment lots > of stuff pull power on that rail when the machine is switched off, > and most ATX supplies only can delive .5-.7 amps here which can easily > be overwhelmed. > > BTW you all know that you ATX machine use about 25W even when switched > off right ? This gets into real money if you have a handfull of machines, > and you get absolutly nothing for it :) 25W? 4 or 5 according to my measurements. -- | / o / /_ _ email: wilko@FreeBSD.org |/|/ / / /( (_) Bulte Arnhem, The Netherlands To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 10:28:10 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 6544F37B40B for ; Fri, 28 Sep 2001 10:28:07 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8SHS5E28842; Fri, 28 Sep 2001 10:28:05 -0700 (PDT) (envelope-from dillon) Date: Fri, 28 Sep 2001 10:28:05 -0700 (PDT) From: Matt Dillon Message-Id: <200109281728.f8SHS5E28842@earth.backplane.com> To: rick norman Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: bind : address already inuse References: <3BB4AB67.61217AA4@lmco.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :--Boundary_(ID_mQ1p0DshiB00ke/hmU0cHA) :Content-type: text/plain; charset=us-ascii :Content-transfer-encoding: 7BIT : :When an app binds an address and port to a listen socket, what :variables :can I adjust so the address may be reused immediately after the app :exits. :My understanding was that :int on = 1; :setsockopt(s,SOL_SOCKET,SO_REUSEADDR,&on,sizeof(on)); :would do it but there still seems to be a significant amount of time :between :the exit and bind allowing a new app to use the address, even though :there :are no inbound connections pending in the listen queue when the exit :occurs. :I am debugging a server and the process requires restarting often. : :Thanks, :Rick SO_REUSEADDR is the correct socket opt and it will allow the address to be reused immediately. If you still get 'address already in use' then there is still another process listening on the socket... probably an older named that you missed, or perhaps the other named simply wasn't exiting quickly enough before you started the new one. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 10:47:38 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id DC95537B406; Fri, 28 Sep 2001 10:47:35 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8SHlUP29063; Fri, 28 Sep 2001 10:47:30 -0700 (PDT) (envelope-from dillon) Date: Fri, 28 Sep 2001 10:47:30 -0700 (PDT) From: Matt Dillon Message-Id: <200109281747.f8SHlUP29063@earth.backplane.com> To: Douglas Swarin Cc: hackers@freebsd.org, tmoestl@gmx.net, bp@freebsd.org Subject: Re: more on Re: Please review: bugfix for vinvalbuf() References: <20010711003926.B8799@crow.dom2ip.de> <200107110643.f6B6hTB24707@earth.backplane.com> <20010926204333.A15865@staff.texas.net> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :@@ -721,9 +721,9 @@ : } : } : :- while (vp->v_numoutput > 0) { :- vp->v_flag |= VBWAIT; :- tsleep(&vp->v_numoutput, PVM, "vnvlbv", 0); :+ if (VOP_GETVOBJECT(vp, &object) == 0) { :+ while (object->paging_in_progress) :+ vm_object_pip_sleep(object, "vnvlbv"); : } : : splx(s); Hey Douglas, try the patch fragment below and see if you can reproduce the problem. -Matt @@ -721,10 +746,21 @@ } } - while (vp->v_numoutput > 0) { - vp->v_flag |= VBWAIT; - tsleep(&vp->v_numoutput, PVM, "vnvlbv", 0); - } + /* + * Wait for I/O to complete. XXX needs cleaning up. The vnode can + * have write I/O in-progress but if there is a VM object then the + * VM object can also have read-I/O in-progress. + */ + do { + while (vp->v_numoutput > 0) { + vp->v_flag |= VBWAIT; + tsleep(&vp->v_numoutput, PVM, "vnvlbv", 0); + } + if (VOP_GETVOBJECT(vp, &object) == 0) { + while (object->paging_in_progress) + vm_object_pip_sleep(object, "vnvlbx"); + } + } while (vp->v_numoutput > 0); splx(s); To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 10:49:27 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from milliways.chance.ru (milliways.chance.ru [195.190.107.35]) by hub.freebsd.org (Postfix) with ESMTP id 3BF1E37B40E for ; Fri, 28 Sep 2001 10:49:20 -0700 (PDT) Received: from do-labs.spb.ru (ppp-4.chance.ru [195.190.107.7]) by milliways.chance.ru (8.9.0/8.9.0) with SMTP id VAA28151 for ; Fri, 28 Sep 2001 21:49:10 +0400 (MSD) Received: (qmail 489 invoked by uid 1000); 28 Sep 2001 21:15:13 -0000 Date: Fri, 28 Sep 2001 21:15:13 +0000 From: Vladimir Dozen To: Robert Watson Cc: hackers@FreeBSD.org Subject: Re: calling open() from inside kernel Message-ID: <20010928211513.A294@eix.do-labs.spb.ru> References: <20010927234624.A403@eix.do-labs.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from rwatson@FreeBSD.org on Fri, Sep 28, 2001 at 08:29:10AM -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ehlo. > Generally speaking, you don't want to invoke system call functionss from > within the kernel due to address space expectations, you want to invoke > the supporting service calls. Probably what that maps into in your case is > using NDINIT()/namei() on a string in UIO_SYSSPACE, and then using > vn_open(). You'll want to look carefully at the open() code to see if > there are other things to do/watch out for. That was exactly what I've done. The problem here is that each time someone in kernel space will try to open a file, he/she has to retype the same code already in vfs_syscalls.c. I would like to see more general version of open and two short wrappers -- one for userspace and one for kernel space, something like: /* generic */ open_generic(struct proc* proc,struct open_args* oa,boolean_t inkernel) { if( inkernel ) NDINIT(...,UIO_SYSSPACE,...); else NDINIT(...,UIO_USERSPACE,...); ... } /* user-space */ open(proc,oa){ return open_generic(proc,oa,0); } /* kernel space */ open_sysspace(proc,oa){ return open_generic(proc,oa,1); } Well, I see I was wanted too much. > In the Linux emulation code, when the kernel wants to change arguments > around and keep them in userspace, it uses space allocated out of the > stack gap, a section of VM I assume is otherwise unused in userspace (and > presumably is per-stack, or there would be problems with linux threading). This cure is more dangerous than illness itself. ;) -- dozen @ home To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 10:56: 8 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id A718037B40E for ; Fri, 28 Sep 2001 10:56:01 -0700 (PDT) Received: from mail.cicely.de (cicely20 [10.1.1.22]) by srv1.cosmo-project.de (8.11.0/8.11.0) with ESMTP id f8SHtw619278; Fri, 28 Sep 2001 19:55:58 +0200 (CEST) Received: (from ticso@localhost) by mail.cicely.de (8.11.0/8.11.0) id f8SHuZt12829; Fri, 28 Sep 2001 19:56:35 +0200 (CEST) Date: Fri, 28 Sep 2001 19:56:34 +0200 From: Bernd Walter To: rick norman Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: bind : address already inuse Message-ID: <20010928195634.A12561@cicely20.cicely.de> References: <3BB4AB67.61217AA4@lmco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3BB4AB67.61217AA4@lmco.com>; from rick.norman@lmco.com on Fri, Sep 28, 2001 at 09:55:03AM -0700 X-Operating-System: NetBSD cicely20.cicely.de 1.5 sparc Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Sep 28, 2001 at 09:55:03AM -0700, rick norman wrote: > When an app binds an address and port to a listen socket, what > variables > can I adjust so the address may be reused immediately after the app > exits. > My understanding was that > int on = 1; > setsockopt(s,SOL_SOCKET,SO_REUSEADDR,&on,sizeof(on)); > would do it but there still seems to be a significant amount of time > between > the exit and bind allowing a new app to use the address, even though > there > are no inbound connections pending in the listen queue when the exit > occurs. > I am debugging a server and the process requires restarting often. You need to do this before bind/listen and I asume you should fork between creating this socket and setsockopt. You might want to check if there is still a process listening - a child comes into mind - or if there are only existing state informations about old connects. SO_REUSEADDR protects you only for the later. netstat and sockstat are your friends. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 11:35: 4 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from whizzo.transsys.com (whizzo.TransSys.COM [144.202.42.10]) by hub.freebsd.org (Postfix) with ESMTP id 1348037B40C for ; Fri, 28 Sep 2001 11:35:01 -0700 (PDT) Received: from whizzo.transsys.com (#6@localhost.transsys.com [127.0.0.1]) by whizzo.transsys.com (8.11.4/8.11.4) with ESMTP id f8SIYdZ77174; Fri, 28 Sep 2001 14:34:39 -0400 (EDT) (envelope-from louie@whizzo.transsys.com) Message-Id: <200109281834.f8SIYdZ77174@whizzo.transsys.com> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Andrew Gallatin Cc: tlambert2@mindspring.com, Jonathan Lemon , the_srinivas@hotmail.com, hackers@FreeBSD.ORG X-Image-URL: http://www.transsys.com/louie/images/louie-mail.jpg From: "Louis A. Mamakos" Subject: timestamp offload [was Re: TCP&IP cksum offload on FreeBSD 4.2] References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> <15284.32914.874966.260722@grasshopper.cs.duke.edu> In-reply-to: Your message of "Fri, 28 Sep 2001 09:52:18 EDT." <15284.32914.874966.260722@grasshopper.cs.duke.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 28 Sep 2001 14:34:39 -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The cool thing I've always wanted to do with these programmable network adapters is to have them capture timestamps of when packets are received for high-accuracy latency measurements. The network adapter could drop a timestamp into some header when it's DMA'ed into the host's memory. The timestamp could come from some other PCI device (e.g., Datum bc635) which can provide a timestamp in 3 PCI bus cycles. The same sort of think could be done on transmit for suitably tagged packets. Some work I did a year or so ago measured the interrupt response time latency, and it was pretty impressive at how large and variable it could be. louie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 11:48:12 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by hub.freebsd.org (Postfix) with ESMTP id BCDFC37B410 for ; Fri, 28 Sep 2001 11:47:54 -0700 (PDT) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.9.3/8.9.3) with ESMTP id OAA19988; Fri, 28 Sep 2001 14:47:50 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.3/8.9.1) id f8SIlOZ91144; Fri, 28 Sep 2001 14:47:24 -0400 (EDT) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15284.50620.103948.237567@grasshopper.cs.duke.edu> Date: Fri, 28 Sep 2001 14:47:24 -0400 (EDT) To: "Louis A. Mamakos" Cc: hackers@FreeBSD.ORG Subject: Re: timestamp offload [was Re: TCP&IP cksum offload on FreeBSD 4.2] In-Reply-To: <200109281834.f8SIYdZ77174@whizzo.transsys.com> References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> <15284.32914.874966.260722@grasshopper.cs.duke.edu> <200109281834.f8SIYdZ77174@whizzo.transsys.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Louis A. Mamakos writes: > > Some work I did a year or so ago measured the interrupt response time > latency, and it was pretty impressive at how large and variable it > could be. > > louie Yes. Me too, but with a pamette, not a nic. Have you read the pci pamette perf paper (Systems Performance Measurement on PCI Pamette (1997), Laurent Moll & Mark Shand)? http://citeseer.nj.nec.com/1690.html If anybody cares, I have freebsd drivers for the pamette. Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 12:11:32 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 0774C37B403 for ; Fri, 28 Sep 2001 12:11:29 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.4/8.11.4) with ESMTP id f8SJB5v77781; Fri, 28 Sep 2001 21:11:05 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Andrew Gallatin Cc: "Louis A. Mamakos" , hackers@FreeBSD.ORG Subject: Re: timestamp offload [was Re: TCP&IP cksum offload on FreeBSD 4.2] In-Reply-To: Your message of "Fri, 28 Sep 2001 14:47:24 EDT." <15284.50620.103948.237567@grasshopper.cs.duke.edu> Date: Fri, 28 Sep 2001 21:11:05 +0200 Message-ID: <77779.1001704265@critter> From: Poul-Henning Kamp Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <15284.50620.103948.237567@grasshopper.cs.duke.edu>, Andrew Gallatin writes: > >Louis A. Mamakos writes: > > > > Some work I did a year or so ago measured the interrupt response time > > latency, and it was pretty impressive at how large and variable it > > could be. > > > > louie > >Yes. Me too, but with a pamette, not a nic. The "HotWorks" from www.vcc.com can do the same, a lot cheaper and it has a FreeBSD driver: sys/pci/xrpu.c -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 12:12:11 2001 Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 618) id BE98A37B403; Fri, 28 Sep 2001 12:12:05 -0700 (PDT) Subject: Driver kit for BCM570x gigE NICs To: freebsd-hackers@freebsd.org Date: Fri, 28 Sep 2001 12:12:05 -0700 (PDT) Cc: freebsd-net@freebsd.org X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-Id: <20010928191205.BE98A37B403@hub.freebsd.org> From: wpaul@FreeBSD.ORG (Bill Paul) Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I have made a driver retrofit kit available for Broadcom BCM570x-based gigabit ethernet cards at the following URL: http://www.freebsd.org/~wpaul/Broadcom/4.x/bcm570x_drv.tar.gz This kit contains source and pre-compiled driver modules for FreeBSD 4.3-RELEASE and later. This should make it easier for people to test the bge driver on existing 4.x systems without having to recompile the kernel. Again, I would appreciate hearing from people who have cards that use this chipset and can test the driver. If you can run speed tests with things like netperf, ttcp or just plain file/data transfers, I'd be interested to see the results. Share and enjoy! -Bill -- ============================================================================= -Bill Paul (510) 749-2329 | Senior Engineer, Master of Unix-Fu wpaul@windriver.com | Wind River Systems ============================================================================= "You stole fizzy lifting drinks. You broke the rules. You LOSE." -W. Wonka ============================================================================= To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 12:31:18 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.webmonster.de (datasink.webmonster.de [194.162.162.209]) by hub.freebsd.org (Postfix) with SMTP id C534537B409 for ; Fri, 28 Sep 2001 12:31:06 -0700 (PDT) Received: (qmail 33771 invoked by uid 1000); 28 Sep 2001 19:31:25 -0000 Date: Fri, 28 Sep 2001 21:31:25 +0200 From: "Karsten W. Rohrbach" To: Dave Hayes Cc: freebsd-hackers@freebsd.org Subject: Re: Problems with many ATA drives Message-ID: <20010928213125.A33572@mail.webmonster.de> Mail-Followup-To: "Karsten W. Rohrbach" , Dave Hayes , freebsd-hackers@freebsd.org References: <200109231643.JAA09454@hokkshideh.jetcafe.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="fUYQa+Pmc3FrFX/N" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109231643.JAA09454@hokkshideh.jetcafe.org>; from dave@jetcafe.org on Sun, Sep 23, 2001 at 09:43:25AM -0700 X-Arbitrary-Number-Of-The-Day: 42 X-URL: http://www.webmonster.de/ X-Disclaimer: My opinions do not necessarily represent those of my employer X-Work-URL: http://www.ngenn.net/ X-Work-Address: nGENn GmbH, Schloss Kransberg, D-61250 Usingen-Kransberg, Germany X-Work-Phone: +49-6081-682-304 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --fUYQa+Pmc3FrFX/N Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Dave Hayes(dave@jetcafe.org)@2001.09.23 09:43:25 +0000: > We've been attempting to set up a vinum raid box with a bunch of IDE > drives. Each drive is partitioned with a vinum partition on A, such > that the entire drive is on partition a. Initial partitioning is done > with /stand/sysinstall so it "fixes" my geometry, this has always > worked in the past. >=20 > I had been getting "funny" stuff from the drives, so I devised the > following simple test: >=20 > # dd if=3D/dev/rad1a of=3D/dev/null >=20 > This eventually produces: >=20 > ad1: READ command timeout tag=3D0 serv=3D0 - resetting > ata0: resetting devices .. done > ad1a: hard error reading fsbn 5068879 (ad1 bn 5068879; cn 315 tn 133 sn= =20 > 25)ad1a: hard error reading fsbn 5068879 (ad1 bn 5068879; cn 315 tn 133 s= n 25)=20 > status=3D59 error=3D40 >=20 > I notice 3 out of 11 drives produce this error, so far one on each > controller (ruling out a specific controller issue). I didn't want to > just assume the failure rate of 80GB IDE drives is that large, so > I'm asking this list for it's opinion: media errors due to broken qa in production? i did not have that occur with maxtor drives, yet. several ibm drives (DTLA, 45 and 75gb) were fried in my workstation the last weeks all with the same error. after reading several posts on the linux-kernel mailing list it seems to me that the smart firmware on the drives might be b0rked (the ibm case). i did no experience any problems with the maxtor 80gb (4W*) drives. to me it all boils down to this: high capacity cheap-o ide drives suck because the cut the costs in firmware development and quality assurance). for mission critical server hardware i am still building servers on scsi u3w with 32gb ibm disks (DDYS) without a single outage in hundreds of units. besides that, what cabling are you using? cheers, /k >=20 > a) Is this a bug or consequence of software drivers? (see > bug kern/17592) >=20 > b) Or is it just that IDE drives are cheap and fail this much? >=20 > Relevant data from dmesg: >=20 > atapci0: port 0xb000-0xb00f,0xb400-0xb403,0xb= 800-0x > b807,0xd000-0xd003,0xd400-0xd407 mem 0xf5800000-0xf5803fff irq 6 at devic= e=20 > 10.0 on pci2 > ata2: at 0xd400 on atapci0 > ata3: at 0xb800 on atapci0 > atapci1: port 0x9400-0x940f,0x9800-0x9803,0xa= 000-0x > a007,0xa400-0xa403,0xa800-0xa807 mem 0xf5000000-0xf5003fff irq 9 at devic= e=20 > 11.0 on pci2 > ata4: at 0xa800 on atapci1 > ata5: at 0xa000 on atapci1 > ... > atapci2: port 0x8800-0x880f at device 31.1= on=20 > pci0 > ata0: at 0x1f0 irq 14 on atapci2 > ata1: at 0x170 irq 15 on atapci2 > ... > ad0: 78167MB [158816/16/63] at ata0-master UDMA100 > ad1: 78167MB [158816/16/63] at ata0-slave UDMA100 > ad2: 78167MB [158816/16/63] at ata1-master UDMA100 > ad3: 78167MB [158816/16/63] at ata1-slave UDMA100 > ad4: 78167MB [158816/16/63] at ata2-master WDMA2 > ad5: 78167MB [158816/16/63] at ata2-slave WDMA2 > ad6: 78167MB [158816/16/63] at ata3-master WDMA2 > ad7: 78167MB [158816/16/63] at ata3-slave WDMA2 > ad8: 78167MB [158816/16/63] at ata4-master WDMA2 > ad9: 78167MB [158816/16/63] at ata4-slave WDMA2 >=20 > Yes, we know that the "WDMA2" is happening, this state proved to be > independant of a drive failing. It has to do with 10 drives in a tower=20 > and cable lengths... =3D( > ------ > Dave Hayes - Consultant - Altadena CA, USA - dave@jetcafe.org=20 > >>> The opinions expressed above are entirely my own <<< >=20 > There is no distinctly native American criminal class except Congress. > -- Mark Twa= in >=20 >=20 >=20 > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message --=20 > To avoid criticism, do nothing, say nothing, be nothing. --Elbert Hubbard KR433/KR11-RIPE -- WebMonster Community Founder -- nGENn GmbH Senior Techie http://www.webmonster.de/ -- ftp://ftp.webmonster.de/ -- http://www.ngenn.n= et/ karsten&rohrbach.de -- alpha&ngenn.net -- alpha&scene.org -- catch@spam.de GnuPG 0x2964BF46 2001-03-15 42F9 9FFF 50D4 2F38 DBEE DF22 3340 4F4E 2964 B= F46 Please do not remove my address from To: and Cc: fields in mailing lists. 1= 0x --fUYQa+Pmc3FrFX/N Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7tNANM0BPTilkv0YRAuG3AJ9lvSxl4XlOopLj6Yo6Wsa1QEuCdwCeKiPk Pd/u64D32Yb6UcKhmf5uoxU= =pKF1 -----END PGP SIGNATURE----- --fUYQa+Pmc3FrFX/N-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 12:43:20 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.webmonster.de (datasink.webmonster.de [194.162.162.209]) by hub.freebsd.org (Postfix) with SMTP id E585337B40C for ; Fri, 28 Sep 2001 12:43:11 -0700 (PDT) Received: (qmail 34055 invoked by uid 1000); 28 Sep 2001 19:43:32 -0000 Date: Fri, 28 Sep 2001 21:43:32 +0200 From: "Karsten W. Rohrbach" To: Dan Cc: kbstew99@hotmail.com, freebsd-hackers@FreeBSD.ORG Subject: Re: power supplies Message-ID: <20010928214332.B33572@mail.webmonster.de> Mail-Followup-To: "Karsten W. Rohrbach" , Dan , kbstew99@hotmail.com, freebsd-hackers@FreeBSD.ORG References: <3BB3E2BF.648840A7@owt.com> <20010927200213.U1885-100000@gandalf.bravenet.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="V0207lvV8h4k8FAm" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010927200213.U1885-100000@gandalf.bravenet.com>; from dphoenix@bravenet.com on Thu, Sep 27, 2001 at 08:04:40PM -0700 X-Arbitrary-Number-Of-The-Day: 42 X-URL: http://www.webmonster.de/ X-Disclaimer: My opinions do not necessarily represent those of my employer X-Work-URL: http://www.ngenn.net/ X-Work-Address: nGENn GmbH, Schloss Kransberg, D-61250 Usingen-Kransberg, Germany X-Work-Phone: +49-6081-682-304 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --V0207lvV8h4k8FAm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Dan(dphoenix@bravenet.com)@2001.09.27 20:04:40 +0000: >=20 > ya but even putting the old nic back in the machine does not still boot > up. I don't think this has to do with the nic but you never know. > fxp1: questions of the pragmatic field-engineer: "did it produce any smoke?" "did it stink?" "does it still stink, even if it's turned off?" q3 if answered with yes means that the electret condenser are gone. had this with a whole series of korean 300va atx power supplies. you don't happen to have those nice nvidia or voodoo agp4x cards in there, don't you? if yes -> kick'em out, they don't do any good in servers. rageiic or other agp1x cards <$20 have proven to be the best choice for me. good luck, /k >=20 >=20 >=20 > On Thu, 27 Sep 2001, Kent Stewart wrote: >=20 > > Date: Thu, 27 Sep 2001 19:38:55 -0700 > > From: Kent Stewart > > Reply-To: kbstew99@hotmail.com > > To: Dan > > Cc: freebsd-hackers@FreeBSD.ORG > > Subject: Re: power supplies > > > > > > > > Dan wrote: > > > > > > I had the stangest situation today where a new nic card was put into a > > > machine and then the machine did not start up. Placed the old nic card > > > back in the box and it still did not start up. Switched power supplies > > > with an exactly equal box and both machine booted up fine. This has > > > happened twice since we started replacing nic cards today with ones w= ith > > > more buffer space available on them out of about 8 machines now. > > > > > > Does this make any sense to anyone? > > > > There are problems with PSes when you use NICs with wake up > > capability. The NIC may exceed the capability of one of your low > > amperage voltages. > > > > Kent > > > > > > > > -- > > > Dan > > > > > > +------------------------------------------------------+ > > > | BRAVENET WEB SERVICES | > > > | dan@bravenet.com | > > > | screen;cd /usr/src;make buildworld;cd ~ | > > > | cp MYKERNEL /sys/i386/conf;cd /usr/src | > > > | make buildkernel KERNCONF=3DMYKERNEL | > > > |make installkernel KERNCONF=3DMYKERNEL;make installworld| > > > +______________________________________________________+ > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > > with "unsubscribe freebsd-hackers" in the body of the message > > > > -- > > Kent Stewart > > Richland, WA > > > > mailto:kbstew99@hotmail.com > > http://kstewart.urx.com/kstewart/index.html > > FreeBSD News http://daily.daemonnews.org/ > > >=20 >=20 > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message --=20 > Yes, it is inconvenient. Security and convenience are usually mutually > exclusive concepts. --Erik Trulsson on freebsd-stable, Jun 2001 KR433/KR11-RIPE -- WebMonster Community Founder -- nGENn GmbH Senior Techie http://www.webmonster.de/ -- ftp://ftp.webmonster.de/ -- http://www.ngenn.n= et/ karsten&rohrbach.de -- alpha&ngenn.net -- alpha&scene.org -- catch@spam.de GnuPG 0x2964BF46 2001-03-15 42F9 9FFF 50D4 2F38 DBEE DF22 3340 4F4E 2964 B= F46 Please do not remove my address from To: and Cc: fields in mailing lists. 1= 0x --V0207lvV8h4k8FAm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7tNLkM0BPTilkv0YRAq6lAJ9hXWAXO8TgX6HdgzMpXsAEZAyTaACgsIZF 5I6DJUuXTcY56G8czFFLqmY= =OfVg -----END PGP SIGNATURE----- --V0207lvV8h4k8FAm-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 14:51:50 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from birch.ripe.net (birch.ripe.net [193.0.1.96]) by hub.freebsd.org (Postfix) with ESMTP id 1230537B410 for ; Fri, 28 Sep 2001 14:51:47 -0700 (PDT) Received: from laptop.6bone.nl (penguin.ripe.net [193.0.1.232]) by birch.ripe.net (8.11.6/8.11.6) with SMTP id f8SLpjv21601 for ; Fri, 28 Sep 2001 23:51:45 +0200 Received: (nullmailer pid 5773 invoked by uid 1000); Fri, 28 Sep 2001 21:51:45 -0000 Date: Fri, 28 Sep 2001 23:51:45 +0200 From: Mark Santcroos To: freebsd-hackers@freebsd.org Subject: debugging vmware3 linux binary Message-ID: <20010928235145.A4655@laptop.6bone.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i X-Handles: MS6-6BONE, MS18417-RIPE Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I've been working on porting vmware3-beta to FreeBSD. Just as with vmware2 a sourcecode version for Linux is available for vmware3 and I have modified that to FreeBSD using the same approach as with vmware2. The kernel module compiles cleanly now and should probably work... [TM] However, the vmware program itself is only shipped as a Linux binary. Did the vmware2 binary ran flawlessly on FreeBSD, the vmware3 one does not. My question now: how the hell can I track this down? (ktrace doesn't seem to provide any valuable input, but maybe I overlook something) Thanks Mark -- Mark Santcroos RIPE Network Coordination Centre http://www.ripe.net/home/mark/ New Projects Group/TTM To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 15: 6:26 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from h132-197-97-45.gte.com (h132-197-97-45.gte.com [132.197.97.45]) by hub.freebsd.org (Postfix) with ESMTP id F14C237B409 for ; Fri, 28 Sep 2001 15:06:21 -0700 (PDT) Received: (from ak03@localhost) by h132-197-97-45.gte.com (8.11.6/8.11.4) id f8SM6JC28641; Fri, 28 Sep 2001 18:06:19 -0400 (EDT) (envelope-from ak03) Message-ID: X-Mailer: XFMail 1.4.7p2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20010928235145.A4655@laptop.6bone.nl> Date: Fri, 28 Sep 2001 18:06:19 -0400 (EDT) Organization: Verizon Laboratories Inc. From: "Alexander N. Kabaev" To: Mark Santcroos Subject: RE: debugging vmware3 linux binary Cc: freebsd-hackers@FreeBSD.ORG Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark, linux_kdump port will properly decode Linux syscalls so that you will be able to see where vmware3 binary fails exactly. -- E-Mail: Alexander N. Kabaev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 16:29:17 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from guru.mired.org (okc-94-248-46.mmcable.com [24.94.248.46]) by hub.freebsd.org (Postfix) with SMTP id 69F9437B40E for ; Fri, 28 Sep 2001 16:29:10 -0700 (PDT) Received: (qmail 63354 invoked by uid 100); 28 Sep 2001 23:29:09 -0000 From: Mike Meyer MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15285.1989.876502.979545@guru.mired.org> Date: Fri, 28 Sep 2001 18:29:09 -0500 To: Samuel Greear Cc: freebsd-hackers@freebsd.org Subject: Re: dirlist mangling In-Reply-To: <01092501160402.10676@beware.dragonknight.net> References: <01092501160402.10676@beware.dragonknight.net> X-Mailer: VM 6.90 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid X-face: "5Mnwy%?j>IIV\)A=):rjWL~NB2aH[}Yq8Z=u~vJ`"(,&SiLvbbz2W`;h9L,Yg`+vb1>RG% *h+%X^n0EZd>TM8_IB;a8F?(Fb"lw'IgCoyM.[Lg#r\ Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Samuel Greear types: > Never done any kernel hacking before so I'm just looking > for some pointers. What's needed is a mechanism to > specify a directory (or set of them) and whenever a request > is made for the contents of that directory, if it exists in the > list then what is returned needs to be mangled in some > ways. For instance an ls in a directory in the list might > only return a list of files that you own, or that you have > permission to read. Since no one else bothered to answer this one - at least in public - I'll take a crack at it. I think you want to do this as a file system. Take a look at the mount_umap man to see what kind of things you can do. You could use that as a starting point - except you really ought to get the umap file system working first. http://www.mired.org/home/mwm/ Q: How do you make the gods laugh? A: Tell them your plans. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 16:45:29 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from ussenterprise.ufp.org (ussenterprise.ufp.org [208.185.30.210]) by hub.freebsd.org (Postfix) with ESMTP id 1601237B408 for ; Fri, 28 Sep 2001 16:45:27 -0700 (PDT) Received: (from bicknell@localhost) by ussenterprise.ufp.org (8.11.1/8.11.1) id f8SNjQE40283 for freebsd-hackers@freebsd.org; Fri, 28 Sep 2001 19:45:26 -0400 (EDT) (envelope-from bicknell) Date: Fri, 28 Sep 2001 19:45:26 -0400 From: Leo Bicknell To: freebsd-hackers@freebsd.org Subject: Mirrored / redundant disks. Message-ID: <20010928194526.A40019@ussenterprise.ufp.org> Mail-Followup-To: Leo Bicknell , freebsd-hackers@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Organization: United Federation of Planets Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I need to configure a server that can deal with a disk failure. I've been looking at CCD and Vinum, but both seem to have issues that make automatic recovery in the face of one dead disk (in a mirror) less than optimal. So, if you know how to make it so a disk can die and the box keeps running, and more importantly _can be rebooted_ and keep running either via careful configuration or some crufty scripts if you could contact me that would be welcome. No need to keep on-list unless you think there's something of great value to hackers. -- Leo Bicknell - bicknell@ufp.org Systems Engineer - Internetworking Engineer - CCIE 3440 Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Sep 28 17: 8:44 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mw3.texas.net (mw3.texas.net [206.127.30.13]) by hub.freebsd.org (Postfix) with ESMTP id F3E3B37B406 for ; Fri, 28 Sep 2001 17:08:39 -0700 (PDT) Received: from staff3.texas.net (staff3.texas.net [207.207.0.40]) by mw3.texas.net (8.11.6/8.11.6) with ESMTP id f8T08cE19305; Fri, 28 Sep 2001 19:08:38 -0500 (CDT) Received: (from doug@localhost) by staff3.texas.net (8.11.6/8.11.6) id f8T08cX46772; Fri, 28 Sep 2001 19:08:38 -0500 (CDT) (envelope-from doug) Date: Fri, 28 Sep 2001 19:08:38 -0500 From: Doug Swarin To: hackers@freebsd.org, Leo Bicknell Subject: Re: Mirrored / redundant disks. Message-ID: <20010928190838.A46600@staff.texas.net> References: <20010928194526.A40019@ussenterprise.ufp.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010928194526.A40019@ussenterprise.ufp.org>; from bicknell@ufp.org on Fri, Sep 28, 2001 at 07:45:26PM -0400 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I've got some experience with this. One of the solutions I have used in the past is as follows: 1. Configure the two disks with a root partition, swap partition, and a vinum partition covering the rest of the disk. If you make your root partition 128M, you can just barely fit a base system in there. Finagle a bit with disklabel -e so both disks have their partitions configured as: a: root filesystem b: swap space c: (entire disk) e: vinum Also, disklabel -B the second disk to make it bootable. 2. Set up vinum mirror volumes for /usr and /var. 3. Move the data from /usr and /var to their vinum equivalents in single user mode, and set up your fstab to mount the vinum volumes (for example, /dev/vinum/varvol). Make sure you add start_vinum="YES" to your rc.conf. 5. Set up the remainder of your system. cvsup sources and ports and so forth as necessary to configure it. 6. Mount the second disk's 'a' partition somewhere and use dump/restore to copy the primary disk's root partition to the secondary's. Optionally set up a cron job to keep the two in sync. With this configuration, if your primary disk fails, you can pull it and boot off the secondary disk. If your secondary disk fails, you'll still have the primary and can replace the secondary at your leisure. The only problem is if the primary disk fails completely, is not bootable, and is at a location where the box cannot have the drive removed. The only solution I know of in this case would be a hardware or pseudo-hardware RAID of some sort (I say pseudo- hardware because I'm not sure how much the various IDE RAID systems that are available depend on software to do their job). Other possibilities I am aware of are the various IDE flash-based drives. They have no moving parts and modern flash supports many write cycles (I would still suggest using them only as a boot drive with minimal writes and using a fully mirrored pair of standard hard drives for /usr, /var, /tmp, and so forth). You might also consider compiling your / into the kernel and having it mounted as a MD at runtime. You would not be able to make permanent changes without re-compiling your kernel, but then the unmirrored root partition would never be accessed after boot time, and you would ideally have duplicate copies on each disk to boot from. Doug On Fri, Sep 28, 2001 at 07:45:26PM -0400, Leo Bicknell wrote: > > I need to configure a server that can deal with a disk failure. > I've been looking at CCD and Vinum, but both seem to have issues > that make automatic recovery in the face of one dead disk (in a > mirror) less than optimal. > > So, if you know how to make it so a disk can die and the box keeps > running, and more importantly _can be rebooted_ and keep running > either via careful configuration or some crufty scripts if you > could contact me that would be welcome. No need to keep on-list > unless you think there's something of great value to hackers. > > -- > Leo Bicknell - bicknell@ufp.org > Systems Engineer - Internetworking Engineer - CCIE 3440 > Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 3:27: 7 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from hotmail.com (f55.law3.hotmail.com [209.185.241.55]) by hub.freebsd.org (Postfix) with ESMTP id 4C9A237B40A for ; Sat, 29 Sep 2001 03:27:05 -0700 (PDT) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Sat, 29 Sep 2001 03:27:05 -0700 Received: from 62.248.33.150 by lw3fd.law3.hotmail.msn.com with HTTP; Sat, 29 Sep 2001 10:27:05 GMT X-Originating-IP: [62.248.33.150] From: "dark man" To: hackers@FreeBSD.org Subject: for Mailing List Date: Sat, 29 Sep 2001 10:27:05 +0000 Mime-Version: 1.0 Content-Type: text/html Message-ID: X-OriginalArrivalTime: 29 Sep 2001 10:27:05.0341 (UTC) FILETIME=[49036AD0:01C148D1] Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG
ADD ME...

DARKMAN

Get your FREE download of MSN Explorer at http://explorer.msn.com
To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 4:57: 0 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from milliways.chance.ru (milliways.chance.ru [195.190.107.35]) by hub.freebsd.org (Postfix) with ESMTP id CE4D437B40E for ; Sat, 29 Sep 2001 04:56:49 -0700 (PDT) Received: from do-labs.spb.ru (ppp-5.chance.ru [195.190.107.8]) by milliways.chance.ru (8.9.0/8.9.0) with SMTP id PAA16216 for ; Sat, 29 Sep 2001 15:56:40 +0400 (MSD) Received: (qmail 652 invoked by uid 1000); 29 Sep 2001 15:59:41 -0000 Date: Sat, 29 Sep 2001 15:59:41 +0000 From: Vladimir Dozen To: hackers@FreeBSD.org Subject: VM: dynamic swap remapping (patch) Message-ID: <20010929155941.A291@eix.do-labs.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ehlo. (Sorry for long pre-history, I believe it is necessary.) My current employer develops large CORBA-based data mining servers. They are usually run under HP-UX, but, following the current fashion to build processing farms, I was targeted to build version for free unices. Initial platform was Linux, and build itself was done smoothly, but very soon we were got problem: we use pthreads; to be more precise, we use thread-per-client model. This means that at the same time we may compute from single to a few tens client sessions. Each session may eat as much as 1G of address space, and even more (actually, there is no limits except for hardware ones). The problem was how Linux (and FreeBSD, as we discovered soon) treats out-of-memory (OOM) situation. Under HPUX memory is precommited (i.e., swap is reserved for every allocated page), so as soon as we get into OOM, malloc() or operator new() returns NULL or throws exception, so we have opportunity to unroll stack, tell client we cannot perform his request currently and, most important, are able to continue execution of other clients requests. Linux and FreeBSD simply were killing whole our process and we have no any chance to know we are out of memory! All our data of all our clients (some of them were in processing days before) were lost. :((((( Very unfriendly, and, what can be more important, this kind of interaction (absence of it, really) between OS and application reduces chances of porting really large applications onto FreeBSD due to fact that no one can trust OS that can simply trash user data with no warning. It seems to me, OS must use any chance to continue execution of application instead of killing it. I do think it is Right Way. I have wrote a patch that modifies behaivour (have I spelled this word right? ;) of VM when we are out of memory. Instead of killing largest process, we remap parts of it's address space onto temporal files (exactly as HP-UX does when swapping into dir turned on). Of course, we cannot do it when we absolutely out of swap, we do it a bit early, when swap daemon founds swap free pages lowed to nswap_lowat. I called this patch OOM Keeper as opposite to OOM Killer used in Linux (yah, I prefer BSD). Here is generic algorithm: 1. Swap daemon founds vm_swap_size < nswap_lowat; it calls vm_oomkeeper_swap_almost_full(); 2. vm_oomkeeper_swap_almost_full() searches process having largest vm_object of type OBJT_SWAP, and sends it signal (proposed name: SIGXMEM). 3. process gets signal, and calls special syscall (proposed name: remap). 4. (we are again in kernel, this time curproc is our big process, in vm_oomkeeper_process). while free swap blocks are lower than nswap_hiwat, we do following: a) find largest object of OBJT_SWAP in current process b) create temporal file and unlink() it c) save first 1M of object into file d) cut first 1M of map (here we can get free swap blocks) e) mmap the file onto the place where the data was before. If any of above will fail, then old killproc() will trigger, so system will still be able to drop buggy processes. Note: process now has chance to do something in OOM situation. It can simply ignore signal, and it will be killed soon. It can call remap(), and it will be remapped onto files -- this will slow things down, but will allow to continue processing. It can free some space (e.g., by unmapping anonymous mmap). It can finally save current data and terminate, if nothing of above is acceptable. Note also that ulimits and quota are in action since files are created under process credentials. This patch was tested on my home PC with 64M RAM and 64M swap; I was able to run processes with _committed_ address space up to 512M in various scenarios: large malloc then commit, small incremental mallocs with immediate commit, random commit, parallel run of two or three such memory eaters, etc. No doubts, it requires additional testing. The patch is at whole in separate file -- vm_oomkeeper.c, and it requeres only single intrusion point in current code -- add single line in swap_pager.c:swp_sizechk(). But, to fully implement it, I have to add new signal and new syscall into system. I do not want to go so far until I'll know if my patch acceptable for FreeBSD team. To make it fully controllable it would also be useful to set nswap_{hi,lo}wat via sysctl interface. In any case, when using OOMK these two should be raised about 4 to 8 times (from 400K to 2-4M). It would be also valueable if default action for SIGXMEM would be not SIG_IGN, but calling remap(). This requires patching of libc. Special environment variable ($REMAPDIR) might be used to set location of temporal files. I can send the vm_oomkeeper.c by request (it is 12K long, and I do not want to post it into mail list with no permission). Comments? -- dozen @ home To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 5:10:32 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id 46C2837B409 for ; Sat, 29 Sep 2001 05:10:30 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id EE76881D0B; Sat, 29 Sep 2001 07:10:24 -0500 (CDT) Date: Sat, 29 Sep 2001 07:10:24 -0500 From: Alfred Perlstein To: Vladimir Dozen Cc: hackers@FreeBSD.org Subject: Re: VM: dynamic swap remapping (patch) Message-ID: <20010929071024.Q59854@elvis.mu.org> References: <20010929155941.A291@eix.do-labs.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929155941.A291@eix.do-labs.spb.ru>; from vladimir-dozen@mail.ru on Sat, Sep 29, 2001 at 03:59:41PM +0000 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Vladimir Dozen [010929 06:57] wrote: > ehlo. > > (Sorry for long pre-history, I believe it is necessary.) [snip] > Comments? Wow! This is really awesome work you've done, perhaps you can put the patch up on a URL someplace? If not mail it to me in private and I can put it up for people to see. One thing though, I think that this behaviour should be toggled via a sysctl, but I think I can manage doing that for you. One other question, why not just set an option to make FreeBSD not overcommit? I've always wanted the ability to turn off overcommit for exactly the same reasons you do. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 5:13:58 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from freebie.xs4all.nl (freebie.xs4all.nl [213.84.32.253]) by hub.freebsd.org (Postfix) with ESMTP id 937CF37B407 for ; Sat, 29 Sep 2001 05:13:54 -0700 (PDT) Received: (from wkb@localhost) by freebie.xs4all.nl (8.11.6/8.11.6) id f8TCDni80890; Sat, 29 Sep 2001 14:13:49 +0200 (CEST) (envelope-from wkb) Date: Sat, 29 Sep 2001 14:13:49 +0200 From: Wilko Bulte To: Alfred Perlstein Cc: Vladimir Dozen , hackers@FreeBSD.ORG Subject: Re: VM: dynamic swap remapping (patch) Message-ID: <20010929141349.A80876@freebie.xs4all.nl> References: <20010929155941.A291@eix.do-labs.spb.ru> <20010929071024.Q59854@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929071024.Q59854@elvis.mu.org>; from bright@mu.org on Sat, Sep 29, 2001 at 07:10:24AM -0500 X-OS: FreeBSD 4.4-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Sep 29, 2001 at 07:10:24AM -0500, Alfred Perlstein wrote: > * Vladimir Dozen [010929 06:57] wrote: > > ehlo. > > > > (Sorry for long pre-history, I believe it is necessary.) > [snip] > > Comments? > > Wow! This is really awesome work you've done, perhaps you can put > the patch up on a URL someplace? If not mail it to me in private > and I can put it up for people to see. One thing though, I think > that this behaviour should be toggled via a sysctl, but I think I > can manage doing that for you. > > One other question, why not just set an option to make FreeBSD not > overcommit? I've always wanted the ability to turn off overcommit > for exactly the same reasons you do. FWIW: Tru64 has had this capability since day one. You can select swap-overcommit mode by removing a symlink (/sbin/swapdefault -> /dev/foob) were /dev/foob is the primary swap partition. W/ -- | / o / /_ _ email: wilko@FreeBSD.org |/|/ / / /( (_) Bulte Arnhem, The Netherlands To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 7:48: 3 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from imo-r02.mx.aol.com (imo-r02.mx.aol.com [152.163.225.98]) by hub.freebsd.org (Postfix) with ESMTP id 2FF9237B40B for ; Sat, 29 Sep 2001 07:48:01 -0700 (PDT) Received: from Bsdguru@aol.com by imo-r02.mx.aol.com (mail_out_v31_r1.7.) id 3.f4.100fd30a (16781); Sat, 29 Sep 2001 10:47:54 -0400 (EDT) From: Bsdguru@aol.com Message-ID: Date: Sat, 29 Sep 2001 10:47:54 EDT Subject: Re: ecc on i386 To: ticso@mail.cicely.de Cc: hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: AOL 5.0 for Windows sub 138 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In a message dated 9/28/01 10:21:58 AM Eastern Daylight Time, ticso@mail.cicely.de writes: > > I dont think this is "good". Back in the XT days we used to get a false > > parity error every once on a while on an ISA card...taking the machine > down > > on a bit error (which XTs used to do) was completly wrong and unnecessary. > If > > Haeh - if your memory content has been changed behind you can only hope > that it doesn't trashed some important metadata and won't trash the > whole system. > Well it's much better if you check the use of the memory region and do > some inteligent handling. > But ignoring is definately a very dangerous thing. > Well if you didnt snip out the context you wouldnt have an argument at all, since I said to make it an option to shut down or issue a warning. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 9:10: 7 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from air.linkclub.or.jp (air.linkclub.or.jp [210.250.19.40]) by hub.freebsd.org (Postfix) with ESMTP id 18B9937B40C for ; Sat, 29 Sep 2001 09:09:59 -0700 (PDT) Received: from localhost.jp.FreeBSD.org (1Cust157.tnt1.hanno.jp.da.uu.net [63.12.195.157]) by air.linkclub.or.jp (8.11.4/8.11.4) with ESMTP id f8TG58W28445 for ; Sun, 30 Sep 2001 01:05:10 +0900 (JST) (envelope-from toshi@jp.FreeBSD.org) Date: Sun, 30 Sep 2001 00:27:54 +0900 (JST) Message-Id: <200109291527.f8TFRrU76727.toshi@jp.FreeBSD.org> From: Toshihiko ARAI To: freebsd-hackers@FreeBSD.org Subject: Doubt of system(3) X-Mailer: VM 5.96 (beta) / Mule 2.3 (SUETSUMUHANA) based on 19.34.1 Mime-Version: 1.0 (generated by tm-edit 7.106) Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I consider the following code of system(3). pid is changed by return value of _wait4(). I feel this need a correction. default: /* parent */ do { pid = _wait4(pid, &pstat, 0, (struct rusage *)0); } while (pid == -1 && errno == EINTR); break; Please review and commit this patch. Index: src/lib/libc/stdlib/system.c =================================================================== RCS file: /home/ncvs/src/lib/libc/stdlib/system.c,v retrieving revision 1.7 diff -u -r1.7 system.c --- src/lib/libc/stdlib/system.c 2001/01/24 13:00:59 1.7 +++ src/lib/libc/stdlib/system.c 2001/09/29 08:53:14 @@ -86,9 +86,9 @@ execl(_PATH_BSHELL, "sh", "-c", command, (char *)NULL); _exit(127); default: /* parent */ - do { - pid = _wait4(pid, &pstat, 0, (struct rusage *)0); - } while (pid == -1 && errno == EINTR); + while (_wait4(pid, &pstat, 0, (struct rusage *)0) == -1 && + errno == EINTR) + ; /* nothing */ break; } (void)_sigaction(SIGINT, &intact, NULL); -- Toshihiko ARAI To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 9:53:39 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 339ED37B40E for ; Sat, 29 Sep 2001 09:53:36 -0700 (PDT) Received: (from dillon@localhost) by earth.backplane.com (8.11.6/8.11.2) id f8TGrRR37689; Sat, 29 Sep 2001 09:53:27 -0700 (PDT) (envelope-from dillon) Date: Sat, 29 Sep 2001 09:53:27 -0700 (PDT) From: Matt Dillon Message-Id: <200109291653.f8TGrRR37689@earth.backplane.com> To: Wilko Bulte Cc: Alfred Perlstein , Vladimir Dozen , hackers@FreeBSD.ORG Subject: Re: VM: dynamic swap remapping (patch) References: <20010929155941.A291@eix.do-labs.spb.ru> <20010929071024.Q59854@elvis.mu.org> <20010929141349.A80876@freebie.xs4all.nl> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :> overcommit? I've always wanted the ability to turn off overcommit :> for exactly the same reasons you do. : :FWIW: Tru64 has had this capability since day one. You can select :swap-overcommit mode by removing a symlink (/sbin/swapdefault -> /dev/foob) :were /dev/foob is the primary swap partition. : :W/ : :-- :| / o / /_ _ email: wilko@FreeBSD.org :|/|/ / / /( (_) Bulte Arnhem, The Netherlands Well, the overcommit argument comes up once or twice a year. Frankly I don't see much of a point to it. While it is true that you could implement a signal the plain fact of the matter is that having to deal with the possibility in a program at the N points (generally hundreds of points) where that program allocates memory, either directly or indirectly, virtually guarentees that you will introduce bugs into the system. You also cannot guarentee that your process will have time to cleanup prior to the system killing, nor can you guarentee that all the standard system utilities and daemons will be able to gracefully handle the out of memory condition. In otherwords, you could implement the signal and even have the program use it, but you will still likely leave gaping holes in the implementation that will result in lost data. It is much easier to manage memory manually. For example, if these programs require 1G of independant memory to run it ought to be a fairly simple matter to simply create a 1GB file for each process (using dd rather then ftruncate() to create the file so the blocks are preallocated), mmap() it using PROT_READ|PROT_WRITE, MAP_SHARED|MAP_NOSYNC, and do your memory management out of that. The memory space will be backed by the file rather then by swap. You get all the benefits of the standard overcommit capabilities of the system as well as the ability to pre-reserve the main workspace for the programs and you automatically get persistent storage for the data. Problem solved. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 9:56:40 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id 5CC3237B401 for ; Sat, 29 Sep 2001 09:56:37 -0700 (PDT) Received: from mail.cicely.de (cicely20 [10.1.1.22]) by srv1.cosmo-project.de (8.11.0/8.11.0) with ESMTP id f8TGuZ627641; Sat, 29 Sep 2001 18:56:35 +0200 (CEST) Received: (from ticso@localhost) by mail.cicely.de (8.11.0/8.11.0) id f8TGvEA16217; Sat, 29 Sep 2001 18:57:14 +0200 (CEST) Date: Sat, 29 Sep 2001 18:57:13 +0200 From: Bernd Walter To: Bsdguru@aol.com Cc: ticso@mail.cicely.de, hackers@FreeBSD.ORG Subject: Re: ecc on i386 Message-ID: <20010929185712.A16031@cicely20.cicely.de> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from Bsdguru@aol.com on Sat, Sep 29, 2001 at 10:47:54AM -0400 X-Operating-System: NetBSD cicely20.cicely.de 1.5 sparc Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Sep 29, 2001 at 10:47:54AM -0400, Bsdguru@aol.com wrote: > Well if you didnt snip out the context you wouldnt have an argument at all, > since I said to make it an option to shut down or issue a warning. You mean this: sysctl machdep.panic_on_nmi But I still beleave it's not a good idea to change behavour. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 10:20:56 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mta10.onebox.com (mta10.onebox.com [64.68.76.184]) by hub.freebsd.org (Postfix) with ESMTP id D142A37B40F for ; Sat, 29 Sep 2001 10:20:53 -0700 (PDT) Received: from onebox.com ([10.1.111.8]) by mta10.onebox.com (InterMail vM.4.01.03.23 201-229-121-123-20010418) with SMTP id <20010929172053.XCRL16495.mta10.onebox.com@onebox.com> for ; Sat, 29 Sep 2001 10:20:53 -0700 Received: from [66.32.0.155] by onebox.com with HTTP; Sat, 29 Sep 2001 10:20:53 -0700 Date: Sat, 29 Sep 2001 10:20:53 -0700 Subject: Problem(s) With FreeBSD-Current From: "Glenn Gombert" To: hackers@FreeBSD.org Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit MIME-Version: 1.0 Message-Id: <20010929172053.XCRL16495.mta10.onebox.com@onebox.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, I have tried to compile FreeBSD Current (several times) over the last couple of days and get the following message when trying to run XFree86 and a number of other applications: /user/lib/libpam.so.1 : Undefined symbol "__stdoutp"... Does anyone else have the same problem ?? Thanks, -- Glenn Gombert glenngombert@onebox.com - email (513) 587-2643 x2263 - voicemail/fax __________________________________________________ FREE voicemail, email, and fax...all in one place. Sign Up Now! http://www.onebox.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 11:39:23 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from netbank.com.br (garrincha.netbank.com.br [200.203.199.88]) by hub.freebsd.org (Postfix) with ESMTP id 26C2237B403 for ; Sat, 29 Sep 2001 11:39:14 -0700 (PDT) Received: from 1-011.ctame701-1.telepar.net.br (1-011.ctame701-1.telepar.net.br [200.181.137.11]) by netbank.com.br (Postfix) with ESMTP id 617F84680C; Sat, 29 Sep 2001 15:34:15 -0300 (BRST) Received: (from localhost user: 'riel', uid#500) by imladris.surriel.com with ESMTP id ; Sat, 29 Sep 2001 15:38:28 -0300 Date: Sat, 29 Sep 2001 15:38:23 -0300 (BRST) From: Rik van Riel X-X-Sender: To: Vladimir Dozen Cc: Subject: Re: VM: dynamic swap remapping (patch) In-Reply-To: <20010929155941.A291@eix.do-labs.spb.ru> Message-ID: X-spambait: aardvark@kernelnewbies.org X-spammeplease: aardvark@nl.linux.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 29 Sep 2001, Vladimir Dozen wrote: > I have wrote a patch that modifies behaivour (have I spelled this > word right? ;) of VM when we are out of memory. Instead of killing > largest process, we remap parts of it's address space onto temporal > files (exactly as HP-UX does when swapping into dir turned on). This is not instead of killing, this is just a way to delay the killing of processes longer. Once your disk is full you'd still run into the choice between a deadlock and a kill... It's an awesome way of delaying the out of memory problem, though, because a suspended application won't be able to allocate anything more, giving the system a better chance to let the running apps run to completion. Alternatively, the one leaky application is suspended and the rest of the system continues to run without any problems. In short, I like it ;) regards, Rik -- IA64: a worthy successor to i860. http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to aardvark@nl.linux.org (spam digging piggy) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 12: 0:20 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from harrier.mail.pas.earthlink.net (harrier.mail.pas.earthlink.net [207.217.121.12]) by hub.freebsd.org (Postfix) with ESMTP id 9851637B40C for ; Sat, 29 Sep 2001 12:00:16 -0700 (PDT) Received: from mindspring.com (dialup-209.245.140.105.Dial1.SanJose1.Level3.net [209.245.140.105]) by harrier.mail.pas.earthlink.net (EL-8_9_3_3/8.9.3) with ESMTP id MAA19906; Sat, 29 Sep 2001 12:00:09 -0700 (PDT) Message-ID: <3BB61A6B.A97E955A@mindspring.com> Date: Sat, 29 Sep 2001 12:00:59 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Kenneth D. Merry" Cc: Jonathan Lemon , the_srinivas@hotmail.com, hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> <20010928074347.A36960@panzer.kdm.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Kenneth D. Merry" wrote: [ ... transmit checksum offload ... ] > You've got things confused. I think that may be a limitation of some > SysKonnect boards, but certainly isn't a Tigon limitation. Yes, it's not Tigon chipset specific. > Tigon boards come with 512KB, 1MB, or 2MB (never seen one of these) of SRAM > on board. The firmware takes a couple hundred KB, so you generally have at > least 200KB, and possibly a good bit more for packet storage space on the > card. > > So there is buffer space for a number of packets on both the send and > receive sides, and checksumming works in both directions. This depends on the driver division of memory; fortunately, the FreeBSD driver is OK in this regard. > > Also note that this card is not known to be the fastest > > one ot there, so you may be better off doing the checksum > > on your 1GHz+ host processor instead. > > You can get wire rate with a Tigon 2 board, with jumbo frames at least. > (That's with checksum offloading enabled.) Unfortunately, it can not correctly interoperate with a number of cards in jumbogram mode, so unless you know the card on the other end and manually configure it (it can't negotiate properly), you can't really use jumbograms. Or you could rewrite the firmware (of course). I have never seen the board with the stock FreeBSD driver get better than about half wire rate with normal size packets and window depth. On the other hand, the Tigon III is capable of 960 megabits -- about the wire rate limit -- with normal size packets, if you implement software interrupt coelescing (which doesn't help, unless you crank the load up to close to wire speed and/or do more of the stack processing at interrupt time). The Tigon II also has the unfortunate need to download its firmware, and a FreeBSD bug -- the lack of a distinct state for firmware download -- means that the firmware get sent to the card each time the thing is config'ed up, or an IP alias is added to the thing. Plus the damn things are more expensive than the Tigon III based cards. To put a final nail in the coffin, the card can't offload TCP checksum with VLANs enabled (it breaks). All in all, the Tigon II is a bad board. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 12:38:42 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from milliways.chance.ru (milliways.chance.ru [195.190.107.35]) by hub.freebsd.org (Postfix) with ESMTP id EF1C537B40A for ; Sat, 29 Sep 2001 12:38:35 -0700 (PDT) Received: from do-labs.spb.ru (ppp-3.chance.ru [195.190.107.6]) by milliways.chance.ru (8.9.0/8.9.0) with SMTP id XAA24301 for ; Sat, 29 Sep 2001 23:38:25 +0400 (MSD) Received: (qmail 670 invoked by uid 1000); 29 Sep 2001 23:29:53 -0000 Date: Sat, 29 Sep 2001 23:29:53 +0000 From: Vladimir Dozen To: Matt Dillon Cc: Wilko Bulte , Alfred Perlstein , hackers@FreeBSD.ORG Subject: Re: VM: dynamic swap remapping (patch) Message-ID: <20010929232953.B341@eix.do-labs.spb.ru> References: <20010929155941.A291@eix.do-labs.spb.ru> <20010929071024.Q59854@elvis.mu.org> <20010929141349.A80876@freebie.xs4all.nl> <200109291653.f8TGrRR37689@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <200109291653.f8TGrRR37689@earth.backplane.com>; from dillon@earth.backplane.com on Sat, Sep 29, 2001 at 09:53:27AM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ehlo. > You also cannot guarentee that your process will have time to > cleanup prior to the system killing, nor can you guarentee that all the > standard system utilities and daemons will be able to gracefully handle > the out of memory condition. In otherwords, you could implement > the signal and even have the program use it, but you will still likely > leave gaping holes in the implementation that will result in lost data. Actually, the things as I coded them better suited namely for poorly written daemons that never check for malloc result. Precommit will just kill them as soon as malloc() will return NULL, and they dereference it. Killproc() will kill them too. Remapping will save them. Disk space now is large enough to make them live till root will notice that they grow to much and do something (kill them manually, probably ;). > It is much easier to manage memory manually. For example, if these > programs require 1G of independant memory to run it ought to be a > fairly simple matter to simply create a 1GB file for each process > (using dd rather then ftruncate() to create the file so the blocks are > preallocated), mmap() it using PROT_READ|PROT_WRITE,MAP_SHARED|MAP_NOSYNC, > and do your memory management out of that. First at all, it is NOT easier. Doing own memory management is not too simple, especially for threads and SMP -- we seen 50% performance impact when two threads on two processors were doing intensive allocations (it was not FreeBSD, and these was kernel threads). Second, application not always grows to 1G, most of the time it keeps as small as 500M ;). Why should we precommit 1G for 500M data? Doing multi-mmap memory management is additional pain. Third, swapping to device is faster, and, while we have enough swap, I would prefer to swap there. Even a few percent for 5-day computation make sense. > Problem solved. If I'm the developer -- probably, yes. What if I'm system administrator, and has to run something large _and important_? The day I'll notice that monster creates swap files I'll know I have to add RAM. I will have time since it still works, it was not killed. P.S. Anyway, I do NOT insist my solution is better, and even that it is good for anything at all. It was fun for me to hack in BSD kernel, and it was interesting challenge, and I feel need to share results with others. At worst, I will recommend our customer to setup processing farm under FreeBSD with applied patch. -- dozen @ home To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 13:34:36 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id EA82837B405 for ; Sat, 29 Sep 2001 13:34:33 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id DE8D981D0B; Sat, 29 Sep 2001 15:34:33 -0500 (CDT) Date: Sat, 29 Sep 2001 15:34:33 -0500 From: Alfred Perlstein To: Toshihiko ARAI Cc: freebsd-hackers@FreeBSD.org Subject: Re: Doubt of system(3) Message-ID: <20010929153433.U59854@elvis.mu.org> References: <200109291527.f8TFRrU76727.toshi@jp.FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200109291527.f8TFRrU76727.toshi@jp.FreeBSD.org>; from toshi@jp.FreeBSD.org on Sun, Sep 30, 2001 at 12:27:54AM +0900 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Toshihiko ARAI [010929 11:10] wrote: > I consider the following code of system(3). pid is changed by return > value of _wait4(). I feel this need a correction. > > default: /* parent */ > do { > pid = _wait4(pid, &pstat, 0, (struct rusage *)0); > } while (pid == -1 && errno == EINTR); > break; > > Please review and commit this patch. Why does it need to be corrected? What sort of bad behaviour are you seeing? You do 'a' and you see 'b' when you should see 'c'. What's a, b and c? -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 13:37: 1 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id 1B1A937B40B for ; Sat, 29 Sep 2001 13:37:00 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id 117B481D0B; Sat, 29 Sep 2001 15:37:00 -0500 (CDT) Date: Sat, 29 Sep 2001 15:37:00 -0500 From: Alfred Perlstein To: Glenn Gombert Cc: hackers@FreeBSD.org Subject: Re: Problem(s) With FreeBSD-Current Message-ID: <20010929153700.V59854@elvis.mu.org> References: <20010929172053.XCRL16495.mta10.onebox.com@onebox.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929172053.XCRL16495.mta10.onebox.com@onebox.com>; from glenngombert@onebox.com on Sat, Sep 29, 2001 at 10:20:53AM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Glenn Gombert [010929 12:21] wrote: > Hello, > > I have tried to compile FreeBSD Current (several times) over the last > couple of days and get the following message when trying to run XFree86 > and a number of other applications: > > /user/lib/libpam.so.1 : Undefined symbol "__stdoutp"... > > Does anyone else have the same problem ?? Yes, a lot of people. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 13:38:44 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from ntlg.sibnet.ru (dns.sibnet.ru [217.70.96.34]) by hub.freebsd.org (Postfix) with ESMTP id 4A2A237B408 for ; Sat, 29 Sep 2001 13:38:41 -0700 (PDT) Received: from tlg5-ppp35.sibnet.ru (tlg5-ppp35.sibnet.ru [217.70.97.36]) by ntlg.sibnet.ru (8.9.3+Sun/8.9.3) with ESMTP id AAA29649; Sun, 30 Sep 2001 00:38:24 +0400 (MSD) Date: Sun, 30 Sep 2001 03:39:02 +0700 (NOVST) From: "Semen A. Ustimenko" X-Sender: semenu@default To: Matt Dillon Cc: hackers@FreeBSD.org, Alfred Perlstein , Bruce Evans , Poul-Henning Kamp , Julian Elischer Subject: Re: VM Corruption - stumped, anyone have any ideas? In-Reply-To: <200109241914.f8OJE4l95477@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi! On Mon, 24 Sep 2001, Matt Dillon wrote: > A number of people have been seeing these on STABLE: > > panic: vm_page_remove(): page not found in hash > There was rummors that lowering "maxusers" from 512 to 128 worksaround this problem on a machine with 3Gb of memory... Bye! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 13:43:48 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.yadt.co.uk (yadt.demon.co.uk [158.152.4.134]) by hub.freebsd.org (Postfix) with SMTP id F143D37B40B for ; Sat, 29 Sep 2001 13:43:41 -0700 (PDT) Received: (qmail 3610 invoked from network); 29 Sep 2001 20:43:39 -0000 Received: from gattaca.local.yadt.co.uk (HELO mail.gattaca.yadt.co.uk) (qmailr@10.0.0.2) by xfiles.yadt.co.uk with SMTP; 29 Sep 2001 20:43:39 -0000 Received: (qmail 57979 invoked by uid 1000); 29 Sep 2001 20:43:38 -0000 Date: Sat, 29 Sep 2001 21:43:38 +0100 From: David Taylor To: freebsd-hackers@FreeBSD.org Subject: Re: Doubt of system(3) Message-ID: <20010929214338.A57903@gattaca.yadt.co.uk> Mail-Followup-To: freebsd-hackers@FreeBSD.org References: <200109291527.f8TFRrU76727.toshi@jp.FreeBSD.org> <20010929153433.U59854@elvis.mu.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="vtzGhvizbBRQ85DL" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929153433.U59854@elvis.mu.org>; from bright@mu.org on Sat, Sep 29, 2001 at 15:34:33 -0500 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --vtzGhvizbBRQ85DL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, 29 Sep 2001, Alfred Perlstein wrote: > * Toshihiko ARAI [010929 11:10] wrote: > > I consider the following code of system(3). pid is changed by return > > value of _wait4(). I feel this need a correction. > >=20 > > default: /* parent */ > > do { > > pid =3D _wait4(pid, &pstat, 0, (struct rusage *= )0); > > } while (pid =3D=3D -1 && errno =3D=3D EINTR); > > break; > >=20 > > Please review and commit this patch. >=20 > Why does it need to be corrected? What sort of bad behaviour > are you seeing? You do 'a' and you see 'b' when you should > see 'c'. >=20 > What's a, b and c? >=20 Well, hypothetically (I have no time to attempt to set something up to test this), it looks to me like: If you: 1. Fork, and create a child (say, pid 10) 2. Call system, which forks and creates a child (say, pid 11) 3. Make the child (pid 10) exit now. 3. Interrupt the call to _wait4(pid [=3D11], ...); You _should_ see nothing happen. i.e. system() would simply loop again until the other child exits. However, what I think you'd see is system() setting pid =3D=3D -1, calling _wait4(-1, ...); and _wait4() returning 10, and system() returning whilst your child is still running happily away. If I run out of bugs to squash in ircd, I might try to produce this (I suppose reproduce is the wrong word, since no one has reported it in the first place...) --=20 David Taylor davidt@yadt.co.uk --vtzGhvizbBRQ85DL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7tjJ6fIqKXSsJ/xERAmxIAJ9L/wXrlMIVp8+OOzXNvcXL7nZsGwCeLQdC /8JrSSOGQ0YSKudT/LDA7wI= =ecRM -----END PGP SIGNATURE----- --vtzGhvizbBRQ85DL-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 14: 5: 3 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id A99F237B40C for ; Sat, 29 Sep 2001 14:05:00 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id 9890281D0B; Sat, 29 Sep 2001 16:05:00 -0500 (CDT) Date: Sat, 29 Sep 2001 16:05:00 -0500 From: Alfred Perlstein To: David Taylor Cc: freebsd-hackers@FreeBSD.org Subject: Re: Doubt of system(3) Message-ID: <20010929160500.X59854@elvis.mu.org> References: <200109291527.f8TFRrU76727.toshi@jp.FreeBSD.org> <20010929153433.U59854@elvis.mu.org> <20010929214338.A57903@gattaca.yadt.co.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929214338.A57903@gattaca.yadt.co.uk>; from davidt@yadt.co.uk on Sat, Sep 29, 2001 at 09:43:38PM +0100 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * David Taylor [010929 15:44] wrote: > On Sat, 29 Sep 2001, Alfred Perlstein wrote: > > * Toshihiko ARAI [010929 11:10] wrote: > > > I consider the following code of system(3). pid is changed by return > > > value of _wait4(). I feel this need a correction. > > > > > > default: /* parent */ > > > do { > > > pid = _wait4(pid, &pstat, 0, (struct rusage *)0); > > > } while (pid == -1 && errno == EINTR); > > > break; > > > > > > Please review and commit this patch. > > > > Why does it need to be corrected? What sort of bad behaviour > > are you seeing? You do 'a' and you see 'b' when you should > > see 'c'. > > > > What's a, b and c? > > > > Well, hypothetically (I have no time to attempt to set something up to test > this), it looks to me like: > > If you: > > 1. Fork, and create a child (say, pid 10) > 2. Call system, which forks and creates a child (say, pid 11) > 3. Make the child (pid 10) exit now. > 3. Interrupt the call to _wait4(pid [=11], ...); > > You _should_ see nothing happen. i.e. system() would simply loop again > until the other child exits. > > However, what I think you'd see is system() setting pid == -1, calling > _wait4(-1, ...); and _wait4() returning 10, and system() returning whilst > your child is still running happily away. > > If I run out of bugs to squash in ircd, I might try to produce this (I > suppose reproduce is the wrong word, since no one has reported it in the > first place...) Yes, makes sense. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 14:17:36 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.yadt.co.uk (yadt.demon.co.uk [158.152.4.134]) by hub.freebsd.org (Postfix) with SMTP id DC9A937B407 for ; Sat, 29 Sep 2001 14:17:00 -0700 (PDT) Received: (qmail 3751 invoked from network); 29 Sep 2001 21:16:58 -0000 Received: from gattaca.local.yadt.co.uk (HELO mail.gattaca.yadt.co.uk) (qmailr@10.0.0.2) by xfiles.yadt.co.uk with SMTP; 29 Sep 2001 21:16:58 -0000 Received: (qmail 59977 invoked by uid 1000); 29 Sep 2001 21:16:58 -0000 Date: Sat, 29 Sep 2001 22:16:58 +0100 From: David Taylor To: freebsd-hackers@FreeBSD.org Subject: Re: Doubt of system(3) Message-ID: <20010929221658.B57903@gattaca.yadt.co.uk> Mail-Followup-To: freebsd-hackers@FreeBSD.org References: <200109291527.f8TFRrU76727.toshi@jp.FreeBSD.org> <20010929153433.U59854@elvis.mu.org> <20010929214338.A57903@gattaca.yadt.co.uk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="EuxKj2iCbKjpUGkD" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929214338.A57903@gattaca.yadt.co.uk>; from davidt@yadt.co.uk on Sat, Sep 29, 2001 at 21:43:38 +0100 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --EuxKj2iCbKjpUGkD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, 29 Sep 2001, David Taylor wrote: > On Sat, 29 Sep 2001, Alfred Perlstein wrote: > >=20 > > Why does it need to be corrected? What sort of bad behaviour > > are you seeing? You do 'a' and you see 'b' when you should > > see 'c'. > >=20 > > What's a, b and c? > >=20 >=20 > Well, hypothetically (I have no time to attempt to set something up to te= st > this), it looks to me like: >=20 > If you: >=20 > 1. Fork, and create a child (say, pid 10) > 2. Call system, which forks and creates a child (say, pid 11) > 3. Make the child (pid 10) exit now. > 3. Interrupt the call to _wait4(pid [=3D11], ...); >=20 I've now managed to reproduce this... Test program at http://www.yadt.demon.co.uk/system-bug.tar.gz I'm pretty sure that's incorrect behaviour, and I also beleive the original patch posted to this list should fix it... --=20 David Taylor davidt@yadt.co.uk --EuxKj2iCbKjpUGkD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7tjpJfIqKXSsJ/xERAlePAJ9bWY/y+K7k+tn3goQYc+D/pcPpwgCgx3RP 7pfD9QO69643fLyHrUUDCWA= =DXpd -----END PGP SIGNATURE----- --EuxKj2iCbKjpUGkD-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 14:39:50 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from panzer.kdm.org (panzer.kdm.org [216.160.178.169]) by hub.freebsd.org (Postfix) with ESMTP id 0B1FA37B40C for ; Sat, 29 Sep 2001 14:39:44 -0700 (PDT) Received: (from ken@localhost) by panzer.kdm.org (8.11.6/8.9.1) id f8TLdW350867; Sat, 29 Sep 2001 15:39:32 -0600 (MDT) (envelope-from ken) Date: Sat, 29 Sep 2001 15:39:32 -0600 From: "Kenneth D. Merry" To: Terry Lambert Cc: Jonathan Lemon , the_srinivas@hotmail.com, hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 Message-ID: <20010929153932.A50715@panzer.kdm.org> References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> <20010928074347.A36960@panzer.kdm.org> <3BB61A6B.A97E955A@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <3BB61A6B.A97E955A@mindspring.com>; from tlambert2@mindspring.com on Sat, Sep 29, 2001 at 12:00:59PM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Sep 29, 2001 at 12:00:59 -0700, Terry Lambert wrote: > "Kenneth D. Merry" wrote: > [ ... transmit checksum offload ... ] > > > You've got things confused. I think that may be a limitation of some > > SysKonnect boards, but certainly isn't a Tigon limitation. > > Yes, it's not Tigon chipset specific. It's not a Tigon chipset problem at all. > > Tigon boards come with 512KB, 1MB, or 2MB (never seen one of these) of SRAM > > on board. The firmware takes a couple hundred KB, so you generally have at > > least 200KB, and possibly a good bit more for packet storage space on the > > card. > > > > So there is buffer space for a number of packets on both the send and > > receive sides, and checksumming works in both directions. > > This depends on the driver division of memory; fortunately, > the FreeBSD driver is OK in this regard. I suppose if you set the transmit/receive buffer ratio all the way to one side or the other, you'd have problems, but any driver writer with half a clue wouldn't do that. > > > Also note that this card is not known to be the fastest > > > one ot there, so you may be better off doing the checksum > > > on your 1GHz+ host processor instead. > > > > You can get wire rate with a Tigon 2 board, with jumbo frames at least. > > (That's with checksum offloading enabled.) > > Unfortunately, it can not correctly interoperate with a > number of cards in jumbogram mode, so unless you know the > card on the other end and manually configure it (it can't > negotiate properly), you can't really use jumbograms. Or > you could rewrite the firmware (of course). Huh? Care to explain? Why wouldn't it interoperate with jumbo frames? Why are jumbo frames something that need be negotiated by the card? They're not negotiated by the card, but rather by the TCP stack. > I have never seen the board with the stock FreeBSD driver > get better than about half wire rate with normal size > packets and window depth. Yep, it doesn't handle large numbers of small packets very well. You can easily get wire rate with jumbo frames, though. > On the other hand, the Tigon III > is capable of 960 megabits -- about the wire rate limit -- > with normal size packets, if you implement software interrupt > coelescing (which doesn't help, unless you crank the load up > to close to wire speed and/or do more of the stack processing > at interrupt time). > > The Tigon II also has the unfortunate need to download its > firmware, and a FreeBSD bug -- the lack of a distinct state > for firmware download -- means that the firmware get sent > to the card each time the thing is config'ed up, or an IP > alias is added to the thing. Plus the damn things are more > expensive than the Tigon III based cards. That's partially because Bill wanted to have a process context when he was downloading the firmware, and didn't want to do a kernel thread to do it. So it gets done when the card is ifconfiged up. I wrote some code for my former employer to wait up to 10 seconds for link to show up when the board is configured. The alternative is a 'sleep 5' (for fiber boards) or 'sleep 8' (for copper boards) after it is ifconfiged up. Having downloadable firmware is actually a huge advantage. You can do things with the Tigon II that just aren't possible with any other chip, either because they don't have downloadable firmware, or because the vendor won't release firmware source. This is a problem with the Broadcom Tigon III boards, and to some extent with the Tigon II. Basically it looks like the firmware for the Tigon II is very hard to get now that 3Com has control of it, and I don't think Broadcom will release the Tigon III firmware. (Assuming it is a firmware-based chip.) > To put a final nail in the coffin, the card can't offload > TCP checksum with VLANs enabled (it breaks). Probably something that could be fixed in the firmware. Too bad the vendor isn't really supporting it very much now. > All in all, the Tigon II is a bad board. That's your opinion, but I disagree. Ken -- Kenneth Merry ken@kdm.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 15: 3:41 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id C9C7C37B409 for ; Sat, 29 Sep 2001 15:03:37 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id A8A2281D06; Sat, 29 Sep 2001 17:03:32 -0500 (CDT) Date: Sat, 29 Sep 2001 17:03:32 -0500 From: Alfred Perlstein To: David Taylor Cc: freebsd-hackers@FreeBSD.org Subject: Re: Doubt of system(3) Message-ID: <20010929170332.Y59854@elvis.mu.org> References: <200109291527.f8TFRrU76727.toshi@jp.FreeBSD.org> <20010929153433.U59854@elvis.mu.org> <20010929214338.A57903@gattaca.yadt.co.uk> <20010929221658.B57903@gattaca.yadt.co.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929221658.B57903@gattaca.yadt.co.uk>; from davidt@yadt.co.uk on Sat, Sep 29, 2001 at 10:16:58PM +0100 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * David Taylor [010929 16:17] wrote: > On Sat, 29 Sep 2001, David Taylor wrote: > > On Sat, 29 Sep 2001, Alfred Perlstein wrote: > > > > > > Why does it need to be corrected? What sort of bad behaviour > > > are you seeing? You do 'a' and you see 'b' when you should > > > see 'c'. > > > > > > What's a, b and c? > > > > > > > Well, hypothetically (I have no time to attempt to set something up to test > > this), it looks to me like: > > > > If you: > > > > 1. Fork, and create a child (say, pid 10) > > 2. Call system, which forks and creates a child (say, pid 11) > > 3. Make the child (pid 10) exit now. > > 3. Interrupt the call to _wait4(pid [=11], ...); > > > > I've now managed to reproduce this... > > Test program at http://www.yadt.demon.co.uk/system-bug.tar.gz > > I'm pretty sure that's incorrect behaviour, and I also beleive the original > patch posted to this list should fix it... I should have realized that, especially since I just spent a week trying to track down the same problem... *smacks head* The posted patch is ok, except for the fact that it returns successful if an rfork thread has reaped the child, another variation is here, which returns an error ECHILD if someone has reaped our spawed child, and also keeps EFAULT propogation although that's not going to happen because of the arguments being on the stack and NULL. What do you think? Should we silently mask ECHILD? what if wait4() returns a different error code in the future? Shouldn't we propogate that? cvs diff: Diffing . Index: system.c =================================================================== RCS file: /home/ncvs/src/lib/libc/stdlib/system.c,v retrieving revision 1.7 diff -u -r1.7 system.c --- system.c 2001/01/24 13:00:59 1.7 +++ system.c 2001/09/29 21:55:41 @@ -53,7 +53,7 @@ __system(command) const char *command; { - pid_t pid; + pid_t pid, savedpid; int pstat; struct sigaction ign, intact, quitact; sigset_t newsigblock, oldsigblock; @@ -86,8 +86,9 @@ execl(_PATH_BSHELL, "sh", "-c", command, (char *)NULL); _exit(127); default: /* parent */ + savedpid = pid; do { - pid = _wait4(pid, &pstat, 0, (struct rusage *)0); + pid = _wait4(savedpid, &pstat, 0, (struct rusage *)0); } while (pid == -1 && errno == EINTR); break; } -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 15:28:53 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.yadt.co.uk (yadt.demon.co.uk [158.152.4.134]) by hub.freebsd.org (Postfix) with SMTP id CCC5037B40B for ; Sat, 29 Sep 2001 15:28:42 -0700 (PDT) Received: (qmail 4036 invoked from network); 29 Sep 2001 22:28:40 -0000 Received: from gattaca.local.yadt.co.uk (HELO mail.gattaca.yadt.co.uk) (qmailr@10.0.0.2) by xfiles.yadt.co.uk with SMTP; 29 Sep 2001 22:28:40 -0000 Received: (qmail 3960 invoked by uid 1000); 29 Sep 2001 22:28:39 -0000 Date: Sat, 29 Sep 2001 23:28:39 +0100 From: David Taylor To: Alfred Perlstein Cc: freebsd-hackers@FreeBSD.org Subject: Re: Doubt of system(3) Message-ID: <20010929232839.A91153@gattaca.yadt.co.uk> Mail-Followup-To: Alfred Perlstein , freebsd-hackers@FreeBSD.org References: <200109291527.f8TFRrU76727.toshi@jp.FreeBSD.org> <20010929153433.U59854@elvis.mu.org> <20010929214338.A57903@gattaca.yadt.co.uk> <20010929221658.B57903@gattaca.yadt.co.uk> <20010929170332.Y59854@elvis.mu.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="ew6BAiZeqk4r7MaW" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929170332.Y59854@elvis.mu.org>; from bright@mu.org on Sat, Sep 29, 2001 at 17:03:32 -0500 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --ew6BAiZeqk4r7MaW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, 29 Sep 2001, Alfred Perlstein wrote: > * David Taylor [010929 16:17] wrote: > > On Sat, 29 Sep 2001, David Taylor wrote: > > >=20 > > > If you: > > >=20 > > > 1. Fork, and create a child (say, pid 10) > > > 2. Call system, which forks and creates a child (say, pid 11) > > > 3. Make the child (pid 10) exit now. > > > 3. Interrupt the call to _wait4(pid [=3D11], ...); > > >=20 > >=20 > > I've now managed to reproduce this... > >=20 > > Test program at http://www.yadt.demon.co.uk/system-bug.tar.gz > >=20 > > I'm pretty sure that's incorrect behaviour, and I also beleive the orig= inal > > patch posted to this list should fix it... >=20 > I should have realized that, especially since I just spent a week > trying to track down the same problem... >=20 > *smacks head* >=20 > The posted patch is ok, except for the fact that it returns successful > if an rfork thread has reaped the child, another variation is here, > which returns an error ECHILD if someone has reaped our spawed > child, and also keeps EFAULT propogation although that's not going > to happen because of the arguments being on the stack and NULL. >=20 > What do you think? Should we silently mask ECHILD? what if wait4() > returns a different error code in the future? Shouldn't we propogate > that? Well, system(3) claims system() will only return errors from invoking fork() or waitpid() [which, being pedantic, it doesnt actually invoke]... Errors from fork() are just passed straight through, so that's ok... As for wait4(), returning ECHILD from system() doesn't make much sense to m= e. We obviously _did_ have a child at some point, and if it went away before our wait4() call because someone else called wait4() or whatever, I don't think we should complain. If, somehow, EFAULT is returned, it should probably get propagated. However, on second thoughts, since system() is supposed to return the child's exit status, I'm not sure what to do with ECHILD other than just propagate it back to the program, since the exit status has already been given to some other program, and there's no way we can get it. So we either lie about the exit status and possibly break something that relies on that, or return a rather unexpected error. IMO, the below patch is probably the best solution. =20 > cvs diff: Diffing . > Index: system.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/ncvs/src/lib/libc/stdlib/system.c,v > retrieving revision 1.7 > diff -u -r1.7 system.c > --- system.c 2001/01/24 13:00:59 1.7 > +++ system.c 2001/09/29 21:55:41 > @@ -53,7 +53,7 @@ > __system(command) > const char *command; > { > - pid_t pid; > + pid_t pid, savedpid; > int pstat; > struct sigaction ign, intact, quitact; > sigset_t newsigblock, oldsigblock; > @@ -86,8 +86,9 @@ > execl(_PATH_BSHELL, "sh", "-c", command, (char *)NULL); > _exit(127); > default: /* parent */ > + savedpid =3D pid; > do { > - pid =3D _wait4(pid, &pstat, 0, (struct rusage *)0); > + pid =3D _wait4(savedpid, &pstat, 0, (struct rusage *)0); > } while (pid =3D=3D -1 && errno =3D=3D EINTR); > break; > } >=20 --=20 David Taylor davidt@yadt.co.uk --ew6BAiZeqk4r7MaW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7tksXfIqKXSsJ/xERAtj4AJ0fIyOLlyU5zF2wOK2N/xnsM0NOwQCgxL/h Z7WwPSc7CtoEGg1VjZHAgKU= =JBQU -----END PGP SIGNATURE----- --ew6BAiZeqk4r7MaW-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 15:56:57 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id D1EB737B40E for ; Sat, 29 Sep 2001 15:56:53 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id D5BD081D06; Sat, 29 Sep 2001 17:56:53 -0500 (CDT) Date: Sat, 29 Sep 2001 17:56:53 -0500 From: Alfred Perlstein To: Vladimir Dozen Cc: Matt Dillon , Wilko Bulte , hackers@FreeBSD.ORG Subject: Re: VM: dynamic swap remapping (patch) Message-ID: <20010929175653.Z59854@elvis.mu.org> References: <20010929155941.A291@eix.do-labs.spb.ru> <20010929071024.Q59854@elvis.mu.org> <20010929141349.A80876@freebie.xs4all.nl> <200109291653.f8TGrRR37689@earth.backplane.com> <20010929232953.B341@eix.do-labs.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929232953.B341@eix.do-labs.spb.ru>; from vladimir-dozen@mail.ru on Sat, Sep 29, 2001 at 11:29:53PM +0000 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Vladimir Dozen [010929 14:38] wrote: > P.S. Anyway, I do NOT insist my solution is better, and even that it > is good for anything at all. It was fun for me to hack in BSD kernel, > and it was interesting challenge, and I feel need to share results > with others. At worst, I will recommend our customer to setup > processing farm under FreeBSD with applied patch. I'm really impressed with the work you put into this, but it seems that you've tried to tackle two problems at the same time, and by tying them together made it less flexible and possibly more error prone. My suggestion, (but not my final say, i'm still open to ideas): Implement a memory status signal to notify processes of changes in the relative amount of system memory. When memory reaches a low or high watermark, the signal is broadcast to all running processes. The default disposition will be to ignore the signal. The signal will be named SIGMEMINFO. (SIGXfoo means 'process has exceeded resource foo') The signal will pass via the siginfo struct information such that the process can determine if the system has just exceeded the low watermark (danger) or has reclaimed down to the high watermark (enough free memory). This is just to provide processes with a warning to scale back consumption, exit, or release reasources, the good part is that it's broadcast and all interested parties will do something, hopefully the right thing. To achieve nearly the same effect as your patch, I would implement the above low/high water mark notification, then either: a) over allocate swap a bit and set the low watermark carefully. b) do the following enhancement: Provide a system whereby you can swap to the filesystem without additional upcalls/syscalls from userspace, basically, provide some means of paging to the filesystem automatically. then, set your lowwater mark to the size of your swap partition, now your system will alert your processes and automatically swap _anyone_ to the filesystem. I really think that this would be more flexible and still allow you to achieve what you want... What do you think? -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 17: 3: 1 2001 Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 618) id B7EEF37B403; Sat, 29 Sep 2001 17:02:58 -0700 (PDT) Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 In-Reply-To: <20010929153932.A50715@panzer.kdm.org> from "Kenneth D. Merry" at "Sep 29, 2001 03:39:32 pm" To: ken@kdm.org (Kenneth D. Merry) Date: Sat, 29 Sep 2001 17:02:58 -0700 (PDT) Cc: jlemon@flugsvamp.com, the_srinivas@hotmail.com, hackers@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-Id: <20010930000258.B7EEF37B403@hub.freebsd.org> From: wpaul@FreeBSD.ORG (Bill Paul) Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > On the other hand, the Tigon III > > is capable of 960 megabits -- about the wire rate limit -- > > with normal size packets, if you implement software interrupt > > coelescing (which doesn't help, unless you crank the load up > > to close to wire speed and/or do more of the stack processing > > at interrupt time). I've been able to get 906Mbps between two Dell PowerEdge 2550 servers with built-in BCM5700/Tigon III chipsets with a stock FreeBSD kernel and the bge driver. > Having downloadable firmware is actually a huge advantage. You can do > things with the Tigon II that just aren't possible with any other chip, > either because they don't have downloadable firmware, or because the vendor > won't release firmware source. > > This is a problem with the Broadcom Tigon III boards, and to some extent > with the Tigon II. Basically it looks like the firmware for the Tigon II > is very hard to get now that 3Com has control of it, and I don't think > Broadcom will release the Tigon III firmware. (Assuming it is a > firmware-based chip.) I still have a copy of the last release of the Tigon II firmware and the development environment (which is what I used to generate the firmware image included with FreeBSD). As for the Tigon III, there is a default firmware image included in the EEPROM on the card, which is auto-loaded when the chip powers up. It is possible for a driver to load a custom image into the NIC's memory which will override the auto-loaded one, and it's also possible to load a new image into the EEPROM, however this requires an additional manual on top of the BCM5700 driver developer's guide as well as the firmware development kit, which you can only get from Broadcom/3Com/whatever under NDA. These custom images are called "value-add" firmware which are used to provide features like TCP segmentation, which you can't do with the default firmware image. Note that the BCM5700/Tigon III only has a limited amount of on-board RAM (256KB, I think). You're supposed to be able to attach up to 16MB of static SRAM to the BCM5700. The BCM5701 doesn't support external SSRAM at all, which I find a little confusing. -Bill To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 17:51:26 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from iguana.aciri.org (iguana.aciri.org [192.150.187.36]) by hub.freebsd.org (Postfix) with ESMTP id 8B0A037B403 for ; Sat, 29 Sep 2001 17:51:23 -0700 (PDT) Received: (from rizzo@localhost) by iguana.aciri.org (8.11.3/8.11.1) id f8U0mbj98590; Sat, 29 Sep 2001 17:48:37 -0700 (PDT) (envelope-from rizzo) From: Luigi Rizzo Message-Id: <200109300048.f8U0mbj98590@iguana.aciri.org> Subject: Re: Problem(s) With FreeBSD-Current In-Reply-To: <20010929172053.XCRL16495.mta10.onebox.com@onebox.com> from Glenn Gombert at "Sep 29, 2001 10:20:53 am" To: glenngombert@onebox.com (Glenn Gombert) Date: Sat, 29 Sep 2001 17:48:37 -0700 (PDT) Cc: hackers@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL43 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > Hello, > > I have tried to compile FreeBSD Current (several times) over the last > couple of days and get the following message when trying to run XFree86 > and a number of other applications: > > /user/lib/libpam.so.1 : Undefined symbol "__stdoutp"... > > Does anyone else have the same problem ?? same here (though cross-compilig CURRENT on 4.3), for me the fix was to revert the sep.20 change on src/include/stdio.h @@ -198,7 +198,7 @@ __END_DECLS #endif /* To be removed by 5.0-RELEASE */ -#if (defined(__i386__) || defined(__alpha__)) && !defined(_FIXED_STDIO) +#if (defined(__i386__) || defined(__alpha__)) && defined(_OLD_STDIO) #define stdin (&__sF[0]) #define stdout (&__sF[1]) #define stderr (&__sF[2]) Maybe it is the wrong fix, but it did work. cheers luigi ----------------------------------+----------------------------------------- Luigi RIZZO, luigi@iet.unipi.it . ACIRI/ICSI (on leave from Univ. di Pisa) http://www.iet.unipi.it/~luigi/ . 1947 Center St, Berkeley CA 94704 Phone: (510) 666 2927 ----------------------------------+----------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 19:33:44 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mail.webmonster.de (datasink.webmonster.de [194.162.162.209]) by hub.freebsd.org (Postfix) with SMTP id 04C9637B403 for ; Sat, 29 Sep 2001 19:33:40 -0700 (PDT) Received: (qmail 62005 invoked by uid 1000); 30 Sep 2001 02:34:00 -0000 Date: Sun, 30 Sep 2001 04:34:00 +0200 From: "Karsten W. Rohrbach" To: Vladimir Dozen Cc: hackers@FreeBSD.org Subject: Re: VM: dynamic swap remapping (patch) Message-ID: <20010930043400.C61409@mail.webmonster.de> Mail-Followup-To: "Karsten W. Rohrbach" , Vladimir Dozen , hackers@FreeBSD.org References: <20010929155941.A291@eix.do-labs.spb.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="9Ek0hoCL9XbhcSqy" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010929155941.A291@eix.do-labs.spb.ru>; from vladimir-dozen@mail.ru on Sat, Sep 29, 2001 at 03:59:41PM +0000 X-Arbitrary-Number-Of-The-Day: 42 X-URL: http://www.webmonster.de/ X-Disclaimer: My opinions do not necessarily represent those of my employer X-Work-URL: http://www.ngenn.net/ X-Work-Address: nGENn GmbH, Schloss Kransberg, D-61250 Usingen-Kransberg, Germany X-Work-Phone: +49-6081-682-304 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --9Ek0hoCL9XbhcSqy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Vladimir Dozen(vladimir-dozen@mail.ru)@2001.09.29 15:59:41 +0000: > ehlo. >=20 > (Sorry for long pre-history, I believe it is necessary.) >=20 > My current employer develops large CORBA-based data mining servers. > They are usually run under HP-UX, but, following the current fashion > to build processing farms, I was targeted to build version for free > unices. Initial platform was Linux, and build itself was done smoothly, > but very soon we were got problem: we use pthreads; to be more precise, > we use thread-per-client model. This means that at the same time we may > compute from single to a few tens client sessions. Each session may eat > as much as 1G of address space, and even more (actually, there is no > limits except for hardware ones). IIRC from the problems we had with a project some while ago, mm might help. [http://www.engelschall.com/sw/mm/] it wraps malloc() and friends into a neat api, including preallocation in fs space (the features are somewhat os dependent) and fast shared memory. /k --=20 > Did you know that there are 71.9 acres of nipple tissue in the U.S.? KR433/KR11-RIPE -- WebMonster Community Founder -- nGENn GmbH Senior Techie http://www.webmonster.de/ -- ftp://ftp.webmonster.de/ -- http://www.ngenn.n= et/ karsten&rohrbach.de -- alpha&ngenn.net -- alpha&scene.org -- catch@spam.de GnuPG 0x2964BF46 2001-03-15 42F9 9FFF 50D4 2F38 DBEE DF22 3340 4F4E 2964 B= F46 Please do not remove my address from To: and Cc: fields in mailing lists. 1= 0x --9Ek0hoCL9XbhcSqy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE7toSXM0BPTilkv0YRAltkAKCTsszy3bY0cXh8G9nFEYD+Ms2zpwCgn+zs 39ebzqImR1QbsgCeO9bABEU= =5LwJ -----END PGP SIGNATURE----- --9Ek0hoCL9XbhcSqy-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 21:28:49 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 5977C37B405 for ; Sat, 29 Sep 2001 21:28:41 -0700 (PDT) Received: from mindspring.com (dialup-209.247.143.253.Dial1.SanJose1.Level3.net [209.247.143.253]) by falcon.mail.pas.earthlink.net (8.11.5/8.9.3) with ESMTP id f8U4SRC24073; Sat, 29 Sep 2001 21:28:27 -0700 (PDT) Message-ID: <3BB69F42.5C1ED705@mindspring.com> Date: Sat, 29 Sep 2001 21:27:46 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Kenneth D. Merry" Cc: Jonathan Lemon , the_srinivas@hotmail.com, hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> <20010928074347.A36960@panzer.kdm.org> <3BB61A6B.A97E955A@mindspring.com> <20010929153932.A50715@panzer.kdm.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Kenneth D. Merry" wrote: > > Unfortunately, it can not correctly interoperate with a > > number of cards in jumbogram mode, so unless you know the > > card on the other end and manually configure it (it can't > > negotiate properly), you can't really use jumbograms. Or > > you could rewrite the firmware (of course). > > Huh? Care to explain? Why wouldn't it interoperate with jumbo frames? > Why are jumbo frames something that need be negotiated by the card? > They're not negotiated by the card, but rather by the TCP stack. Because at the time the Tigon II was released, the jumbogram wire format had not solidified. Therefore cards built during that time used different wire data for the jumbogram framing. > > I have never seen the board with the stock FreeBSD driver > > get better than about half wire rate with normal size > > packets and window depth. > > Yep, it doesn't handle large numbers of small packets very well. You can > easily get wire rate with jumbo frames, though. Again, that's not useful, since it requires hand negotiation; you can't automatically set the MTU up as a result of doing a driver level negotiation with the other end of the link (unless the other end is also a Tigon II). > > On the other hand, the Tigon III > > is capable of 960 megabits -- about the wire rate limit -- > > with normal size packets, if you implement software interrupt > > coelescing (which doesn't help, unless you crank the load up > > to close to wire speed and/or do more of the stack processing > > at interrupt time). > > > > The Tigon II also has the unfortunate need to download its > > firmware, and a FreeBSD bug -- the lack of a distinct state > > for firmware download -- means that the firmware get sent > > to the card each time the thing is config'ed up, or an IP > > alias is added to the thing. Plus the damn things are more > > expensive than the Tigon III based cards. > > That's partially because Bill wanted to have a process context > when he was downloading the firmware, and didn't want to do a > kernel thread to do it. Bill and I discussed this at length, in person. The problem is that FreeBSD resets the ethernet cards, for fear that it is the user's intent to recover a hung card tis way. The firmware download issue is because there is not a seperate entry point for firmware download, combined with the initial gratuitous ARP request being placed on the transmit queue for the card before it is really up, and the only other alternative place to hook the firmware download meant that you would have to hang the ifconfig command -- nad therefore the /etc/rc, and therefore the boot process -- waiting for th download to complete. The upshot of this is that adding a virtual IP address to the interface is assumed to be a very infrequent event. This assumption is incorrect for a large number of IP takeover protocols (e.g. VRRP), whose timing granularity is small enough that the firmware download exceeds the interval by a factor of 10 or more. By the time the alias address assignment (and the concommitant firmware download) are completed, the takeover window has been greatly exceeded. For all three of the takeover protocols for which there is published documentation available, this results in the address being configured on, the firmware downloaded, the takeover window being missed, losing the contention race for the takeover, and the address being configured off -- and the firmware being downloaded yet again. Another "amusing" consequence is that, if there is traffic at the time of the event, the driver goes into a perpetual "watchdog timeout: resetting" loop, which can oly be resolved by a system reboot. > So it gets done when the card is ifconfiged up. I wrote some code for my > former employer to wait up to 10 seconds for link to show up when the board > is configured. The alternative is a 'sleep 5' (for fiber boards) or 'sleep > 8' (for copper boards) after it is ifconfiged up. The kernel code is trivial, but it's not in there by default, and wouldn't need to be there, if there were a firmware download phase for the card which could be triggered to occur asynchronously (e.g. so that the driver did not hand user space in the attach). The "sleep" workaround doesn't really work, unless you're willing to hack up your rc files to deal correctly with a mix of cards; even so, then it adds an additional ~10 seconds per ti interface to the boot process, and to any reconfiguration process that can occur later (and later reconfiguration processes risk going into the "watchdog timeout: resetting" infinite loop, if there are any proceses with network links active at the time -- something which is almost inevitable). > Having downloadable firmware is actually a huge advantage. You can do > things with the Tigon II that just aren't possible with any other chip, > either because they don't have downloadable firmware, or because the vendor > won't release firmware source. I agree; I'm well aware of a number of people who have done things to the firmware. The problem is the FreeBSD ethernet driver architecture doesn't provide for an entry point hook for asynchronous firmware download. > This is a problem with the Broadcom Tigon III boards, and to some extent > with the Tigon II. Basically it looks like the firmware for the Tigon II > is very hard to get now that 3Com has control of it, and I don't think > Broadcom will release the Tigon III firmware. (Assuming it is a > firmware-based chip.) The Tigon II firmware is available from their web site. You can license the Tigon III firmware, but I agree, that it is definitely a step backward, compared to general availability. > > To put a final nail in the coffin, the card can't offload > > TCP checksum with VLANs enabled (it breaks). > > Probably something that could be fixed in the firmware. Too bad > the vendor isn't really supporting it very much now. The card has been end of lifed. > > All in all, the Tigon II is a bad board. > > That's your opinion, but I disagree. I can more than double my connections per second, and triple or quadruple my overall throughput by switching from a Tigon II to a Tigon III, with no other changes. The Tigon III is the first board to be able to do Gigabit wire rate with normal sized packets. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 21:48: 6 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from falcon.mail.pas.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id B39D437B403; Sat, 29 Sep 2001 21:48:03 -0700 (PDT) Received: from mindspring.com (dialup-209.247.143.253.Dial1.SanJose1.Level3.net [209.247.143.253]) by falcon.mail.pas.earthlink.net (8.11.5/8.9.3) with ESMTP id f8U4m1C14072; Sat, 29 Sep 2001 21:48:02 -0700 (PDT) Message-ID: <3BB6A432.FF49BAC3@mindspring.com> Date: Sat, 29 Sep 2001 21:48:50 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bill Paul Cc: "Kenneth D. Merry" , jlemon@flugsvamp.com, the_srinivas@hotmail.com, hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 References: <20010930000258.B7EEF37B403@hub.freebsd.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bill Paul wrote: > It is possible for a driver > to load a custom image into the NIC's memory which will override the > auto-loaded one, and it's also possible to load a new image into > the EEPROM, however this requires an additional manual on top of > the BCM5700 driver developer's guide as well as the firmware development > kit, which you can only get from Broadcom/3Com/whatever under NDA. Yes. This is annoying as hell. One wonder what they are thinking. > These custom images are called "value-add" firmware which are used to > provide features like TCP segmentation, which you can't do with the > default firmware image. Note that the BCM5700/Tigon III only has > a limited amount of on-board RAM (256KB, I think). You're supposed > to be able to attach up to 16MB of static SRAM to the BCM5700. The > BCM5701 doesn't support external SSRAM at all, which I find a little > confusing. The hardware based two card failover is based partly on card firmware changes, as well. They support this for Linux because they themselves wrote the code. They won't let third parties license this for porting, unfortunately, even for binary only ditributions. 8-(. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Sep 29 22:15:15 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from panzer.kdm.org (panzer.kdm.org [216.160.178.169]) by hub.freebsd.org (Postfix) with ESMTP id F360837B408 for ; Sat, 29 Sep 2001 22:14:57 -0700 (PDT) Received: (from ken@localhost) by panzer.kdm.org (8.11.6/8.9.1) id f8U5Ei552527; Sat, 29 Sep 2001 23:14:44 -0600 (MDT) (envelope-from ken) Date: Sat, 29 Sep 2001 23:14:44 -0600 From: "Kenneth D. Merry" To: Terry Lambert Cc: Jonathan Lemon , the_srinivas@hotmail.com, hackers@FreeBSD.ORG Subject: Re: TCP&IP cksum offload on FreeBSD 4.2 Message-ID: <20010929231444.B52304@panzer.kdm.org> References: <200109270157.f8R1vZ546863@prism.flugsvamp.com> <3BB42E50.B32F5E95@mindspring.com> <20010928074347.A36960@panzer.kdm.org> <3BB61A6B.A97E955A@mindspring.com> <20010929153932.A50715@panzer.kdm.org> <3BB69F42.5C1ED705@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <3BB69F42.5C1ED705@mindspring.com>; from tlambert2@mindspring.com on Sat, Sep 29, 2001 at 09:27:46PM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Sep 29, 2001 at 21:27:46 -0700, Terry Lambert wrote: > "Kenneth D. Merry" wrote: > > > Unfortunately, it can not correctly interoperate with a > > > number of cards in jumbogram mode, so unless you know the > > > card on the other end and manually configure it (it can't > > > negotiate properly), you can't really use jumbograms. Or > > > you could rewrite the firmware (of course). > > > > Huh? Care to explain? Why wouldn't it interoperate with jumbo frames? > > Why are jumbo frames something that need be negotiated by the card? > > They're not negotiated by the card, but rather by the TCP stack. > > Because at the time the Tigon II was released, the jumbogram > wire format had not solidified. Therefore cards built during > that time used different wire data for the jumbogram framing. Can you give more details? Did someone decide on a different ethertype than 0x8870 or something? That's really the only thing that's different between a standard ethernet frame and a jumbo frame. (other than the size) > > > I have never seen the board with the stock FreeBSD driver > > > get better than about half wire rate with normal size > > > packets and window depth. > > > > Yep, it doesn't handle large numbers of small packets very well. You can > > easily get wire rate with jumbo frames, though. > > Again, that's not useful, since it requires hand negotiation; > you can't automatically set the MTU up as a result of doing a > driver level negotiation with the other end of the link (unless > the other end is also a Tigon II). Negotiation, at least for packet size, is generally done at a higher level than the transport layer (ethernet in this case). Your routing hardware should be able to send back ICMP need fragment responses if an IP packet has the don't frag bit set, or it should be able to fragment the packet if it doesn't have that bit set. > > > On the other hand, the Tigon III > > > is capable of 960 megabits -- about the wire rate limit -- > > > with normal size packets, if you implement software interrupt > > > coelescing (which doesn't help, unless you crank the load up > > > to close to wire speed and/or do more of the stack processing > > > at interrupt time). > > > > > > The Tigon II also has the unfortunate need to download its > > > firmware, and a FreeBSD bug -- the lack of a distinct state > > > for firmware download -- means that the firmware get sent > > > to the card each time the thing is config'ed up, or an IP > > > alias is added to the thing. Plus the damn things are more > > > expensive than the Tigon III based cards. > > > > That's partially because Bill wanted to have a process context > > when he was downloading the firmware, and didn't want to do a > > kernel thread to do it. > > Bill and I discussed this at length, in person. The problem is > that FreeBSD resets the ethernet cards, for fear that it is the > user's intent to recover a hung card tis way. > > The firmware download issue is because there is not a seperate > entry point for firmware download, combined with the initial > gratuitous ARP request being placed on the transmit queue for > the card before it is really up, and the only other alternative > place to hook the firmware download meant that you would have > to hang the ifconfig command -- nad therefore the /etc/rc, and > therefore the boot process -- waiting for th download to complete. > > The upshot of this is that adding a virtual IP address to the > interface is assumed to be a very infrequent event. > > This assumption is incorrect for a large number of IP takeover > protocols (e.g. VRRP), whose timing granularity is small enough > that the firmware download exceeds the interval by a factor of > 10 or more. By the time the alias address assignment (and the > concommitant firmware download) are completed, the takeover > window has been greatly exceeded. > > For all three of the takeover protocols for which there is > published documentation available, this results in the address > being configured on, the firmware downloaded, the takeover > window being missed, losing the contention race for the takeover, > and the address being configured off -- and the firmware being > downloaded yet again. > > Another "amusing" consequence is that, if there is traffic at > the time of the event, the driver goes into a perpetual > "watchdog timeout: resetting" loop, which can oly be resolved > by a system reboot. That certainly sounds like a problem. > > So it gets done when the card is ifconfiged up. I wrote some code for my > > former employer to wait up to 10 seconds for link to show up when the board > > is configured. The alternative is a 'sleep 5' (for fiber boards) or 'sleep > > 8' (for copper boards) after it is ifconfiged up. > > The kernel code is trivial, but it's not in there by default, > and wouldn't need to be there, if there were a firmware download > phase for the card which could be triggered to occur asynchronously > (e.g. so that the driver did not hand user space in the attach). True. > The "sleep" workaround doesn't really work, unless you're willing > to hack up your rc files to deal correctly with a mix of cards; > even so, then it adds an additional ~10 seconds per ti interface > to the boot process, and to any reconfiguration process that can > occur later (and later reconfiguration processes risk going into > the "watchdog timeout: resetting" infinite loop, if there are any > proceses with network links active at the time -- something which > is almost inevitable). Yep. > > Having downloadable firmware is actually a huge advantage. You can do > > things with the Tigon II that just aren't possible with any other chip, > > either because they don't have downloadable firmware, or because the vendor > > won't release firmware source. > > I agree; I'm well aware of a number of people who have done > things to the firmware. The problem is the FreeBSD ethernet > driver architecture doesn't provide for an entry point hook for > asynchronous firmware download. I spent quite a lot of time working on Tigon II boards, in fact. > > This is a problem with the Broadcom Tigon III boards, and to some extent > > with the Tigon II. Basically it looks like the firmware for the Tigon II > > is very hard to get now that 3Com has control of it, and I don't think > > Broadcom will release the Tigon III firmware. (Assuming it is a > > firmware-based chip.) > > The Tigon II firmware is available from their web site. Do you have a URL for that? All I can find is a download page for developer information on mostly obsolete products, not the 3c985B or the Alteon ACEnics that they bought from Alteon/Nortel. > You can license the Tigon III firmware, but I agree, that it > is definitely a step backward, compared to general availability. It is a shame. > > > To put a final nail in the coffin, the card can't offload > > > TCP checksum with VLANs enabled (it breaks). > > > > Probably something that could be fixed in the firmware. Too bad > > the vendor isn't really supporting it very much now. > > The card has been end of lifed. > > > > All in all, the Tigon II is a bad board. > > > > That's your opinion, but I disagree. > > I can more than double my connections per second, and triple > or quadruple my overall throughput by switching from a Tigon II > to a Tigon III, with no other changes. The Tigon III is the > first board to be able to do Gigabit wire rate with normal sized > packets. I'm sure the Tigon III is a better chip -- it's almost inevitable that as time goes on better things will come out. The main drawback there is that you can't get firmware without an NDA, and that hinders development. Ken -- Kenneth Merry ken@kdm.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message