Date: Thu, 07 Dec 2000 21:33:27 -0800 From: Mike Smith <msmith@freebsd.org> To: Terry Lambert <tlambert@primenet.com> Cc: smp@FreeBSD.ORG Subject: Re: Netgraph and SMP Message-ID: <200012080533.eB85XRN00458@mass.osd.bsdi.com> In-Reply-To: Your message of "Fri, 08 Dec 2000 03:50:04 GMT." <200012080350.UAA03298@usr08.primenet.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > In Solaris, the entry into the driver would hold a reference,
> > > which would result in the reference count being incremented.
> > > Only modules with a 0 reference count can be unloaded. This
> > > same mechanism is used for vnodes, and for modules on which
> > > other modules depend. It works well, ans is very light weight.
> >
> > The whole problem is that it *isn't* very light weight.
> >
> > The reference count has to be atomic, which means that it ping-pongs
> > around from CPU to CPU, causing a lot of extra cache traffic.
> >
> > OTOH, there's not much we can do about this short of going looking for
> > better multi-CPU reference count implementations once we have time to
> > worry about performance.
>
> Actually, you can just put it in non-cacheable memory, and the
> penalty will only be paid by the CPU(s) doing the referencing.
Yes. And you'll pay the penalty *all* the time. At least when the
ping-pong is going on, there will be times when you'll hit the counter
valid in your own cache. Marking it uncacheable (or even write-back
cacheable) is worse.
> Still, for a very large number of CPUs, this would work fine
> for all but frequently contended objects.
Er. We're talking about an object which is susceptible to being *very*
frequently contended.
> I think that it is making more and more sense to lock interrupts
> to a single CPU.
No, it's not. Stop this nonsense. It's not even practical on some of
the platforms we're looking at.
> What happens if you write to a page that's marked non-cachable
> on the CPU on which you are running, but cacheable on another
> CPU? Does it do the right thing, and update the cache on the
> caching CPU?
Er, what are you smoking Terry? You never 'update' the cache on another
processor; the other processor snoops your cache/memory activity and
invalidates its own cache based on your broadcasts.
> If so, locking the interrupt processing for each
> card to a particular CPU could be very worthwhile, since you
> would never take the hit, unless you were doing something
> extraordinary.
With the way our I/O structure is currently laid out, this blows because
you end up serialising everything.
--
... every activity meets with opposition, everyone who acts has his
rivals and unfortunately opponents also. But not because people want
to be opponents, rather because the tasks and relationships force
people to take different points of view. [Dr. Fritz Todt]
V I C T O R Y N O T V E N G E A N C E
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200012080533.eB85XRN00458>