Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 6 Nov 2012 02:00:59 -0800 (PST)
From:      "Rodney W. Grimes" <freebsd@pdx.rh.CN85.ChatUSA.com>
To:        Juli Mallett <juli@clockworksquid.com>
Cc:        "freebsd-mips@FreeBSD.org" <freebsd-mips@freebsd.org>
Subject:   Re: CACHE_LINE_SIZE macro.
Message-ID:  <201211061000.qA6A0xpT017479@pdx.rh.CN85.ChatUSA.com>
In-Reply-To: <CACVs6=_BrwJ19CPj7OqKvV8boHfujVWqn96u3VPUmZ040JpAeQ@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
> Fellow FreeBSD/MIPSists,
> 
> CACHE_LINE_SIZE is being used increasingly-much in ways which may have ABI
> implications, etc.  It is currentyl 2^6, whereas at least the Cavium Octeon
> has cache lines that are actually 2^7 bytes in size.  It would be nice to
> expose the correct value to reduce false line sharing, etc., but making it
> dependent on the CPU type raises ABI issues, as well as questions about how
> to reliably get the right value to userland.  It seems to me that
> increasing it to 2^7 is the most viable approach, but I can imagine there
> might be some concerns about that, so I wanted to run it past this list
> first.  Questions, comments, concerns?  Are there MIPS CPUs with 2^8-byte
> or larger cache lines that we support or will support or which are likely
> coming over the horizon?

I've let this stew in my head for a day.. and have come up with an idea, maybe
a bit crazy, maybe not.

If people are really thinking that cache aligning mutex's is a good idea and that
it can improve performance on some, if not most, architectures, then by all means
find a way to maximize this with minal impact... so......

Allocate a page in the kernel for ``mutexes''.  Mutexes in this page are to be
cache aligned, possibly with 2 or more mutexes in a cache line.  (Hey, why cant
a mutex be a byte, and fit N of them in a cache line, if they reside in the same
cache line then this can either be a big benifit, or a real cache thrasher!).  In
most designs the cost of a byte vs int access in cached data is the same.... except
in a write cycle where you may have to do a read/merge/write to perserve other
bytes.

Structures that use to have mutexs should have a pointer to a mutex in this page,
the pointer is properly aligned on a pointer boundary.

With carefull application of defines all single existance mutexes can be compile
time referenced with mutex_table[MUTEX_NAME] and even the indirection goes away.

You could carry this further and if you know the colours of a cache (or ways) you
can break the mutex table up so that mutes tend to either fall in one way always,
or to use as many ways as possible.  Just for those that like to micro-pessimize
code that is....


There are probably hotter items in the kernel that would have more performance
impact with this type of micro - optimizations.

Always remeber when trying to play the cache makes me faster game that it only
benifits you to have it cached if you reference 2 or more times before it gets
invalidated, and cache is a limited resource that should not be abused just
to make some synthetic benchmark faster, and the cache is a very machine specific
item and what helps one design run faster is gona make others run slower.

-- 
Rod Grimes                                                 freebsd@freebsd.org



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201211061000.qA6A0xpT017479>