Date: Wed, 13 Dec 2006 02:16:51 +1100 (EST) From: Bruce Evans <bde@zeta.org.au> To: Luigi Rizzo <rizzo@icir.org> Cc: Luigi Rizzo <luigi@freebsd.org>, src-committers@freebsd.org, cvs-src@freebsd.org, cvs-all@freebsd.org, imp@freebsd.org, ticso@cicely.de Subject: Re: cvs commit: src/sys/net if_ethersubr.c Message-ID: <20061213011843.X58129@delplex.bde.org> In-Reply-To: <20061212055756.A61182@xorpc.icir.org> References: <200612081036.kB8AakMD029277@repoman.freebsd.org> <20061212131333.GU54209@cicely12.cicely.de> <20061212055756.A61182@xorpc.icir.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 12 Dec 2006, Luigi Rizzo wrote:
> On Tue, Dec 12, 2006 at 02:13:34PM +0100, Bernd Walter wrote:
>> On Fri, Dec 08, 2006 at 10:36:46AM +0000, Luigi Rizzo wrote:
>>> luigi 2006-12-08 10:36:45 UTC
>>>
>>> FreeBSD src repository
>>>
>>> Modified files:
>>> sys/net if_ethersubr.c
>>> Log:
>>> Fix an oscure bug triggered by a recent change in kern_socket.c.
>>
>> This change produces an alignment panic on arm.
>> Reverting it gets my system back to live.
Something reduced my ttcp tx bandwidth for tiny packets on an i386 (Athlon
XP 2700 overclocked) from 350 kpps to 300 kpps (after I thought increased
it from 320-330 kpps to 350-360 kpps) by reducing copying). I suspected
this commit, then misalignment. I still suspect alignment...
> then i suppose the proper fix is to revert to m_copy() and
> work on if_simloop() so that 1. it handles a readonly chain, and
> 2. when doing so, it passes up a properly aligned packet...
>
> however note that there is already some code in net/if_loop.c::if_simloop(),
> just that it uses this:
>
> #if defined(__ia64__) || defined(__sparc64__)
> /*
> * Some archs do not like unaligned data, so
> * we move data down in the first mbuf.
> */
> if (mtod(m, vm_offset_t) & 3) {
> KASSERT(hlen >= 3, ("if_simloop: hlen too small"));
> bcopy(m->m_data,
> (char *)(mtod(m, vm_offset_t)
> - (mtod(m, vm_offset_t) & 3)),
> m->m_len);
> m->m_data -= (mtod(m,vm_offset_t) & 3);
> }
> #endif
>
> to detect whether the architecture is alignment-sensitive.
> Is there any other identifier that we can use to check ?
__NO_STRICT_ALIGNMENT. Many drivers already use this. It's stiall an
abuse since __FOO is in the implementation namespace, and of course
__NO_STRICT_ALIGNMENT is documented mainly (only?) in its log message.
No arches "like" unaligned data; some just handle it without trapping
or other extreme slowness. I think copying of data to fix its alignment
is best for most arches if the data is small but not if it is large.
About my ttcp tx bandwith problem:
- tx is almost entirely syscall-bound (< 2-4% interrupt overhead)
- the slowdown was mostly caused by adding about 500 bytes in unrelated
code (in nfs_diskless.c) that was never executed. Removing these bytes
or moving nfs_diskless.o to the end of the object file list recovered
30-40 kpps of the lost bandwidth. It looks like there are lots of
cache conflicts for the I-cache, but I thought that this problem mostly
went away with i486's.
- the bandwidth depended mysteriously on related and unrelated previous
changes, but no single change (including ones that added much more than
500 bytes) affected it by as much as 360 -> 300.
- the bandwidth also depends on the ttcp binary and has a high variance
in separate runs (but not in the same run). I now suspect that I-cache
conflicts cause this too.
- I found some silly data alignment that doesn't seem to affect ttcp.
struct sockaddr is essentially char [16] so it asks for pessimal
alignment (not misalignment unless it is type punned). I think gcc
normally gives struct sockaddr's on the stack 4-byte alignment.
sun_noname is perfectly misaligned in all my kernels, because it
happens to get allocated after some strings with an odd total size
in the read-only text section. It is a sockaddr, not a sockaddr_un
despite its name. struct sockaddr_un is essentially char [106],
so it also asks for pessimal alignment, with more cost since it is
quite large. Grepping nm output showed this some other alignment
problems. E.g., futex functions in i386/support.s have random
alignment and other bugs caused by not using ENTRY() for them.
Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061213011843.X58129>