Date: Fri, 24 Jun 2011 12:15:37 +1000 From: Peter Jeremy <peter.jeremy@alcatel-lucent.com> To: freebsd-arch@freebsd.org Subject: Context Switching Oddities Message-ID: <20110624021537.GD10304@pjdesk.au.alcatel-lucent.com> In-Reply-To: <20110623053051.GL65891@pjdesk.au.alcatel-lucent.com> References: <20110623053051.GL65891@pjdesk.au.alcatel-lucent.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--sgneBHv3152wZ8jf
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
I originally asked a variant of this in -sparc64 since the oddities
initially seemed to be related to sparc64 but it was suggested that
this might be a more appropriate list. I've also added more results.
I have a tool that measures the rate at which a single-byte token can
be passed between two processes via a socketpair - basically counting
while (1) {
read(fd[0], buf, 1);
write(fd[1], "a", 1);
}
I was initially running it multiple copies of it on an otherwise idle
V890 (16-CPU, 64GB RAM running -current from about a week ago),
capturing 'vmstat -s' output. In the process, I have found several
oddities. That prompted me to additionally test it on a V440 (4-CPU,
8GB RAM, same world as the V890), a SB1500 (1 CPU, 4GB RAM, -current
=66rom a few days ago) and an Athlon-64 (dual-core, 8GB RAM, -stable
amd64 from a few months ago). The SPARC systems are all using 4BSD
and the amd64 is using ULE. I don't have access to any large x86
boxes to cross-check.
1) The number of context switches doesn't match my expectations.
See http://i.imgur.com/28OHu.jpg (1 CPU)
http://i.imgur.com/6YRh8.jpg (2 core)
http://i.imgur.com/r0v7M.jpg (4 CPU)
http://i.imgur.com/hkCA2.jpg (16 CPU)
http://i.imgur.com/9Tt9Q.gif (combined)
Based on one process writing a token to a second process requiring one
context switch, I would expect the number of context switches to
roughly match the green (based on token passing rate) or blue (based
on syscall rate) lines. Instead, it's generally far too low, though
the 4- and 16-CPU graphs start out unexpectedly high.
2) The transfer rate doesn't gradually tail off
See http://i.imgur.com/YoDQ5.jpg (1 CPU)
http://i.imgur.com/0Wl1Y.jpg (2 core)
http://i.imgur.com/699zr.jpg (4 CPU)
http://i.imgur.com/0ujRN.jpg (16 CPU)
http://i.imgur.com/omxG1.gif (combined & scaled)
I would expect a fairly flat peak from 1 to about n-CPU pairs (since
there are that many execution threads available) that then tailed off
as scheduler overheads increased. Instead the 4- & 16-CPU tests show
a dip initially then rising to a peak before tailing off. Each graph
shows both the rate reported by the program and the rate estimated
=66rom the syscall rate.
Can anyone offer an explanation for this behaviour?
--=20
Peter Jeremy
--sgneBHv3152wZ8jf
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (FreeBSD)
iEYEARECAAYFAk4D80kACgkQ/opHv/APuIc38wCfYhu4ewvyV4EETy00ObKPH7iV
2+EAnRi99cQ0oDhkBDE/rWrnJZCPwRgl
=rZo+
-----END PGP SIGNATURE-----
--sgneBHv3152wZ8jf--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110624021537.GD10304>