Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 21 Aug 2001 19:34:13 -0400
From:      Bosko Milekic <bmilekic@technokratis.com>
To:        alfred@freebsd.org, jlemon@freebsd.org, freebsd-net@freebsd.org
Subject:   [PATCH] mbtypes stats re-enabled in -CURRENT
Message-ID:  <20010821193413.A22031@technokratis.com>

next in thread | raw e-mail | index | archive | help

Hi,

  Several weeks ago, Jiangyi Liu <jyliu@163.net> sent me an early patch that
re-enabled mbtypes statistics under the mbuf allocator in -CURRENT (I had
disabled them temporarily while I worked on the allocator semantics itself).
  Just a couple of days ago, I cleaned up the patch, fixed it up, and rolled
a new diff applying to most recent CURRENT. This patch re-enables mbtypes
statistics under CURRENT:

  http://people.freebsd.org/~bmilekic/code/mb_alloc/mbtypes_stats.diff (17k)

  To CURRENT users with either SMP or UP : please test, you will need to:

  1) cvsup fresh CURRENT sources
  2) apply patch found at above URL
  3) rebuild and reinstall kernel
  4) reboot
  5) cp /usr/src/sys/sys/mbuf.h /usr/include/sys/
  6) rebuild and reinstall both /usr/src/usr.bin/netstat and
     /usr/src/usr.bin/systat
  7) go about your business, all network services working as usual
  8) Check `netstat -m' and `systat -mbufs' outputs, more specifically look
     out for "Allocated mbuf types" output in the mbuf section of `netstat -m'
     output and look for allocated mbuf types to be displayed following a
     `systat -mbufs'
  9) Email me (bmilekic@dsuper.net) and let me know how it went.

  I'd like to commit this within the next week.

  To those reviewing the patch, here's how it works:

  Mbuf "types" (e.g. MT_DATA, etc.) are counted with the help of `long' types.
Each per-CPU mbuf statistics data structure (mbpstat) contains an array of
longs called mb_mbtypes[]. The size of this array is, of course, MT_NTYPES,
defined in sys/mbuf.h specifically for this purpose. The general mbuf statistics
structure (called mbstat) contains a `short' m_numtypes member which, during
mbuf_init(), is set to MT_NTYPES. The general container's mbpstat struct, like
the per-CPU container's mbpstat structs, also contains this mb_mbtypes array.
  During mbuf allocation, (type) being an argument already passed to m_get()
gets passed down to the mb_alloc allocation routine (mb_alloc()). For mbuf
clusters, MT_NOTMBUF gets passed down (MT_FREE - no longer used - has been
renamed to MT_NOTMBUF). The mb_alloc() code will then increment
mb_mbtypes[(type)] if (type) is not MT_NOTMBUF. This manipulation is performed
cosistently because it is in the per-CPU container and because it is done
while the per-CPU container's (or the general container's) list is already
held.
  During mbuf freeing, mbuf_being_freed->m_type is passed to mb_free() by
m_free(). Again, in the case of mbuf clusters, MT_NOTMBUF is passed down
instead. So, mb_free() will decrement mb_mbtypes[(type)] if (type) is not
MT_NOTMBUF. This manipulation is handled under the given container's lock
and is therefore guaranteed to be consistent.
  The reason this works is because each container has its own mbpstat per-CPU
structure and can therefore ensure its consistency via its own container lock.
The other reason this works is because mb_mbtypes[] is, in each case, an
array of _longs_ and not _unsigned longs_. This allows for the counters to
go below 0, as is expected in this situation. Consider, for example, that
a thread A runs on CPU 0 and allocates an MT_DATA mbuf, then this happens:

 mbpstat[cpu_0]->mb_mbtypes[MT_DATA]++;

  If one considers that mbpstat[cpu_0]->mb_mbtypes[MT_DATA] was 0 prior to
this manipulation, then its new value is 1.
  Now consider that another thread B on CPU 1 frees this same mbuf. Then this
happens:

 mbpstat[cpu_1]->mb_mbtypes[MT_DATA]--;

  If one considers that mbpstat[cpu_1]->mb_mbtypes[MT_DATA] was 0 prior to
this manipulation, then its new value is -1. So to get the "real"
mbtypes[(type)] value, we do the following:

 mbtypes[(type)] = mbpstat[cpu_1]->mb_mbtypes[(type)] + ... +
		   mbpstat[cpu_n]->mb_mbtypes[(type)];

  We are guaranteed a number >= 0, because of the fact that all the mb_mbtypes[]
are manipulated consistently, under some lock, and because we will never have
a certain type _decremented_ before it's once _incremented_. In other words,
we will never have something like "mb_mbtypes[(type)]--" occur unless there was
already at least one "mb_mbtypes[(type)]++" that occured on at least one of the
other containers.
  MCHTYPE() is implemented to call a function m_chtype() which changes the type
accordingly and manipulates the mb_mbtypes[] of the general container after
acquiring the general container's lock. This function is rarely used, and it
should hopefully stay that way.

Regards,
-- 
 Bosko Milekic
 bmilekic@technokratis.com


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010821193413.A22031>