Date: Tue, 21 Aug 2001 19:34:13 -0400 From: Bosko Milekic <bmilekic@technokratis.com> To: alfred@freebsd.org, jlemon@freebsd.org, freebsd-net@freebsd.org Subject: [PATCH] mbtypes stats re-enabled in -CURRENT Message-ID: <20010821193413.A22031@technokratis.com>
next in thread | raw e-mail | index | archive | help
Hi, Several weeks ago, Jiangyi Liu <jyliu@163.net> sent me an early patch that re-enabled mbtypes statistics under the mbuf allocator in -CURRENT (I had disabled them temporarily while I worked on the allocator semantics itself). Just a couple of days ago, I cleaned up the patch, fixed it up, and rolled a new diff applying to most recent CURRENT. This patch re-enables mbtypes statistics under CURRENT: http://people.freebsd.org/~bmilekic/code/mb_alloc/mbtypes_stats.diff (17k) To CURRENT users with either SMP or UP : please test, you will need to: 1) cvsup fresh CURRENT sources 2) apply patch found at above URL 3) rebuild and reinstall kernel 4) reboot 5) cp /usr/src/sys/sys/mbuf.h /usr/include/sys/ 6) rebuild and reinstall both /usr/src/usr.bin/netstat and /usr/src/usr.bin/systat 7) go about your business, all network services working as usual 8) Check `netstat -m' and `systat -mbufs' outputs, more specifically look out for "Allocated mbuf types" output in the mbuf section of `netstat -m' output and look for allocated mbuf types to be displayed following a `systat -mbufs' 9) Email me (bmilekic@dsuper.net) and let me know how it went. I'd like to commit this within the next week. To those reviewing the patch, here's how it works: Mbuf "types" (e.g. MT_DATA, etc.) are counted with the help of `long' types. Each per-CPU mbuf statistics data structure (mbpstat) contains an array of longs called mb_mbtypes[]. The size of this array is, of course, MT_NTYPES, defined in sys/mbuf.h specifically for this purpose. The general mbuf statistics structure (called mbstat) contains a `short' m_numtypes member which, during mbuf_init(), is set to MT_NTYPES. The general container's mbpstat struct, like the per-CPU container's mbpstat structs, also contains this mb_mbtypes array. During mbuf allocation, (type) being an argument already passed to m_get() gets passed down to the mb_alloc allocation routine (mb_alloc()). For mbuf clusters, MT_NOTMBUF gets passed down (MT_FREE - no longer used - has been renamed to MT_NOTMBUF). The mb_alloc() code will then increment mb_mbtypes[(type)] if (type) is not MT_NOTMBUF. This manipulation is performed cosistently because it is in the per-CPU container and because it is done while the per-CPU container's (or the general container's) list is already held. During mbuf freeing, mbuf_being_freed->m_type is passed to mb_free() by m_free(). Again, in the case of mbuf clusters, MT_NOTMBUF is passed down instead. So, mb_free() will decrement mb_mbtypes[(type)] if (type) is not MT_NOTMBUF. This manipulation is handled under the given container's lock and is therefore guaranteed to be consistent. The reason this works is because each container has its own mbpstat per-CPU structure and can therefore ensure its consistency via its own container lock. The other reason this works is because mb_mbtypes[] is, in each case, an array of _longs_ and not _unsigned longs_. This allows for the counters to go below 0, as is expected in this situation. Consider, for example, that a thread A runs on CPU 0 and allocates an MT_DATA mbuf, then this happens: mbpstat[cpu_0]->mb_mbtypes[MT_DATA]++; If one considers that mbpstat[cpu_0]->mb_mbtypes[MT_DATA] was 0 prior to this manipulation, then its new value is 1. Now consider that another thread B on CPU 1 frees this same mbuf. Then this happens: mbpstat[cpu_1]->mb_mbtypes[MT_DATA]--; If one considers that mbpstat[cpu_1]->mb_mbtypes[MT_DATA] was 0 prior to this manipulation, then its new value is -1. So to get the "real" mbtypes[(type)] value, we do the following: mbtypes[(type)] = mbpstat[cpu_1]->mb_mbtypes[(type)] + ... + mbpstat[cpu_n]->mb_mbtypes[(type)]; We are guaranteed a number >= 0, because of the fact that all the mb_mbtypes[] are manipulated consistently, under some lock, and because we will never have a certain type _decremented_ before it's once _incremented_. In other words, we will never have something like "mb_mbtypes[(type)]--" occur unless there was already at least one "mb_mbtypes[(type)]++" that occured on at least one of the other containers. MCHTYPE() is implemented to call a function m_chtype() which changes the type accordingly and manipulates the mb_mbtypes[] of the general container after acquiring the general container's lock. This function is rarely used, and it should hopefully stay that way. Regards, -- Bosko Milekic bmilekic@technokratis.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010821193413.A22031>