From owner-freebsd-arch@FreeBSD.ORG Wed May 7 03:29:20 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72986106567C; Wed, 7 May 2008 03:29:20 +0000 (UTC) (envelope-from keramida@freebsd.org) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id D31F38FC26; Wed, 7 May 2008 03:29:19 +0000 (UTC) (envelope-from keramida@freebsd.org) Received: from kobe.laptop (adsl64-185.kln.forthnet.gr [77.49.191.185]) (authenticated bits=128) by igloo.linux.gr (8.14.2/8.14.2/Debian-4) with ESMTP id m473HnMb015969 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 7 May 2008 06:17:56 +0300 Received: from kobe.laptop (kobe.laptop [127.0.0.1]) by kobe.laptop (8.14.2/8.14.2) with ESMTP id m473Hn1e009904; Wed, 7 May 2008 06:17:49 +0300 (EEST) (envelope-from keramida@freebsd.org) Received: (from keramida@localhost) by kobe.laptop (8.14.2/8.14.2/Submit) id m473HmGd009903; Wed, 7 May 2008 06:17:48 +0300 (EEST) (envelope-from keramida@freebsd.org) From: Giorgos Keramidas To: gnn@freebsd.org References: <4811E384.5020604@freebsd.org> Date: Wed, 07 May 2008 06:17:48 +0300 In-Reply-To: (gnn@freebsd.org's message of "Sun, 27 Apr 2008 17:07:56 -0700") Message-ID: <8763tqkdk3.fsf@kobe.laptop> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-MailScanner-ID: m473HnMb015969 X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-4.339, required 5, autolearn=not spam, ALL_TRUSTED -1.80, AWL 0.06, BAYES_00 -2.60) X-Hellug-MailScanner-From: keramida@freebsd.org X-Spam-Status: No Cc: arch@freebsd.org, Andre Oppermann Subject: Re: Accounting for mbufs and clusters assigned to a socket buffer X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2008 03:29:20 -0000 On Sun, 27 Apr 2008 17:07:56 -0700, gnn@freebsd.org wrote: >At Fri, 25 Apr 2008 15:58:28 +0200, andre wrote: >>gnn@freebsd.org wrote: >>> Howdy, >>> >>> The following patch updates the kernel (CURRENT as of 23 April or >>> so) and netstat(1) to show not only the bytes in the receive and >>> send queues but also the mbuf and cluster usage per socket buffer. >>> I'd be interested in people's comments on this. I'd like to extend >>> such counting to show more information, in particular how much of a >>> cluster or mbuf is actually in use. >> >> The intent of tracking that information is good. However there are >> some problems with your approach: M_EXT does not mean the mbuf has a >> 2k cluster attached. It could by any external storage. That is a 2k >> (classic) cluster, a 4k (page size) cluster, a 9k cluster, a VM page >> (sendfile), and so on. > > Yup, this is a first cut. > >> The field sb_mbcnt already gives you the aggregated gross storage >> space in use for the socket. And sb_cc tells how much actual payload >> it contains. > > Right but it does not tell us the mix of clusters vs. mbufs, which is > something useful for tuning. That would be very useful indeed :-) I guess we could check *both* (m_flags & M_EXT) and m_ext.ext_type. This won't really show how many mbufs use a classic cluster vs. a 9k cluster, but checking for all the m_ext.ext_types is probably going to look a bit ugly in a single if-check :( We have at least EXT_CLUSTER, EXT_JUMBOP, EXT_JUMBO9, EXT_JUMBO16 and EXT_PACKET which may be interesting for cluster-related accounting, but in socketvar.h we can't really use the uma zone accounting. I just noticed that the 'interesting' ext_types are conveniently short integers though. Maybe we could change sb_ccnt to a small array, and then change if ((m)->m_flags & M_EXT) \ (sb)->sb_ccnt += 1; to something like: if ((m)->m_flags & M_EXT && (m)->m_ext.ext_type < EXT_MBUF) \ (sb)->sb_ccnt[(m)->m_ext.ext_type - 1] += 1; \ With our current ext_type's #define EXT_CLUSTER 1 /* mbuf cluster */ #define EXT_SFBUF 2 /* sendfile(2)'s sf_bufs */ #define EXT_JUMBOP 3 /* jumbo cluster 4096 bytes */ #define EXT_JUMBO9 4 /* jumbo cluster 9216 bytes */ #define EXT_JUMBO16 5 /* jumbo cluster 16184 bytes */ #define EXT_PACKET 6 /* mbuf+cluster from packet zone */ #define EXT_MBUF 7 /* external mbuf reference (M_IOVEC) */ ... the [ext_type - 1] would move the burden of parsing the ext_type at the place where we actually have to print something about the sb_ccnt counters. Would the 'bloat' of u_int[6] for ext_type's which are unused be too much to keep around?