From owner-freebsd-atm Thu Oct 17 15:23:39 1996 Return-Path: owner-atm Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id PAA04382 for atm-outgoing; Thu, 17 Oct 1996 15:23:39 -0700 (PDT) Received: from plains.nodak.edu (tinguely@plains.NoDak.edu [134.129.111.64]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id PAA04355; Thu, 17 Oct 1996 15:23:27 -0700 (PDT) Received: (from tinguely@localhost) by plains.nodak.edu (8.7.6/8.7.3) id RAA09577; Thu, 17 Oct 1996 17:23:18 -0500 (CDT) Date: Thu, 17 Oct 1996 17:23:18 -0500 (CDT) From: Mark Tinguely Message-Id: <199610172223.RAA09577@plains.nodak.edu> To: freebsd-atm@freebsd.org, freebsd-hackers@freebsd.org Subject: custom free for external mbuf Sender: owner-atm@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Religous debate time. :) in sys/mbuf.h the external mbuf is defined : struct m_ext { caddr_t ext_buf; /* start of buffer */ void (*ext_free) /* free routine if not the usual */ __P((caddr_t, u_int)); u_int ext_size; /* size of buffer, for ext_free */ }; (then later we see) #ifdef notyet #define MFREE(m, n) \ { MBUFLOCK(mbstat.m_mtypes[(m)->m_type]--;) \ if ((m)->m_flags & M_EXT) { \ if ((m)->m_ext.ext_free) \ (*((m)->m_ext.ext_free))((m)->m_ext.ext_buf, \ (m)->m_ext.ext_size); \ else \ MCLFREE((m)->m_ext.ext_buf); \ } \ (n) = (m)->m_next; \ FREE((m), mbtypes[(m)->m_type]); \ } (more deleted) from greping the rest of the sources, this is *the* place an external mbuf could call another free routine and it is obviously ifdef out. what I am thinking about doing is making a new breed of mbuf that is prebuilt to look like an external mbuf, but is permanent in that its free routine does not throw away the mbuf external data, mbuf structure, nor the association between mbuf and external data. why do I want these permanent mbufs? because I am working with ATM and I don't want to deal with the allocation, association, tear-down, overhead of many small PDU packets. If I add a new flag entry (say M_PERM) and change the MFREE routine (here I show changes to the "notyet" part of MFREE, the current MFREE changes would be simular): if ((m)->m_flags & M_EXT) { \ + if ((m)->m_flags & M_PERM) { \ + if ((m)->m_ext.ext_free) \ + (*((m)->m_ext.ext_free))((m), \ + (m)->m_ext.ext_size); \ + } \ + else \ if ((m)->m_ext.ext_free) \ (*((m)->m_ext.ext_free))((m)->m_ext.ext_buf, \ (m)->m_ext.ext_size); \ else \ MCLFREE((m)->m_ext.ext_buf); \ } \ (n) = (m)->m_next; \ + if ((m)->m_flags & M_PERM == 0) { \ FREE((m), mbtypes[(m)->m_type]); \ } \ } notice, I need to send the address of the mbuf to my recycle "free" program. and I cannot let it FREE the mbuf lower down in the MFREE routine. with this, I can send these permanent mbufs anywhere a regular external mbuf is used (in protocols like IP stack and sockets). Is there some reason the existing sys/mbuf.h does not allow external mbuf to call the custom ext_free? Does anyone have strong reason to not do this (like those extra instruction will cause significant overhead, etc)? --mark. From owner-freebsd-atm Thu Oct 17 17:14:02 1996 Return-Path: owner-atm Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA11356 for atm-outgoing; Thu, 17 Oct 1996 17:14:02 -0700 (PDT) Received: from alpo.whistle.com (alpo.whistle.com [207.76.204.38]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA11336; Thu, 17 Oct 1996 17:13:54 -0700 (PDT) Received: from current1.whistle.com (current1.whistle.com [207.76.205.22]) by alpo.whistle.com (8.7.6/8.7.3) with SMTP id RAA00336; Thu, 17 Oct 1996 17:10:31 -0700 (PDT) Message-ID: <3266C049.2F1CF0FB@whistle.com> Date: Thu, 17 Oct 1996 17:08:22 -0700 From: Julian Elischer Organization: Whistle Communications X-Mailer: Mozilla 3.0b6 (X11; I; FreeBSD 2.2-CURRENT i386) MIME-Version: 1.0 To: Mark Tinguely CC: freebsd-atm@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: custom free for external mbuf References: <199610172223.RAA09577@plains.nodak.edu> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-atm@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Mark Tinguely wrote: you are looking at a very old version..... look at 2.2 > > Religous debate time. :) > > in sys/mbuf.h the external mbuf is defined : > > struct m_ext { > caddr_t ext_buf; /* start of buffer */ > void (*ext_free) /* free routine if not the usual */ > __P((caddr_t, u_int)); > u_int ext_size; /* size of buffer, for ext_free */ > }; under 2.2 it is: /* * description of external storage mapped into mbuf, * valid if M_EXT set */ struct m_ext { caddr_t ext_buf; /* start of buffer */ void (*ext_free) /* free routine if not the usual */ __P((caddr_t, u_int)); u_int ext_size; /* size of buffer, for ext_free */ void (*ext_ref) /* add a reference to the ext object */ __P((caddr_t, u_int)); }; (then later we see) > #ifdef notyet > #define MFREE(m, n) \ > { MBUFLOCK(mbstat.m_mtypes[(m)->m_type]--;) \ > if ((m)->m_flags & M_EXT) { \ > if ((m)->m_ext.ext_free) \ > (*((m)->m_ext.ext_free))((m)->m_ext.ext_buf, \ > (m)->m_ext.ext_size); \ > else \ > MCLFREE((m)->m_ext.ext_buf); \ > } \ > (n) = (m)->m_next; \ > FREE((m), mbtypes[(m)->m_type]); \ > } > (more deleted) > > from greping the rest of the sources, this is *the* place an external > mbuf could call another free routine and it is obviously ifdef out. not in 2.2 and it is used heavily by myself > > what I am thinking about doing is making a new breed of mbuf that is > prebuilt to look like an external mbuf, but is permanent in that its free > routine does not throw away the mbuf external data, mbuf structure, nor the > association between mbuf and external data. > > why do I want these permanent mbufs? because I am working with ATM and I don't > want to deal with the allocation, association, tear-down, overhead of many > small PDU packets. sure why not, but don't break what's already there.. also see the method OSF uses which is equally neat, and can use MALLOC'd memory easily for this. (ask matt thomas) > > If I add a new flag entry (say M_PERM) and change the MFREE routine (here > I show changes to the "notyet" part of MFREE, the current MFREE changes > would be simular): > > if ((m)->m_flags & M_EXT) { \ > + if ((m)->m_flags & M_PERM) { \ > + if ((m)->m_ext.ext_free) \ > + (*((m)->m_ext.ext_free))((m), \ > + (m)->m_ext.ext_size); \ > + } \ > + else \ > if ((m)->m_ext.ext_free) \ > (*((m)->m_ext.ext_free))((m)->m_ext.ext_buf, \ > (m)->m_ext.ext_size); \ > else \ > MCLFREE((m)->m_ext.ext_buf); \ > } \ > (n) = (m)->m_next; \ > + if ((m)->m_flags & M_PERM == 0) { \ > FREE((m), mbtypes[(m)->m_type]); \ > } \ > } > please go look at 2.2 before changing things the routine is only ever called from there, but the function pointers might be set up in many palces.. especially in 3rd party software.. so don't break it...please > notice, I need to send the address of the mbuf to my recycle "free" program. > and I cannot let it FREE the mbuf lower down in the MFREE routine. > > with this, I can send these permanent mbufs anywhere a regular external > mbuf is used (in protocols like IP stack and sockets). > > Is there some reason the existing sys/mbuf.h does not allow external mbuf > to call the custom ext_free? it was broken by someone? I do it under bsd4.3 ,osf1, mach and freebsd2.2. > > Does anyone have strong reason to not do this (like those extra instruction > will cause significant overhead, etc)? as 2.2 already does this I doubt it.. julian > > --mark. From owner-freebsd-atm Thu Oct 17 17:33:08 1996 Return-Path: owner-atm Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA12553 for atm-outgoing; Thu, 17 Oct 1996 17:33:08 -0700 (PDT) Received: from root.com (implode.root.com [198.145.90.17]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA12530; Thu, 17 Oct 1996 17:32:58 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by root.com (8.7.6/8.6.5) with SMTP id RAA13429; Thu, 17 Oct 1996 17:34:01 -0700 (PDT) Message-Id: <199610180034.RAA13429@root.com> X-Authentication-Warning: implode.root.com: Host localhost [127.0.0.1] didn't use HELO protocol To: Mark Tinguely cc: freebsd-atm@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: custom free for external mbuf In-reply-to: Your message of "Thu, 17 Oct 1996 17:23:18 CDT." <199610172223.RAA09577@plains.nodak.edu> From: David Greenman Reply-To: dg@root.com Date: Thu, 17 Oct 1996 17:34:01 -0700 Sender: owner-atm@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >If I add a new flag entry (say M_PERM) and change the MFREE routine (here >I show changes to the "notyet" part of MFREE, the current MFREE changes >would be simular): > > if ((m)->m_flags & M_EXT) { \ >+ if ((m)->m_flags & M_PERM) { \ >+ if ((m)->m_ext.ext_free) \ >+ (*((m)->m_ext.ext_free))((m), \ >+ (m)->m_ext.ext_size); \ >+ } \ >+ else \ > if ((m)->m_ext.ext_free) \ > (*((m)->m_ext.ext_free))((m)->m_ext.ext_buf, \ > (m)->m_ext.ext_size); \ > else \ > MCLFREE((m)->m_ext.ext_buf); \ > } \ > (n) = (m)->m_next; \ >+ if ((m)->m_flags & M_PERM == 0) { \ > FREE((m), mbtypes[(m)->m_type]); \ > } \ > } It seems to me that the above can be optimized: if ((m)->m_flags & M_EXT) { \ if ((m)->m_ext.ext_free) {\ if ((m)->m_flags & M_PERM) \ (*((m)->m_ext.ext_free))((m), \ (m)->m_ext.ext_size); \ else \ (*((m)->m_ext.ext_free))((m)->m_ext.ext_buf, \ (m)->m_ext.ext_size); \ } else \ MCLFREE((m)->m_ext.ext_buf); \ } \ This would retain the same performance as the original routine in the standard case of no external free function. -DG David Greenman Core-team/Principal Architect, The FreeBSD Project From owner-freebsd-atm Fri Oct 18 07:51:12 1996 Return-Path: owner-atm Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id HAA27841 for atm-outgoing; Fri, 18 Oct 1996 07:51:12 -0700 (PDT) Received: from plains.nodak.edu (tinguely@plains.NoDak.edu [134.129.111.64]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id HAA27811; Fri, 18 Oct 1996 07:51:06 -0700 (PDT) Received: (from tinguely@localhost) by plains.nodak.edu (8.7.6/8.7.3) id JAA29942; Fri, 18 Oct 1996 09:51:00 -0500 (CDT) Date: Fri, 18 Oct 1996 09:51:00 -0500 (CDT) From: Mark Tinguely Message-Id: <199610181451.JAA29942@plains.nodak.edu> To: julian@whistle.com Subject: Re: custom free for external mbuf Cc: freebsd-atm@freebsd.org, freebsd-hackers@freebsd.org Sender: owner-atm@freebsd.org X-Loop: FreeBSD.org Precedence: bulk a little more background: traditionally, devices tell us how much space it needs just before we do the DMA, this card needs (up to 1024) buffers programmed into a queue before data arrives. We get these buffers back when they are full, or at the end of the packet. We have to feed new buffers into the card and consume data. we need buffers in the card for new input, some buffers will be running through the stacks, some beffers will be waiting to be put into the card. The data buffer has to be around forever, and to have the packet consuption routines deallocating the mbuf structure, only to turn around and reallocate either at packet recieve interrupt time or right away when I put it back on the card's queue seems wasteful. julian@whistle.com says: > you are looking at a very old version..... > look at 2.2 let us not say 2.1.5-RELEASE is old. :) > sure why not, but don't break what's already there.. > please go look at 2.2 before changing things > the routine is only ever called from there, > but the function pointers might be set up in many palces.. > especially in 3rd party software.. > so don't break it...please (I agree, ** WARNING: I am a little preachy below **. for the hysterical-intolarent, below rants about people that can but shouldn't change an existing external free routine). SOMEONE BETTER NOT BE CHANGING AN EXISTING FREE FUNCTION, THEY HAVE NO RIGHT TO DO THAT unless they retain the ORIGINAL free operation AFTER they handled their case. Reading between your lines, it appears we agree that if someone says they have and external free routine, they want it honored. I own these buffers, and I do not want anyone higher on the stack to mess with my free routine. It should matter if I am requesting the freeing of the external data or freeing of the mbuf structure itself, no one should mess with my function request. The problem very few people use these specified external free function, so the odds are it is not set before you put in a function (I bet you don't even check). Let us say more drivers use them (and once one does then you have to assume that anyone below you could). Now what are you going to do to specify your external free? from your prosective, that other driver broke your code. If you change my external free function you broke my code. come on, this external free function idea has been around for several years, people must have had this problem before. Do we need a FIFO of functions (and flags if I add a flag that changes the parameter) or do we tell all designers of external free functions they MUST save the pre-existing external free function (and flag for the same reason as above) and they MUST (restore the flag and) call the pre-existing external free AFTER executing their code. And would the same apply to the external reference now in the 2.2 code? we can delay this problem by saying that will have to change my design and I can't reserve these buffers and I will have to allocate every buffer on the fly, but this is not going to solve the problem of the future when two stacks using the same buffer want to use the external free function and would the same go for the external reference function. (now slipping into silly mode) tell me friends, do you want big government (mbuf structure changes to give us a FIFO of routines), or local solutions (make the driver writers to do the right thing). If elected, I promise ... #include --mark.