Date: Wed, 12 Nov 1997 14:41:43 -0200 (EDT) From: Rodolfo Heitor Gevaerd de Faria <rodolfo@ravel.ufrj.br> To: aalves@dei.uc.pt Cc: freebsd-atm@FreeBSD.ORG Subject: Re: ATM driver & udp stream Message-ID: <199711121641.OAA00538@galileo.ravel.ufrj.br> In-Reply-To: <Pine.BSF.3.96.971112005854.20573A-100000@zorg.dei.uc.pt> from Antonio Luis Alves at "Nov 12, 97 03:06:46 am"
next in thread | previous in thread | raw e-mail | index | archive | help
Antonio Luis Alves was saying that,
^
^ I have been working on a port to FreeBSD of a major manufacturer atm
If you don't mind, what is the "major manufacturer" which you're porting ?
^ implementation which includes classical ip and lan emulation. The most
^ difficult part was the development of the low level code to drive the pci
^ adapters as this was my first project working inside the kernel and also
^ on FreeBSD. At this time most of the work as been done and I have only the
^ lan emu code and ILMI to finish the port.
^
^ We have been running the port for several months on Pentium machines with
^ FreeBSD 2.2.2 Release without any problems, until two weeks ago when I
^ started to do some performance tests. One of tests ( a UDP stream ) was
^ crashing the driver allays at the same place in the pdu receive routine.
^
^ The driver allocs several buffers at init time to be used by the card to
^ write the received pdu's. The supply routine allocs an mbuf for each
^ buffer and supplies the card with the dma address of the buffer and
^ the mbuf handle. When the card interrupts, the pdu receive routine reads
^ the receive pdu queue. Each entry in this queue is a receive pdu
^ descriptor. Each descriptor contains several information related to the
^ received pdu, including several segments to form the pdu. Each segment
^ contains the mbuf handle ( allocated by the supply routine ) and the
^ number of bytes in the buffer written by the card. At this time I use 4k
^ buffers and each pdu descriptor with max 16 segments to be able to receive
^ a max pdu of 64k. The mbuf handles are read by the card from the supply
^ queue and written to the segment descriptors after the buffers have been
^ filled with cells payloads.
^ The mbuf chain forming the received pdu is formed and the pdu is send
^ upstairs. Each mbuf in the chain as the ext_buf pointing to the buffer,
^ and pointers to the external_free_routine and an external_ref_function.
^ The buffers will return to the free buffer pool on the driver as soon as
^ the external_free_routine is called and the reference count allows it.
^ These buffers are then used again by the supply routine.
^ When the card is low on buffers, the pdu receive routine does not send the
^ external buffers upstairs, instead it copies the data to mbufs with
^ clusters, so the buffers are made available again to the card faster.
^
^ All was working well until I made a UDP stream test. The sender was a
^ Pentium 233mmx and on the receive side a Pentium 166. The test is a normal
^ netperf UDP_STREAM with default values for the datagram, which is the max
^ udp datagram 9216. The driver crashes at the pdu receive routine when
^ accessing the segments which form the pdu. With the debugger I can see that
^ the mbuf handles are correct but the data on the mbuf is not. Some times
^ the m_type is MT_FREE and m->next as got a valid pointer on it, and other
^ times all the members of the mbuf are zero. On a normal situation the
^ mbufs would be of type MT_DATA, and correctly initialized with the
^ pointers to the external free and reference function, m_len, ext_size and
^ ext_buffer.
^ >From the firmware guide it says the card never touches the mbuf, it just
^ pass the mbuf handle received from the supply routine to the segments on
^ the received pdu descriptor.
^
^ These problem only happens when there is fragmentation and reassembly of
^ packets. If I make the test with the udp datagram below the mtu of the
^ interface ( 9188 ) it goes well without any problems. Also if I pace the
^ transfer with delays and small burst it also works for the max datagram. I
^ made also other tests from a sun as the sending machine ( which is very
^ slow compared with the Pentiums we have here ) and it works ok even with
^ the max datagram of 9216.
^
^ So I suppose there is something related to the full blast of the Pentium
^ 233mmx and the reassembly of the packets at the ip layer. I also think
^ that at this speed ( 155MPS cards ) most of the fragments ( mbuf w/
^ external buffer ) are kept at the ip fragment queue and then as soon as
^ the card gets low on buffers , the copy to mbufs w/ clusters routine
^ enters the game and starts asking the system for a lot of mbufs and
^ clusters.
^
^ At this time I am running out of ideas of where to look to find the
^ problem, so I decided to do a post here and ask for some ideas.
^ I have made already lots of modifications in the driver with lots of
^ paranoid checks but without any success.
^
^ Sorry for the long post, but I tried to explain as much as possible for a
^ better evaluation of this problem.
^
^ Thank you.
^
^
^ Antonio Alves
^
Rodolfo H G Faria
<rodolfo@ravel.ufrj.br>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199711121641.OAA00538>