Date: Fri, 6 Aug 2010 19:55:56 +0200 From: Alexander Fiveg <pebu3op@googlemail.com> To: Jack Vogel <jfvogel@gmail.com> Cc: freebsd-net@freebsd.org Subject: Re: problem with 82599 controller Message-ID: <201008061955.56908.pebu3op@googlemail.com> In-Reply-To: <AANLkTikdJGqtVKz3JYjcnhMPjvUpi9zhorsA59g_7-Ue@mail.gmail.com> References: <201008061903.25659.pebu3op@googlemail.com> <AANLkTikdJGqtVKz3JYjcnhMPjvUpi9zhorsA59g_7-Ue@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday 06 August 2010 19:20:58 Jack Vogel wrote: > What is the exact PCI ID of your adapter (pciconf -l)? 0x10fb % sysctl dev.ix.0 ~ dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.2.1 dev.ix.0.%driver: ix dev.ix.0.%location: slot=0 function=0 dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 subdevice=0x0003 class=0x020000 dev.ix.0.%parent: pci1 dev.ix.0.stats: -1 dev.ix.0.debug: -1 dev.ix.0.flow_control: 3 dev.ix.0.enable_aim: 1 dev.ix.0.rx_processing_limit: 128 % pciconf -l none0@pci0:0:0:0: class=0x058000 card=0x00000000 chip=0x005e10de rev=0xa3 hdr=0x00 isab0@pci0:0:1:0: class=0x060100 card=0xcb8410de chip=0x005110de rev=0xa3 hdr=0x00 none1@pci0:0:1:1: class=0x0c0500 card=0xcb8410de chip=0x005210de rev=0xa2 hdr=0x00 ohci0@pci0:0:2:0: class=0x0c0310 card=0xcb8410de chip=0x005a10de rev=0xa2 hdr=0x00 ehci0@pci0:0:2:1: class=0x0c0320 card=0xcb8410de chip=0x005b10de rev=0xa3 hdr=0x00 atapci0@pci0:0:6:0: class=0x01018a card=0xcb8410de chip=0x005310de rev=0xf2 hdr=0x00 atapci1@pci0:0:7:0: class=0x010485 card=0xcb8410de chip=0x005410de rev=0xf3 hdr=0x00 atapci2@pci0:0:8:0: class=0x010485 card=0xcb8410de chip=0x005510de rev=0xf3 hdr=0x00 pcib1@pci0:0:9:0: class=0x060401 card=0x00000000 chip=0x005c10de rev=0xa2 hdr=0x01 pcib2@pci0:0:13:0: class=0x060400 card=0x00000000 chip=0x005d10de rev=0xa3 hdr=0x01 pcib3@pci0:0:14:0: class=0x060400 card=0x00000000 chip=0x005d10de rev=0xa3 hdr=0x01 hostb0@pci0:0:24:0: class=0x060000 card=0x00000000 chip=0x11001022 rev=0x00 hdr=0x00 hostb1@pci0:0:24:1: class=0x060000 card=0x00000000 chip=0x11011022 rev=0x00 hdr=0x00 hostb2@pci0:0:24:2: class=0x060000 card=0x00000000 chip=0x11021022 rev=0x00 hdr=0x00 hostb3@pci0:0:24:3: class=0x060000 card=0x00000000 chip=0x11031022 rev=0x00 hdr=0x00 hostb4@pci0:0:25:0: class=0x060000 card=0x00000000 chip=0x11001022 rev=0x00 hdr=0x00 hostb5@pci0:0:25:1: class=0x060000 card=0x00000000 chip=0x11011022 rev=0x00 hdr=0x00 hostb6@pci0:0:25:2: class=0x060000 card=0x00000000 chip=0x11021022 rev=0x00 hdr=0x00 hostb7@pci0:0:25:3: class=0x060000 card=0x00000000 chip=0x11031022 rev=0x00 hdr=0x00 hostb8@pci0:0:26:0: class=0x060000 card=0x00000000 chip=0x11001022 rev=0x00 hdr=0x00 hostb9@pci0:0:26:1: class=0x060000 card=0x00000000 chip=0x11011022 rev=0x00 hdr=0x00 hostb10@pci0:0:26:2: class=0x060000 card=0x00000000 chip=0x11021022 rev=0x00 hdr=0x00 hostb11@pci0:0:26:3: class=0x060000 card=0x00000000 chip=0x11031022 rev=0x00 hdr=0x00 hostb12@pci0:0:27:0: class=0x060000 card=0x00000000 chip=0x11001022 rev=0x00 hdr=0x00 hostb13@pci0:0:27:1: class=0x060000 card=0x00000000 chip=0x11011022 rev=0x00 hdr=0x00 hostb14@pci0:0:27:2: class=0x060000 card=0x00000000 chip=0x11021022 rev=0x00 hdr=0x00 hostb15@pci0:0:27:3: class=0x060000 card=0x00000000 chip=0x11031022 rev=0x00 hdr=0x00 vgapci0@pci0:3:1:0: class=0x030000 card=0x80081002 chip=0x47521002 rev=0x27 hdr=0x00 ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 pcib5@pci0:4:1:0: class=0x060400 card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 ioapic0@pci0:4:1:1: class=0x080010 card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 pcib6@pci0:4:2:0: class=0x060400 card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 ioapic1@pci0:4:2:1: class=0x080010 card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 em0@pci0:5:3:0: class=0x020000 card=0x117a15d9 chip=0x10798086 rev=0x03 hdr=0x00 em1@pci0:5:3:1: class=0x020000 card=0x117a15d9 chip=0x10798086 rev=0x03 hdr=0x00 % uname -pr 8.1-STABLE i386 > > How is it configured, on what kind of hardware, how many queues does it > have, > etc, etc? 8 x CPUs - 8 x queues But by reducing the number of queues this problem will appear more often, because the descriptors will be more often reused. (It happens only by the reusing of descriptors. The first (desc_num*queue_num) packets are ALWAYS correctly! after kldload ixgbe.ko in /var/log/messages: Aug 6 16:21:52 ringmap-2 kernel: ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.2.1> port 0xac00-0xac1f mem 0xfe780000-0xfe7fffff,0xfe77c000-0xfe77ffff irq 16 at device 0.0 on pci1 Aug 6 16:21:52 ringmap-2 kernel: ix0: Using MSIX interrupts with 9 vectors Aug 6 16:21:52 ringmap-2 kernel: ix0: [ITHREAD] Aug 6 16:21:52 ringmap-2 last message repeated 8 times Aug 6 16:21:52 ringmap-2 kernel: ix0: Ethernet address: 00:1b:21:5a:67:70 Aug 6 16:21:52 ringmap-2 kernel: ix0: PCI Express Bus: Speed 2.5Gb/s Width x8 Aug 6 16:21:52 ringmap-2 kernel: ix1: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.2.1> port 0xa800-0xa81f mem 0xfe680000-0xfe6fffff,0xfe778000-0xfe77bfff irq 17 at device 0.1 on pci1 Aug 6 16:21:52 ringmap-2 kernel: ix1: Using MSIX interrupts with 9 vectors Aug 6 16:21:52 ringmap-2 kernel: ix1: RX Descriptors exceed system mbuf max, using default instead! Aug 6 16:21:52 ringmap-2 kernel: ix1: [ITHREAD] Aug 6 16:21:52 ringmap-2 last message repeated 8 times Aug 6 16:21:52 ringmap-2 kernel: ix1: Ethernet address: 00:1b:21:5a:67:71 Aug 6 16:21:52 ringmap-2 kernel: ix1: PCI Express Bus: Speed 2.5Gb/s Width x8 ================================================ Something else ? Thanks, Alex > > Jack > > On Fri, Aug 6, 2010 at 10:03 AM, Alexander Fiveg <pebu3op@googlemail.com>wrote: > > Hello, > > > > the following problem I've faced while working with 82599-controller > > (ixgbe driver): > > > > - During packet capturing, after the number of received packets exceeds > > all allocated descriptors (ixgbe_rxd * ixgbe_num_queues), the next new > > incoming packets will be sometimes DMA'ed into the RAM incorrectly. > > > > Output from my tcpdump session: > > > > % tcpdump -i ix1 > > ... > > 15:36:54.970343 IP 10.0.0.2.discard > 12.0.0.160.2200: UDP, length 58 > > 15:36:55.070357 IP 10.0.0.2.discard > 12.0.0.161.2200: UDP, length 58 > > 15:36:55.170373 c7:49:54:a8:00:0c (oui Unknown) > 35:c5:66:d7:df:e8 (oui > > Unknown), ethertype Unknown (0xd53b), length 100: > > 0x0000: 4d98 ed85 7537 3b6b 3b8f 7102 4b1c 2cd4 M...u7;k;.q.K.,. > > 0x0010: 41a8 2f3d 4faa fc8a a039 0fe2 5960 fad5 A./=O....9..Y`.. > > 0x0020: c8b0 964b b0e0 2213 6aa2 330c ef93 80a9 ...K..".j.3..... > > 0x0030: 6ac8 071b a9bd 0d51 ecca 94ba ac9c 873b j......Q.......; > > 0x0040: a83f aeb0 20f4 cfd9 d1fa 93f3 795c 7d20 .?..........y\}. > > 0x0050: 2993 ). > > 15:36:55.270388 IP 10.0.0.2.discard > 12.0.0.163.2200: UDP, length 58 > > ... > > > > For packets generating I am using Linux kernel packet generator. The > > receive- > > and send-interfaces are connected directly (without any hops in the > > middle). > > By the each new generated packet the destination-IP-address will be > > incremented, so I can proof the correctness of received traffic. This > > error > > appears in both -STABLE and -CURRENT > > > > I guess it is not a software bug and I would like to check out whether > > the controller is in order. > > > > Are there any test setups for 8259x controllers that I could use ? > > > > Thanks, > > Alex > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201008061955.56908.pebu3op>