From owner-freebsd-net@FreeBSD.ORG Fri Aug 6 18:06:15 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CD981065675 for ; Fri, 6 Aug 2010 18:06:15 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 560B18FC0C for ; Fri, 6 Aug 2010 18:06:13 +0000 (UTC) Received: by wyj26 with SMTP id 26so10007788wyj.13 for ; Fri, 06 Aug 2010 11:06:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=ftTA6AmVDAHpk/JcGFEzhbd/30j3LRQcLQHFWMiIDiI=; b=PbcDmzsgLgK/RDmvxX9urJIKfX90xqXUmnvM0MAsdc3vNP2IFHv0aJ3PfGnkVFm8q4 /kf3mzskdZKW3VKinB7UlacEsBke9BfRQmTI3bam4ubPymgXLwZNAoSCzt5TNSYkoiXH HkzABUdrglSBhcN1oatPMQGRku4dHjk3zyPf4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=F2BtoTn9do5zPnfiK+lX8QyAlACpCDBuqCZhCHvNJ0+XYM+bmFIfJGA39q+61UEbuP mkdHkESaHB/VpVbdM7/G37ntCTO+yp3XkQzC2jYB9Ttc/qzV6vHeyzwU62mCsqXi3pfs PFvV6xXN/kzHbyLwQSJ3F1imu5saUm4RJz76c= MIME-Version: 1.0 Received: by 10.216.2.129 with SMTP id 1mr1179683wef.40.1281117973135; Fri, 06 Aug 2010 11:06:13 -0700 (PDT) Received: by 10.216.48.6 with HTTP; Fri, 6 Aug 2010 11:06:12 -0700 (PDT) In-Reply-To: <201008061955.56908.pebu3op@googlemail.com> References: <201008061903.25659.pebu3op@googlemail.com> <201008061955.56908.pebu3op@googlemail.com> Date: Fri, 6 Aug 2010 11:06:12 -0700 Message-ID: From: Jack Vogel To: pebu3op@googlemail.com Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org Subject: Re: problem with 82599 controller X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Aug 2010 18:06:15 -0000 Ouch, you're running 32 bit? Can you compare 64 bit and see if that has any effect? Jack On Fri, Aug 6, 2010 at 10:55 AM, Alexander Fiveg wrote: > On Friday 06 August 2010 19:20:58 Jack Vogel wrote: > > What is the exact PCI ID of your adapter (pciconf -l)? > 0x10fb > > % sysctl dev.ix.0 > ~ > dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - > 2.2.1 > dev.ix.0.%driver: ix > dev.ix.0.%location: slot=0 function=0 > dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086 > subdevice=0x0003 class=0x020000 > dev.ix.0.%parent: pci1 > dev.ix.0.stats: -1 > dev.ix.0.debug: -1 > dev.ix.0.flow_control: 3 > dev.ix.0.enable_aim: 1 > dev.ix.0.rx_processing_limit: 128 > > > % pciconf -l > none0@pci0:0:0:0: class=0x058000 card=0x00000000 chip=0x005e10de > rev=0xa3 hdr=0x00 > isab0@pci0:0:1:0: class=0x060100 card=0xcb8410de chip=0x005110de > rev=0xa3 hdr=0x00 > none1@pci0:0:1:1: class=0x0c0500 card=0xcb8410de chip=0x005210de > rev=0xa2 hdr=0x00 > ohci0@pci0:0:2:0: class=0x0c0310 card=0xcb8410de chip=0x005a10de > rev=0xa2 hdr=0x00 > ehci0@pci0:0:2:1: class=0x0c0320 card=0xcb8410de chip=0x005b10de > rev=0xa3 hdr=0x00 > atapci0@pci0:0:6:0: class=0x01018a card=0xcb8410de chip=0x005310de > rev=0xf2 hdr=0x00 > atapci1@pci0:0:7:0: class=0x010485 card=0xcb8410de chip=0x005410de > rev=0xf3 hdr=0x00 > atapci2@pci0:0:8:0: class=0x010485 card=0xcb8410de chip=0x005510de > rev=0xf3 hdr=0x00 > pcib1@pci0:0:9:0: class=0x060401 card=0x00000000 chip=0x005c10de > rev=0xa2 hdr=0x01 > pcib2@pci0:0:13:0: class=0x060400 card=0x00000000 chip=0x005d10de > rev=0xa3 hdr=0x01 > pcib3@pci0:0:14:0: class=0x060400 card=0x00000000 chip=0x005d10de > rev=0xa3 hdr=0x01 > hostb0@pci0:0:24:0: class=0x060000 card=0x00000000 chip=0x11001022 > rev=0x00 hdr=0x00 > hostb1@pci0:0:24:1: class=0x060000 card=0x00000000 chip=0x11011022 > rev=0x00 hdr=0x00 > hostb2@pci0:0:24:2: class=0x060000 card=0x00000000 chip=0x11021022 > rev=0x00 hdr=0x00 > hostb3@pci0:0:24:3: class=0x060000 card=0x00000000 chip=0x11031022 > rev=0x00 hdr=0x00 > hostb4@pci0:0:25:0: class=0x060000 card=0x00000000 chip=0x11001022 > rev=0x00 hdr=0x00 > hostb5@pci0:0:25:1: class=0x060000 card=0x00000000 chip=0x11011022 > rev=0x00 hdr=0x00 > hostb6@pci0:0:25:2: class=0x060000 card=0x00000000 chip=0x11021022 > rev=0x00 hdr=0x00 > hostb7@pci0:0:25:3: class=0x060000 card=0x00000000 chip=0x11031022 > rev=0x00 hdr=0x00 > hostb8@pci0:0:26:0: class=0x060000 card=0x00000000 chip=0x11001022 > rev=0x00 hdr=0x00 > hostb9@pci0:0:26:1: class=0x060000 card=0x00000000 chip=0x11011022 > rev=0x00 hdr=0x00 > hostb10@pci0:0:26:2: class=0x060000 card=0x00000000 chip=0x11021022 > rev=0x00 hdr=0x00 > hostb11@pci0:0:26:3: class=0x060000 card=0x00000000 chip=0x11031022 > rev=0x00 hdr=0x00 > hostb12@pci0:0:27:0: class=0x060000 card=0x00000000 chip=0x11001022 > rev=0x00 hdr=0x00 > hostb13@pci0:0:27:1: class=0x060000 card=0x00000000 chip=0x11011022 > rev=0x00 hdr=0x00 > hostb14@pci0:0:27:2: class=0x060000 card=0x00000000 chip=0x11021022 > rev=0x00 hdr=0x00 > hostb15@pci0:0:27:3: class=0x060000 card=0x00000000 chip=0x11031022 > rev=0x00 hdr=0x00 > vgapci0@pci0:3:1:0: class=0x030000 card=0x80081002 chip=0x47521002 > rev=0x27 hdr=0x00 > ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 > hdr=0x00 > ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 > hdr=0x00 > pcib5@pci0:4:1:0: class=0x060400 card=0x00000000 chip=0x74581022 > rev=0x12 hdr=0x01 > ioapic0@pci0:4:1:1: class=0x080010 card=0x74591022 chip=0x74591022 > rev=0x12 hdr=0x00 > pcib6@pci0:4:2:0: class=0x060400 card=0x00000000 chip=0x74581022 > rev=0x12 hdr=0x01 > ioapic1@pci0:4:2:1: class=0x080010 card=0x74591022 chip=0x74591022 > rev=0x12 hdr=0x00 > em0@pci0:5:3:0: class=0x020000 card=0x117a15d9 chip=0x10798086 rev=0x03 > hdr=0x00 > em1@pci0:5:3:1: class=0x020000 card=0x117a15d9 chip=0x10798086 rev=0x03 > hdr=0x00 > > % uname -pr > 8.1-STABLE i386 > > > > > > How is it configured, on what kind of hardware, how many queues does it > > have, > > etc, etc? > 8 x CPUs - 8 x queues > But by reducing the number of queues this problem will appear more often, > because the descriptors will be more often reused. (It happens only by the > reusing of descriptors. The first (desc_num*queue_num) packets are ALWAYS > correctly! > > > after kldload ixgbe.ko in /var/log/messages: > > Aug 6 16:21:52 ringmap-2 kernel: ix0: Network > Driver, Version - 2.2.1> port 0xac00-0xac1f mem > 0xfe780000-0xfe7fffff,0xfe77c000-0xfe77ffff irq 16 at device 0.0 on pci1 > Aug 6 16:21:52 ringmap-2 kernel: ix0: Using MSIX interrupts with 9 vectors > Aug 6 16:21:52 ringmap-2 kernel: ix0: [ITHREAD] > Aug 6 16:21:52 ringmap-2 last message repeated 8 times > Aug 6 16:21:52 ringmap-2 kernel: ix0: Ethernet address: 00:1b:21:5a:67:70 > Aug 6 16:21:52 ringmap-2 kernel: ix0: PCI Express Bus: Speed 2.5Gb/s Width > x8 > Aug 6 16:21:52 ringmap-2 kernel: ix1: Network > Driver, Version - 2.2.1> port 0xa800-0xa81f mem > 0xfe680000-0xfe6fffff,0xfe778000-0xfe77bfff irq 17 at device 0.1 on pci1 > Aug 6 16:21:52 ringmap-2 kernel: ix1: Using MSIX interrupts with 9 vectors > Aug 6 16:21:52 ringmap-2 kernel: ix1: RX Descriptors exceed system mbuf > max, > using default instead! > Aug 6 16:21:52 ringmap-2 kernel: ix1: [ITHREAD] > Aug 6 16:21:52 ringmap-2 last message repeated 8 times > Aug 6 16:21:52 ringmap-2 kernel: ix1: Ethernet address: 00:1b:21:5a:67:71 > Aug 6 16:21:52 ringmap-2 kernel: ix1: PCI Express Bus: Speed 2.5Gb/s Width > x8 > > ================================================ > > Something else ? > > Thanks, > Alex > > > > > > Jack > > > > On Fri, Aug 6, 2010 at 10:03 AM, Alexander Fiveg > wrote: > > > Hello, > > > > > > the following problem I've faced while working with 82599-controller > > > (ixgbe driver): > > > > > > - During packet capturing, after the number of received packets exceeds > > > all allocated descriptors (ixgbe_rxd * ixgbe_num_queues), the next new > > > incoming packets will be sometimes DMA'ed into the RAM incorrectly. > > > > > > Output from my tcpdump session: > > > > > > % tcpdump -i ix1 > > > ... > > > 15:36:54.970343 IP 10.0.0.2.discard > 12.0.0.160.2200: UDP, length 58 > > > 15:36:55.070357 IP 10.0.0.2.discard > 12.0.0.161.2200: UDP, length 58 > > > 15:36:55.170373 c7:49:54:a8:00:0c (oui Unknown) > 35:c5:66:d7:df:e8 > (oui > > > Unknown), ethertype Unknown (0xd53b), length 100: > > > 0x0000: 4d98 ed85 7537 3b6b 3b8f 7102 4b1c 2cd4 > M...u7;k;.q.K.,. > > > 0x0010: 41a8 2f3d 4faa fc8a a039 0fe2 5960 fad5 > A./=O....9..Y`.. > > > 0x0020: c8b0 964b b0e0 2213 6aa2 330c ef93 80a9 > ...K..".j.3..... > > > 0x0030: 6ac8 071b a9bd 0d51 ecca 94ba ac9c 873b > j......Q.......; > > > 0x0040: a83f aeb0 20f4 cfd9 d1fa 93f3 795c 7d20 > .?..........y\}. > > > 0x0050: 2993 ). > > > 15:36:55.270388 IP 10.0.0.2.discard > 12.0.0.163.2200: UDP, length 58 > > > ... > > > > > > For packets generating I am using Linux kernel packet generator. The > > > receive- > > > and send-interfaces are connected directly (without any hops in the > > > middle). > > > By the each new generated packet the destination-IP-address will be > > > incremented, so I can proof the correctness of received traffic. This > > > error > > > appears in both -STABLE and -CURRENT > > > > > > I guess it is not a software bug and I would like to check out whether > > > the controller is in order. > > > > > > Are there any test setups for 8259x controllers that I could use ? > > > > > > Thanks, > > > Alex > > > _______________________________________________ > > > freebsd-net@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > >