From owner-freebsd-net@freebsd.org Thu May 3 21:11:25 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F6E4FBACF7 for ; Thu, 3 May 2018 21:11:25 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: from mail-io0-x241.google.com (mail-io0-x241.google.com [IPv6:2607:f8b0:4001:c06::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B1B88484B for ; Thu, 3 May 2018 21:11:25 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: by mail-io0-x241.google.com with SMTP id g1-v6so13981976iob.2 for ; Thu, 03 May 2018 14:11:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=P3NyUKlY7Lez37haz5xyYRywQ7YmC5K5IvrhnR/U1PY=; b=J0h3P18UeOG56aCIlSXdV35lwbF8fmFOQjZNs7rHF6XC6VTeaahEBs3X3GMFAbhnmZ mQ7yVpWxj+qeLpmWmtMlWoEID+Z2zGe3kasfz1ysA/ZpTnUAsRQX1MLBXngFn8xPon61 brQp9TFvl91M3VM5Byx4s7TW/wL44hStVCO0gJq4ggsa2j/cIjD/4GoDv9HayqTxLtLg Qc+EGy/0BOBI+cToAgybsxBPR2Hjvhc3h2/d+pdy60FZfvR+y4WAYCExgB2XNzD/S5ck hauvHyO3YtK5HLnbteCbONthipPpb8lR6hB+Agkb8qRnFbJncmfoHbi8j+oUcYI/o/O9 g7eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=P3NyUKlY7Lez37haz5xyYRywQ7YmC5K5IvrhnR/U1PY=; b=Bgwwob1Tzoq5R0+HGzccEiuGiioY/zLPTV9LHp9nj8SdtfF5tNCYBixCnKOscAWQxp DQZzTuGLc9PThzIhhYP4GN1Hx1Tsdy1WesGzpnf4SlAkLsPa+N3kG+QOqsWj+KhaCHti o6yQcrkFNse1fsm4LI27nyB68SBsOgqGwOq2E/Qyu+JhSU8sfu58/keLTMfzv9pyqyWe pvdIaUkV1oZLQnezpgFRX+FIOcDYgeDlSz6XaT2dpgSjEeFUwkERmNbaAcXZz7/VlNaS dmvOrZOClP8x3ww3/F9VNKKNKVOdiYizEl5esu7vX1KmZ1rxL+dxk3TRxxFxS1YM1TQO fTZg== X-Gm-Message-State: ALQs6tB2EBYCc4eh11F0liyu7UvhAy/GSkNGbDUqmI8ubVgK9Qqnp25s WF8WrD/1LDuO++UaEn+6lO9DF4vGlF9zwF6S5UQ= X-Google-Smtp-Source: AB8JxZoh1p32lpRKBULj5Mx2wS5T+dA7KDdWxOiaOvbFPivoam9ZmVW7AJuIiPopaFmrRn8g+5Kq/F55ZTcJ7taf9AE= X-Received: by 2002:a6b:1801:: with SMTP id 1-v6mr26721273ioy.129.1525381884476; Thu, 03 May 2018 14:11:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.192.131.43 with HTTP; Thu, 3 May 2018 14:11:24 -0700 (PDT) From: Dieter BSD Date: Thu, 3 May 2018 14:11:24 -0700 Message-ID: Subject: Re: AX88179 USB-to-Ethernet is slow and silently corrupts data To: pyunyh@gmail.com, freebsd-net@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 May 2018 21:11:25 -0000 > 10.3-RELEASE > amd64 with ECC memory > VIA VL805 USB 3.0 controller > ue0 is Siig USB-to-Ethernet Chipset: AX88179 > > ugen0.7: at usbus0, cfg=0 md=HOST > spd=SUPER (5.0Gbps) pwr=ON (124mA) > > ue0: flags=8c43 metric 0 > mtu 1500 > options=8000b > inet 10.0.210.66 netmask 0xffffff00 broadcast 10.0.210.255 > nd6 options=29 > media: Ethernet autoselect (1000baseT ) > status: active > > If media is set to "1000baseT " it "works", but slowly, and > received data is silently corrupted. :-( Transmitted data is not > corrupted (tested with > 30 GB). > > ifconfig ue0 -txcsum > "works", but still gives silent data corruption > > ifconfig ue0 -rxcsum (acts the same with or without txcsum) > ping out > netstat sees packets both directions, but ping doesn't see the response: > 8 packets transmitted, 0 packets received, 100.0% packet loss > ping in > netstat sees packets in, but no responses going out > > I can see that some Ethernet controllers would not support checksum offloading, > but it seems to me that turning the checksum offloading off should always > work? (at the expense of more cpu load) pyunyh> Which phy driver is used for axge(4)? pyunyh> You can see the phy driver name below axge(4) attachment in dmesg pyunyh> output. axge0: on usbus2 axge1: on usbus0 miibus4: on axge0 miibus5: on axge1 - Do you use manual media configuration instead of auto-negotiation? They auto configure at 1000. I usually set them to 100 which seems to eliminate the silent data corruption. pyunyh> Does the issue happen at which media speed(10Mbps, 100Mbps or 1000Mbs)? The silent data corruption happens at 1000. 100 seems to eliminate the data corruption but 100 isn't always fast enough. I haven't tried setting the AX88179 to 10 Mbps mode, although I tested it by sending data to it from another machine whioh was running at 10 Mbps, with a Netgear switch converting the 10 Mbps to 1000 Mbps. Using usb2 instead of usb3 also seems to eliminate the date corruption. The AX88179 doesn't seem to care about what Ethernet speed it is running at, or what usb speed it is running at. The silent data corruption happens if it receives too many packets per second from the Ethernet. Reducing Ethernet speed or usb speed are simple ways to reduce how many packets per second it handles. pyunyh> Which direction of packet flow is broken(TX or RX or both)? The silent data corruption happens if it receives too many packets per second from the Ethernet. I have not observed any data corruption when the AX88179 transmits data to the Ethernet. Tested with rcp(1). It seems interesting that it is the receive direction that gets data corruption and the receive direction that fails completely when the rxcsum is turned off. Perhaps related? ue0 is now connected to chipset (AMD 990FX SB950) usb controller usb2 ifconfig ue0 -rxcsum ue0: flags=8843 metric 0 mtu 1500 options=8000a media: Ethernet 100baseTX Sent ue0 a bunch of udp packets from another FreeBSD box. dd if=/dev/zero bs=1k count=50 | nc -4u 10.0.210.66 55555 input ue0 output packets errs idrops bytes packets errs bytes colls drops 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 50 0 0 53000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Receiving process got none of them. nc -4nul 55555 > /var/tmp/file_via_udp_ue0 (file is zero bytes) netstat -s -p udp udp: 0 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 0 dropped due to no socket 0 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 0 delivered 0 datagrams output 0 times multicast source filter matched So netstat sees packets coming in, but does not see any datagrams. Where is the data disappearing? In the hardware? In the device driver? Is "ifconfig -rxcsum" really doing the correct thing to the chip? Is there some way to have RXCSUM,TXCSUM turned on, but also have the cpu verify the checksum? I realize the the whole point of RXCSUM,TXCSUM is to reduce the load on the cpu, but data corruption sucks. To see if a different usb controller made any difference, I ran the same test using ue1 = Tek Republic TUN-300 which has the same AX88179 as the Siig, connected to onboard VIA VL805 USB 3.0 controller, and it acts exactly the same as the Siig. So no difference seen between the two usb controllers. Both ue0 & ue1 are running at 100baseTX which appears to eliminate the silent data corruption seen with rcp(1) when they are running at 1000. I also tried the same test with tcp instead of udp. Same results, zero length file and no checksum errors reported by netstat -s. So the protocol doesn't matter. "ifconfig -rxcsum" appears to stop the flow of all incoming packets, icmp. udp, and tcp.