From owner-freebsd-hackers@freebsd.org Mon Apr 23 05:57:10 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CADD0FBF972 for ; Mon, 23 Apr 2018 05:57:09 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6186481A46 for ; Mon, 23 Apr 2018 05:57:09 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: by mail-it0-x236.google.com with SMTP id 186-v6so9254634itu.0 for ; Sun, 22 Apr 2018 22:57:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=DOofGcZbLqEAXybNFWDO4IqlDt2VuGdrKT3RUtM2znk=; b=bLBU74KEni/m6ThhePmcf6c/4rkhZSRj0IGaK/bH8gCTb3VL2vV7dv13xH0KI9pn+P 1x2ph7POuf/+FRkLicZeq4t/eoJf7jKuncUX2NQFrlgxk3p5GEKO5OggQqJf56u4DNWC cBH7X2ZjmfwjfxFVpAhqqhw3IvddUKN1+CExZI8La+6LsPX4sPNvTW3byF153kk9e1ot cS6Scrwfp8vmsssKT3B9Es77m0NrB21mBxKyAIlbivB3/nAcWDVnmgC6KZvrJkiiDhNM ngvIfU3vpUD3wjaP+yrmbQrrXd+GtDT3K0UXVMGHefCsBCC/s6nfVnvDrHtXrFHhuWMr QFeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=DOofGcZbLqEAXybNFWDO4IqlDt2VuGdrKT3RUtM2znk=; b=IOVavG+DZdzvBpCbn7+vMthBbcl+VKMW/91zl+Z3cdAR2l+N96R50J0456iUT7GE+g 12XQWUnm7RvbHPgorrcj26lNT6Leo6atuqrCko6ia/ExzXeS9Z7HlCgVewyL0Vgi2Ohn WvATHOSeE8U5/Zam+oEo9WUpi44bfRgWDCzL+UMIUeDVJqdNY1BTeG1eExzkdaHyJKsD 6GC2OcHlj7AmHuvf5BYj7z2TaAvJpu7c/w4l2mA4Aw5pe8U5km6sfftJDNs66Kw0q1uI eO7XrO8pLIAdC9wd6R+ScGwCdZMO3cCYwU2tHWWssz0a5hvqcem2spoDL3CEYhc9tUgL cRyg== X-Gm-Message-State: ALQs6tALY9XF0j0d7st3pI0ZAfSEogGvFPvXFrsFYrzIy12XDHGBAnhL 5JH+0bUMUZowksLwybTTm5Wpu5RZWDdMeAOHnQc= X-Google-Smtp-Source: AIpwx48fuY+0xZKbn6EDnSY9xv1GzNvZ5XLpQCRf1P82m5Kq2MAos3NzFJ2fxQCm0IfzM7v6plN2l3H8a1+AL31nPsQ= X-Received: by 2002:a24:10c7:: with SMTP id 190-v6mr12488905ity.38.1524463027756; Sun, 22 Apr 2018 22:57:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.192.131.43 with HTTP; Sun, 22 Apr 2018 22:57:07 -0700 (PDT) From: Dieter BSD Date: Sun, 22 Apr 2018 22:57:07 -0700 Message-ID: Subject: Re: Realtek re(4) driver To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2018 05:57:10 -0000 With several days more data, the Realtek driver is slightly different than the stock FreeBSD 10.3 driver, but it still fails a lot, with both TCP and UDP. > Has anyone tried similar testing? > Suggestions of knobs to turn, or other things to try? > I need to receive data via UDP without dropping packets. > Closed source sucks, but I'm stuck with it, and thus with UDP. Gary typed: >> Everyone blames re, but AFAICT >> no-one switched to a non-Realtek chip to run tests and prove that, >> yes, re is really the cause of all the network problems. I have a box with nfe(4) and bge(4) (BCM5750) that can receive data with UDP or TCP without error. This has been working for several years. But a newer, faster machine with re(4) 8111E & 8111F cannot. The nfe is in the chipset, and the BCM5750 is onboard. I had google look for an expansion card with BCM5750 but found nothing. I'm looking into a Broadcom 5719 card, but there is at least one open PR against it, so... Alex typed: > surely it's not a NFS issue NFS is a bug. By design. "Not a File System" My machines are NFS-free. > The only thing that link all bug reports > (and there are tons) is the use of re card sustaining gigabit (duplex) > transfers. My re run (1000baseT ) but the pauses happen even with light traffic. You don't have to run them full blast with rcp/ftp/whatever to get failures. STefan typed: > Well, but you know that the U in UDP means unreliable? Well, :-) you didn't pay attention when I said exactly that: >>> Something like a rcp(1) with another >>> FreeBSD box merely runs slower due to the stalls, but if the other end >>> has buggy network code and/or if the transfer is forced to use UDP >>> (Unreliable Data Protocol) data is lost. :-( >> With reasonably decent network software and TCP, this is only a minor >> problem. However with buggy network software (like a "black box" >> with closed source firmware and maybe a transmit buffer that is *way* >> too small, and/or true real time requirements), and/or with a brain dead >> protocol like UDP, you can lose data. rcp(1) to 8111F using Realtek driver Both machines basically idle. mtu=9000 on both ends but appears to be using 1500 anyway? netstat -w 1 -d -I re0 input re0 output packets errs idrops bytes packets errs bytes colls drops 5 0 0 330 6 0 755 0 0 4 0 0 264 4 0 564 0 0 5 0 0 331 5 0 630 0 0 17454 0 0 26176844 17453 0 1152264 0 0 67883 0 0 101023187 67880 0 4481248 0 0 52133 0 0 77141014 52140 0 3444580 0 0 72410 0 0 107980584 72409 0 4780384 0 0 80736 0 0 120645177 80735 0 5328810 0 0 71898 0 0 107298972 71899 0 4745634 0 0 14452 0 0 21510681 14475 0 955577 0 0 5 0 0 330 6 0 755 0 0 4 0 0 264 4 0 564 0 0 5 0 0 331 5 0 630 0 0 4 0 0 264 4 0 564 0 0 65323 0 0 97533279 65321 0 4311552 0 0 28383 0 0 42398766 28385 0 1873769 0 0 80759 0 0 120315366 80756 0 5330394 0 0 80992 0 0 120693553 80995 0 5345772 0 0 80932 0 0 120692488 80930 0 5341746 0 0 78905 0 0 117912275 78922 0 5209152 0 0 80158 0 0 119592452 80159 0 5290728 0 0 input re0 output packets errs idrops bytes packets errs bytes colls drops 80851 0 0 120590446 80852 0 5336591 0 0 80849 0 0 120589243 80847 0 5336334 0 0 80858 0 0 120598524 80860 0 5336928 0 0 15085 0 0 22529747 15107 0 997296 0 0 4 0 0 264 4 0 564 0 0 4 0 0 264 4 0 564 0 0 5 0 0 331 5 0 630 0 0 4 0 0 264 4 0 564 0 0 63944 0 0 95447185 63943 0 4220604 0 0 80588 0 0 120276192 80589 0 5319233 0 0 80721 0 0 120480130 80722 0 5328011 0 0 79870 0 0 119139685 79870 0 5271720 0 0 79425 0 0 118587402 79442 0 5243472 0 0 80609 0 0 120333747 80609 0 5320494 0 0 80852 0 0 120624944 80850 0 5336532 0 0 80827 0 0 120555558 80831 0 5335073 0 0 79185 0 0 118137499 79185 0 5226510 0 0 76417 0 0 114252562 76431 0 5045010 0 0 79896 0 0 119330393 79900 0 5273436 0 0 79455 0 0 118627478 79456 0 5244330 0 0 77667 0 0 116033902 77662 0 5126256 0 0 input re0 output packets errs idrops bytes packets errs bytes colls drops 79517 0 0 118735507 79515 0 5248547 0 0 78557 0 0 117304898 78578 0 5186118 0 0 79607 0 0 118742143 79625 0 5255616 0 0 79776 0 0 119140720 79775 0 5265450 0 0 75105 0 0 112725906 75107 0 4957230 0 0 79214 0 0 118774181 79214 0 5228424 0 0 13090 0 0 19520508 13113 0 865692 0 0 Same as above except to 2nd 8111F port. input re1 output packets errs idrops bytes packets errs bytes colls drops 75170 0 0 112288950 75165 0 4960956 0 0 74149 0 0 110808698 74150 0 4893834 0 0 75088 0 0 112276960 75088 0 4955808 0 0 74567 0 0 111313160 74566 0 4921356 0 0 77064 0 0 115370280 77065 0 5086290 0 0 71606 0 0 107343414 71622 0 4727052 0 0 70831 0 0 105956278 70832 0 4674849 0 0 70727 0 0 106127094 70725 0 4667916 0 0 73427 0 0 110111478 73425 0 4846182 0 0 72519 0 0 108763678 72522 0 4786320 0 0 5661 0 0 8423884 5681 0 374880 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 196 0 0 0 0 0 1 0 0 98 0 0 0 0 0 563 0 0 838984 561 0 37026 0 0 29144 0 0 43754296 29142 0 1923438 0 0 73986 0 0 110909436 73987 0 4883142 0 0 39601 0 0 59343834 39621 0 2614920 0 0 0 0 0 0 0 0 0 0 0 3 0 0 201 0 0 0 0 0 input re1 output packets errs idrops bytes packets errs bytes colls drops 0 0 0 0 0 0 0 0 0 1 0 0 98 0 0 0 0 0 567 0 0 839958 567 0 37422 0 0 1 0 0 98 0 0 0 0 0 38003 0 0 56780696 38000 0 2508066 0 0 75439 0 0 112523550 75438 0 4978908 0 0 77058 0 0 114890108 77059 0 5085894 0 0 77471 0 0 115473190 77470 0 5113020 0 0 76814 0 0 114644092 76815 0 5069790 0 0 78120 0 0 116633730 78118 0 5155788 0 0 74853 0 0 111727082 74854 0 4940298 0 0 77626 0 0 115697756 77623 0 5123250 0 0 70804 0 0 105582200 70804 0 4672998 0 0 76722 0 0 114492148 76722 0 5063652 0 0 29562 0 0 44157654 29582 0 1952346 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 568 0 0 840298 566 0 37356 0 0 0 0 0 0 0 0 0 0 0 input re1 output packets errs idrops bytes packets errs bytes colls drops 43130 0 0 64590180 43128 0 2846514 0 0 29010 0 0 43374204 29032 0 1916046 0 0 0 0 0 0 0 0 0 0 0 2 0 0 134 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 515 0 0 770942 515 0 33990 0 0 17903 0 0 26853958 17902 0 1181598 0 0 74854 0 0 111699988 74851 0 4940166 0 0 77120 0 0 115061536 77117 0 5089722 0 0 76857 0 0 114664362 76855 0 5072562 0 0 75817 0 0 113174146 75820 0 5003988 0 0 76737 0 0 114435914 76737 0 5064642 0 0 78495 0 0 117128814 78493 0 5180604 0 0 75723 0 0 112970948 75724 0 4997718 0 0 76501 0 0 114196604 76500 0 5049066 0 0 76187 0 0 113779382 76189 0 5028408 0 0 77236 0 0 115269384 77235 0 5097510 0 0 75300 0 0 112389128 75300 0 4969800 0 0 76492 0 0 114455568 76492 0 5048472 0 0 75573 0 0 113011418 75572 0 4987752 0 0 input re1 output packets errs idrops bytes packets errs bytes colls drops 73331 0 0 109541752 73330 0 4839780 0 0 76732 0 0 114631816 76733 0 5064378 0 0 74400 0 0 111067456 74399 0 4910334 0 0 75974 0 0 113463012 75975 0 5014350 0 0 76312 0 0 113999448 76311 0 5036526 0 0 43806 0 0 65415884 43844 0 2893638 0 0 1 0 0 66 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Same as above except to onboard 8111E. input re2 output packets errs idrops bytes packets errs bytes colls drops 53122 0 0 80345556 0 0 0 0 0 53984 0 0 81332624 0 0 0 0 0 61394 0 0 92785548 0 0 0 0 0 53421 0 0 80696090 0 0 0 0 0 30857 0 0 46597698 0 0 0 0 0 76124 0 0 115239504 0 0 0 0 0 69172 0 0 104706904 0 0 0 0 0 76094 0 0 115191956 0 0 0 0 0 76849 0 0 116329874 0 0 0 0 0 68788 0 0 104120096 0 0 0 0 0 69233 0 0 104802146 0 0 0 0 0 50527 0 0 76465702 0 0 0 0 0 75916 0 0 114920192 0 0 0 0 0 66062 0 0 99990092 0 0 0 0 0 76257 0 0 115436194 0 0 0 0 0 47074 0 0 71233028 0 0 0 0 0 51788 0 0 78359104 0 0 0 0 0 44486 0 0 67316276 0 0 0 0 0 76159 0 0 115288566 0 0 0 0 0 64766 0 0 98037852 0 0 0 0 0 76786 0 0 116237604 0 0 0 0 0 input re2 output packets errs idrops bytes packets errs bytes colls drops 50676 0 0 76688424 0 0 0 0 0 76918 0 0 116440932 0 0 0 0 0 65723 0 0 99488694 0 0 0 0 0 73756 0 0 111653288 0 0 0 0 0 71393 0 0 108070674 0 0 0 0 0 63089 0 0 95485346 0 0 0 0 0 69588 0 0 105333248 0 0 0 0 0 59215 0 0 89617670 0 0 0 0 0 57339 0 0 86752950 0 0 0 0 0 75739 0 0 114651942 0 0 0 0 0 68442 0 0 103594692 0 0 0 0 0 49990 0 0 75649100 0 0 0 0 0 27710 0 0 41940348 0 0 0 0 0 50005 0 0 75694058 0 0 0 0 0 76748 0 0 116183384 0 0 0 0 0 52088 0 0 78847168 0 0 0 0 0 76816 0 0 116287048 0 0 0 0 0 70232 0 0 106301624 0 0 0 0 0 76467 0 0 115754502 0 0 0 0 0 68979 0 0 104412918 0 0 0 0 0 76694 0 0 116101068 0 0 0 0 0 input re2 output packets errs idrops bytes packets errs bytes colls drops 68857 0 0 104224114 0 0 0 0 0 76545 0 0 115871154 0 0 0 0 0 65996 0 0 99843912 0 0 0 0 0 76674 0 0 116070772 0 0 0 0 0 68095 0 0 103070230 0 0 0 0 0 74877 0 0 113319642 0 0 0 0 0 39228 0 0 59366824 0 0 0 0 0 53026 0 0 80203068 0 0 0 0 0 ... 77799 0 0 117779926 0 0 0 0 0 72666 0 0 110007220 0 0 0 0 0 69173 0 0 104681578 0 0 0 0 0 55918 0 0 84169172 0 0 0 0 0 24127 0 0 36195390 0 0 0 0 0 input re2 output packets errs idrops bytes packets errs bytes colls drops 0 0 0 0 0 0 0 0 0 19706 0 0 29578468 0 0 0 0 0 53262 0 0 79974084 0 0 0 0 0 59033 0 0 88835122 0 0 0 0 0 54469 0 0 81830514 0 0 0 0 0 55005 0 0 82842706 0 0 0 0 0 47867 0 0 72331934 0 0 0 0 0 65468 0 0 99099624 0 0 0 0 0 80572 0 0 121980632 0 0 0 0 0 58201 0 0 88091802 0 0 0 0 0 51948 0 0 78614144 0 0 0 0 0 80053 0 0 121196242 0 0 0 0 0 52331 0 0 79200470 0 0 0 0 0 80123 0 0 121302582 0 0 0 0 0 43414 0 0 65698001 0 0 0 0 0 77961 0 0 117995130 0 0 0 0 0 62287 0 0 94226926 0 0 0 0 0 75558 0 0 114290892 0 0 0 0 0 64228 0 0 97158320 0 0 0 0 0 59981 0 0 90654186 0 0 0 0 0 43520 0 0 65739096 0 0 0 0 0 input re2 output packets errs idrops bytes packets errs bytes colls drops 56570 0 0 85335700 0 0 0 0 0 50066 0 0 75384196 0 0 0 0 0 55959 0 0 84239934 0 0 0 0 0 53192 0 0 80101232 0 0 0 0 0 49002 0 0 73982980 0 0 0 0 0 49713 0 0 74993370 0 0 0 0 0 43984 0 0 66011800 0 0 0 0 0 54324 0 0 81935448 0 0 0 0 0 48502 0 0 72734828 0 0 0 0 0 26191 0 0 39411806 0 0 0 0 0 0 0 0 0 0 0 0 0 0 42582 0 0 64021036 0 0 0 0 0 47216 0 0 70880104 0 0 0 0 0 52239 0 0 78498398 0 0 0 0 0 52517 0 0 79279690 0 0 0 0 0 44212 0 0 66384032 0 0 0 0 0 57225 0 0 86126642 0 0 0 0 0 47902 0 0 72238300 0 0 0 0 0 44933 0 0 67508656 0 0 0 0 0 54967 0 0 82411976 0 0 0 0 0 50724 0 0 76288600 0 0 0 0 0 ... input re2 output packets errs idrops bytes packets errs bytes colls drops 56426 0 0 84726988 0 0 0 0 0 53332 0 0 79840392 0 0 0 0 0 52763 0 0 78943630 0 0 0 0 0 52190 0 0 78016892 0 0 0 0 0 56889 0 0 85597554 0 0 0 0 0 55265 0 0 82915642 0 0 0 0 0 52162 0 0 78061690 0 0 0 0 0 58041 0 0 87326532 0 0 0 0 0 53244 0 0 79740088 0 0 0 0 0 51387 0 0 76791862 0 0 0 0 0 54187 0 0 81106878 0 0 0 0 0 51846 0 0 77541604 0 0 0 0 0 42316 0 0 63282584 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 590 0 0 890908 0 0 0 0 0 40253 0 0 60411290 0 0 0 0 0 56653 0 0 85264434 0 0 0 0 0 53499 0 0 80125838 0 0 0 0 0 [ ... ] input re2 output packets errs idrops bytes packets errs bytes colls drops 52839 0 0 78935238 0 0 0 0 0 53098 0 0 79307900 0 0 0 0 0 49559 0 0 74231830 0 0 0 0 0 79905 0 0 120972890 0 0 0 0 0 66836 0 0 101174008 0 0 0 0 0 75308 0 0 113998016 0 0 0 0 0 47258 0 0 71543932 0 0 0 0 0 32 0 0 48448 0 0 0 0 0 70026 0 0 106002764 0 0 0 0 0 63937 0 0 96604786 0 0 0 0 0 60947 0 0 92062662 0 0 0 0 0 63772 0 0 96410208 0 0 0 0 0 58129 0 0 87582154 0 0 0 0 0 59204 0 0 89190072 0 0 0 0 0 61375 0 0 92741230 0 0 0 0 0 69159 0 0 104622734 0 0 0 0 0 75748 0 0 114661800 0 0 0 0 0 75478 0 0 114253980 0 0 0 0 0 76814 0 0 116282228 0 0 0 0 0 76931 0 0 116458670 0 0 0 0 0 75299 0 0 113979718 0 0 0 0 0