From owner-freebsd-net@freebsd.org Tue Jan 30 12:25:42 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FC95EC7178 for ; Tue, 30 Jan 2018 12:25:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D26C06A1B7 for ; Tue, 30 Jan 2018 12:25:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 1A0EC1C811 for ; Tue, 30 Jan 2018 12:25:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w0UCPebm077108 for ; Tue, 30 Jan 2018 12:25:40 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w0UCPeQ3077104 for freebsd-net@FreeBSD.org; Tue, 30 Jan 2018 12:25:40 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 225535] Delays in TCP connection over Gigabit Ethernet connections; Regression from 6.3-RELEASE Date: Tue, 30 Jan 2018 12:25:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: aeder@list.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jan 2018 12:25:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D225535 --- Comment #5 from Aleksander Derevianko --- Additional info: I have tryed the following configuration: Xen (CentOS 7 in domain 0) running over Moxa hardware, under Xen - FreeBSD 10.3-RELEASE i386. xn0 configured as bridge. Between two copies of FreeBSD-10.3 running under = Xen on different host computers - this test application produce much better res= ult: grep times fspa2_fspa1_virt.txt | awk '{print $3 " " $4 " " $6 " " $8 " " $10;}' | sort | uniq -c 9863 send_sync 0 0 0 0 38565 send_sync 0 0 0 1 2 send_sync 0 0 0 10 9 send_sync 0 0 0 11 16 send_sync 0 0 0 12 10 send_sync 0 0 0 13 1 send_sync 0 0 0 17 301599 send_sync 0 0 0 2 1 send_sync 0 0 0 233 698526 send_sync 0 0 0 3 5195 send_sync 0 0 0 4 2770 send_sync 0 0 0 5 1361 send_sync 0 0 0 6 14 send_sync 0 0 0 7 5 send_sync 0 0 0 8 4 send_sync 0 0 0 9 1 send_sync 0 0 1 2 1 send_sync 0 0 2 0 4 send_sync 0 0 2 1 51 send_sync 0 0 2 2 79 send_sync 0 0 2 3 6545 send_sync 0 1 0 0 21849 send_sync 0 1 0 1 1 send_sync 0 1 0 10 1 send_sync 0 1 0 11 1 send_sync 0 1 0 12 39671 send_sync 0 1 0 2 244 send_sync 0 1 0 3 23 send_sync 0 1 0 4 8 send_sync 0 1 0 5 1 send_sync 0 1 0 6 1 send_sync 0 1 2 0 1 send_sync 0 14 0 2 17499 send_sync 0 2 0 0 58073 send_sync 0 2 0 1 1 send_sync 0 2 0 10 3 send_sync 0 2 0 11 2 send_sync 0 2 0 12 115462 send_sync 0 2 0 2 572 send_sync 0 2 0 3 7 send_sync 0 2 0 4 4 send_sync 0 2 0 5 1 send_sync 0 2 0 7 2 send_sync 0 2 0 9 7959 send_sync 0 3 0 0 20669 send_sync 0 3 0 1 1 send_sync 0 3 0 11 1 send_sync 0 3 0 12 49220 send_sync 0 3 0 2 267 send_sync 0 3 0 3 3 send_sync 0 3 0 4 2 send_sync 0 3 0 5 1 send_sync 0 3 1 0 23 send_sync 0 4 0 0 35 send_sync 0 4 0 1 54 send_sync 0 4 0 2 2 send_sync 0 4 0 3 1 send_sync 0 49 0 1 1 send_sync 0 5 0 1 1 send_sync 0 5 0 2 1 send_sync 0 6 0 2 1 send_sync 0 7 0 0 1 send_sync 0 7 0 3 1 send_sync 2 0 0 0 2 send_sync 2 0 0 1 16 send_sync 2 0 0 2 41 send_sync 2 0 0 3 1 send_sync 2 1 0 1 1 send_sync 2 1 0 2 >From 1.396.680 samples only 52 >=3D 10ms, and only 2 >=3D 20 ms.=20 >From my POV, it means that problem is somewhere in OS<->hardware layer, not= in OS itself. (In reply to amvandemore from comment #2) No, I have not tried multiqueue - first, it give advantage only under high load, and 40 Kbytes * 3 =3D 120 Kbytes/sec over 1Gibit line is very low loa= d. Seconds, Under 6.3-RELEASE it works fine without multiqueue. (In reply to Eugene Grosbein from comment #3) dmesg will be added as attachment. root@fspa1:~/clock/new_res # sysctl kern.timecounter kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 1 kern.timecounter.invariant_tsc: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) HPET(950) dummy(-1000000) kern.timecounter.hardware: TSC-low kern.timecounter.alloweddeviation: 0 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.TSC-low.quality: 1000 kern.timecounter.tc.TSC-low.frequency: 1247191596 kern.timecounter.tc.TSC-low.counter: 233727093 kern.timecounter.tc.TSC-low.mask: 4294967295 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.counter: 10612645 kern.timecounter.tc.ACPI-fast.mask: 16777215 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 14761 kern.timecounter.tc.i8254.mask: 65535 kern.timecounter.tc.HPET.quality: 950 kern.timecounter.tc.HPET.frequency: 14318180 kern.timecounter.tc.HPET.counter: 3051462975 kern.timecounter.tc.HPET.mask: 4294967295 root@fspa1:~/clock/new_res # sysctl kern.eventtimer kern.eventtimer.periodic: 0 kern.eventtimer.timer: LAPIC kern.eventtimer.idletick: 0 kern.eventtimer.singlemul: 2 kern.eventtimer.choice: LAPIC(600) HPET(550) HPET1(440) HPET2(440) HPET3(44= 0) HP ET4(440) i8254(100) RTC(0) kern.eventtimer.et.i8254.quality: 100 kern.eventtimer.et.i8254.frequency: 1193182 kern.eventtimer.et.i8254.flags: 1 kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.flags: 17 kern.eventtimer.et.HPET4.quality: 440 kern.eventtimer.et.HPET4.frequency: 14318180 kern.eventtimer.et.HPET4.flags: 3 kern.eventtimer.et.HPET3.quality: 440 kern.eventtimer.et.HPET3.frequency: 14318180 kern.eventtimer.et.HPET3.flags: 3 kern.eventtimer.et.HPET2.quality: 440 kern.eventtimer.et.HPET2.frequency: 14318180 kern.eventtimer.et.HPET2.flags: 3 kern.eventtimer.et.HPET1.quality: 440 kern.eventtimer.et.HPET1.frequency: 14318180 kern.eventtimer.et.HPET1.flags: 3 kern.eventtimer.et.HPET.quality: 550 kern.eventtimer.et.HPET.frequency: 14318180 kern.eventtimer.et.HPET.flags: 7 kern.eventtimer.et.LAPIC.quality: 600 kern.eventtimer.et.LAPIC.frequency: 49887687 kern.eventtimer.et.LAPIC.flags: 7 root@fspa1:~/clock/new_res # sysctl kern.hz kern.hz: 1000 root@fspa1:~/clock/new_res # sysctl kern.eventtimer.periodic kern.eventtimer.periodic: 0 I will try and report results here. > And why do use version 10.3 that has "Expected EoL" in April 30, 2018? An= d=20 > 10.4 in October 31, 2018, as whole line of 10.x versions. > Have you considered trying 11.1-RELEASE? It has many more changes to get= =20 > fixes, if found necessary. We have very long verification/release cycle because this software will be = used in critical infrastructure (railway automation). Latest security fixes is n= ot actually important, because it will be used in very closed environment.=20 I will try to update to 10.4 and 11.1 and check if it helps. (In reply to Conrad Meyer from comment #4) > What syscalls are you measuring delay over? It seems that the column wit= h=20 > delay is different between your two sets of samples -- does this mean=20 > anything? Thanks.=20 clock_gettime( CLOCK_MONOTONIC, ...); See attached source code. Actually, delay under 6.3-RELEASE is lower ( and evaluate time is higher ) because 6.3-RELEASE is running on different hardware. Test load (Fibonacci calculation) is executed faster under modern hardware. I will try to install 6.3-RELEASE on Moxa hardware, repeat test and report result. Unfortunelly, I need to have test running for at least 6 hours befo= re I get reliable results. --=20 You are receiving this mail because: You are the assignee for the bug.=