From owner-freebsd-net@freebsd.org Sun Jan 31 17:55:31 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C9A9A6F359 for ; Sun, 31 Jan 2016 17:55:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 886B511B2 for ; Sun, 31 Jan 2016 17:55:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0VHtVZJ080796 for ; Sun, 31 Jan 2016 17:55:31 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 205169] 10.2-RELEASE panic on boot if autobridge with wlan0 (iwn) Date: Sun, 31 Jan 2016 17:55:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: crash, needs-patch, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avos@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: DUPLICATE X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable9? mfc-stable10? X-Bugzilla-Changed-Fields: resolution cc bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2016 17:55:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205169 Andriy Voskoboinyk changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |DUPLICATE CC| |avos@freebsd.org Status|New |Closed --- Comment #1 from Andriy Voskoboinyk --- This is (relatively) recent m_unshare() bug (fixed in r288990); also report= ed in bug 195074 *** This bug has been marked as a duplicate of bug 195074 *** --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Sun Jan 31 21:00:14 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0BC7A7320A for ; Sun, 31 Jan 2016 21:00:14 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E82FE1E70 for ; Sun, 31 Jan 2016 21:00:14 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u0VL01OJ085275 for ; Sun, 31 Jan 2016 21:00:14 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201601312100.u0VL01OJ085275@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-net@FreeBSD.org Subject: Problem reports for freebsd-net@FreeBSD.org that need special attention Date: Sun, 31 Jan 2016 21:00:14 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Jan 2016 21:00:15 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 203422 | mpd/ppoe not working with re(4) with revision 285 New | 203175 | Daily kernel crashes in tcp_twclose
Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 947E0A7565E for ; Mon, 1 Feb 2016 09:20:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8653415E4 for ; Mon, 1 Feb 2016 09:20:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u119KFRV019108 for ; Mon, 1 Feb 2016 09:20:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 205834] rtadvd: accessing freed struct Date: Mon, 01 Feb 2016 09:20:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 09:20:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205834 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Feb 1 09:35:56 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2C01A75DBE for ; Mon, 1 Feb 2016 09:35:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A50D11DB2 for ; Mon, 1 Feb 2016 09:35:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u119Zucr053923 for ; Mon, 1 Feb 2016 09:35:56 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 205834] rtadvd: accessing freed struct Date: Mon, 01 Feb 2016 09:35:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hrs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: hrs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 09:35:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205834 Hiroki Sato changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress Assignee|freebsd-net@FreeBSD.org |hrs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Feb 1 08:38:05 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 085D5A974BD for ; Mon, 1 Feb 2016 08:38:05 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id B62387BD for ; Mon, 1 Feb 2016 08:38:04 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id B1D971068A3; Mon, 1 Feb 2016 08:38:04 +0000 (UTC) Date: Mon, 1 Feb 2016 08:38:04 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5158+325+84c30785c1b48e01@reviews.freebsd.org Subject: [Differential] [Request, 434 lines] D5158: hyperv/hn: Factor out hn_encap from hn_start_locked() Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D5158: hyperv/hn: Factor out hn_encap from hn_start_locked() X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk Thread-Index: MjFmOWM1NjYwZmMxOTMyMjJlM2E1YTQ1Mjll MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_8a4cb9012204fb53e8e197a24337dd20" X-Mailman-Approved-At: Mon, 01 Feb 2016 12:10:50 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 08:38:05 -0000 --b1_8a4cb9012204fb53e8e197a24337dd20 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com created this revision. sepherosa_gmail.com added reviewers: network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com. sepherosa_gmail.com added a subscriber: freebsd-net-list. REVISION SUMMARY It will be shared w/ upcoming ifnet.if_transmit implementaion. No functional changes. REVISION DETAIL https://reviews.freebsd.org/D5158 AFFECTED FILES sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com Cc: freebsd-net-list --b1_8a4cb9012204fb53e8e197a24337dd20 Content-Type: text/x-patch; charset=utf-8; name="D5158.12925.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5158.12925.patch" ZGlmZiAtLWdpdCBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2c2NfZHJ2X2ZyZWVic2Qu YyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2c2NfZHJ2X2ZyZWVic2QuYwotLS0gYS9z eXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMKKysrIGIvc3lzL2Rl di9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJlZWJzZC5jCkBAIC0yNTQsNiArMjU0LDcg QEAKIHN0YXRpYyB2b2lkIGhuX2Rlc3Ryb3lfdHhfcmluZyhzdHJ1Y3QgaG5fc29mdGMgKnNjKTsK IHN0YXRpYyB2b2lkIGhuX3N0YXJ0X3Rhc2tmdW5jKHZvaWQgKnhzYywgaW50IHBlbmRpbmcpOwog c3RhdGljIHZvaWQgaG5fdHhlb2ZfdGFza2Z1bmModm9pZCAqeHNjLCBpbnQgcGVuZGluZyk7Citz dGF0aWMgaW50IGhuX2VuY2FwKHN0cnVjdCBobl9zb2Z0YyAqLCBzdHJ1Y3QgaG5fdHhkZXNjICos IHN0cnVjdCBtYnVmICoqKTsKIAogc3RhdGljIF9faW5saW5lIHZvaWQKIGhuX3NldF9scm9faGl3 YXQoc3RydWN0IGhuX3NvZnRjICpzYywgaW50IGhpd2F0KQpAQCAtNzQ0LDI2NSArNzQ1LDI3MCBA QAogfQogCiAvKgotICogU3RhcnQgYSB0cmFuc21pdCBvZiBvbmUgb3IgbW9yZSBwYWNrZXRzCisg KiBOT1RFOgorICogVGhpcyB0aGlzIGZ1bmN0aW9uIGZhaWxzLCB0aGVuIGJvdGggdHhkIGFuZCBt X2hlYWQwIHdpbGwgYmUgZnJlZWQKICAqLwogc3RhdGljIGludAotaG5fc3RhcnRfbG9ja2VkKHN0 cnVjdCBpZm5ldCAqaWZwLCBpbnQgbGVuKQoraG5fZW5jYXAoc3RydWN0IGhuX3NvZnRjICpzYywg c3RydWN0IGhuX3R4ZGVzYyAqdHhkLCBzdHJ1Y3QgbWJ1ZiAqKm1faGVhZDApCiB7Ci0JaG5fc29m dGNfdCAqc2MgPSBpZnAtPmlmX3NvZnRjOwotCXN0cnVjdCBodl9kZXZpY2UgKmRldmljZV9jdHgg PSB2bWJ1c19nZXRfZGV2Y3R4KHNjLT5obl9kZXYpOwotCW5ldHZzY19kZXYgKm5ldF9kZXYgPSBz Yy0+bmV0X2RldjsKKwlidXNfZG1hX3NlZ21lbnRfdCBzZWdzW0hOX1RYX0RBVEFfU0VHQ05UX01B WF07CisJaW50IGVycm9yLCBuc2VncywgaTsKKwlzdHJ1Y3QgbWJ1ZiAqbV9oZWFkID0gKm1faGVh ZDA7CisJbmV0dnNjX3BhY2tldCAqcGFja2V0OwogCXJuZGlzX21zZyAqcm5kaXNfbWVzZzsKIAly bmRpc19wYWNrZXQgKnJuZGlzX3BrdDsKIAlybmRpc19wZXJfcGFja2V0X2luZm8gKnJwcGk7Ci0J bmRpc184MDIxcV9pbmZvICpycHBpX3ZsYW5faW5mbzsKLQlybmRpc190Y3BfaXBfY3N1bV9pbmZv ICpjc3VtX2luZm87Ci0Jcm5kaXNfdGNwX3Rzb19pbmZvICp0c29faW5mbzsJCi0JdWludDMyX3Qg cm5kaXNfbXNnX3NpemUgPSAwOworCXVpbnQzMl90IHJuZGlzX21zZ19zaXplOwogCi0JaWYgKChp ZnAtPmlmX2Rydl9mbGFncyAmIChJRkZfRFJWX1JVTk5JTkcgfCBJRkZfRFJWX09BQ1RJVkUpKSAh PQotCSAgICBJRkZfRFJWX1JVTk5JTkcpCi0JCXJldHVybiAwOworCXBhY2tldCA9ICZ0eGQtPm5l dHZzY19wa3Q7CisJcGFja2V0LT5pc19kYXRhX3BrdCA9IFRSVUU7CisJcGFja2V0LT50b3RfZGF0 YV9idWZfbGVuID0gbV9oZWFkLT5tX3BrdGhkci5sZW47CiAKLQl3aGlsZSAoIUlGUV9EUlZfSVNf RU1QVFkoJmlmcC0+aWZfc25kKSkgewotCQlidXNfZG1hX3NlZ21lbnRfdCBzZWdzW0hOX1RYX0RB VEFfU0VHQ05UX01BWF07Ci0JCWludCBlcnJvciwgbnNlZ3MsIGksIHNlbmRfZmFpbGVkID0gMDsK LQkJc3RydWN0IGhuX3R4ZGVzYyAqdHhkOwotCQluZXR2c2NfcGFja2V0ICpwYWNrZXQ7Ci0JCXN0 cnVjdCBtYnVmICptX2hlYWQ7Ci0KLQkJSUZRX0RSVl9ERVFVRVVFKCZpZnAtPmlmX3NuZCwgbV9o ZWFkKTsKLQkJaWYgKG1faGVhZCA9PSBOVUxMKQotCQkJYnJlYWs7CisJLyoKKwkgKiBleHRlbnNp b24gcG9pbnRzIHRvIHRoZSBhcmVhIHJlc2VydmVkIGZvciB0aGUKKwkgKiBybmRpc19maWx0ZXJf cGFja2V0LCB3aGljaCBpcyBwbGFjZWQganVzdCBhZnRlcgorCSAqIHRoZSBuZXR2c2NfcGFja2V0 IChhbmQgcnBwaSBzdHJ1Y3QsIGlmIHByZXNlbnQ7CisJICogbGVuZ3RoIGlzIHVwZGF0ZWQgbGF0 ZXIpLgorCSAqLworCXJuZGlzX21lc2cgPSB0eGQtPnJuZGlzX21zZzsKKwkvKiBYWFggbm90IG5l Y2Vzc2FyeSAqLworCW1lbXNldChybmRpc19tZXNnLCAwLCBITl9STkRJU19NU0dfTEVOKTsKKwly bmRpc19tZXNnLT5uZGlzX21zZ190eXBlID0gUkVNT1RFX05ESVNfUEFDS0VUX01TRzsKIAotCQlp ZiAobGVuID4gMCAmJiBtX2hlYWQtPm1fcGt0aGRyLmxlbiA+IGxlbikgewotCQkJLyoKLQkJCSAq IFRoaXMgc2VuZGluZyBjb3VsZCBiZSB0aW1lIGNvbnN1bWluZzsgbGV0IGNhbGxlcnMKLQkJCSAq IGRpc3BhdGNoIHRoaXMgcGFja2V0IHNlbmRpbmcgKGFuZCBzZW5kaW5nIG9mIGFueQotCQkJICog Zm9sbG93aW5nIHVwIHBhY2tldHMpIHRvIHR4IHRhc2txdWV1ZS4KLQkJCSAqLwotCQkJSUZfUFJF UEVORCgmaWZwLT5pZl9zbmQsIG1faGVhZCk7Ci0JCQlyZXR1cm4gMTsKLQkJfQorCXJuZGlzX3Br dCA9ICZybmRpc19tZXNnLT5tc2cucGFja2V0OworCXJuZGlzX3BrdC0+ZGF0YV9vZmZzZXQgPSBz aXplb2Yocm5kaXNfcGFja2V0KTsKKwlybmRpc19wa3QtPmRhdGFfbGVuZ3RoID0gcGFja2V0LT50 b3RfZGF0YV9idWZfbGVuOworCXJuZGlzX3BrdC0+cGVyX3BrdF9pbmZvX29mZnNldCA9IHNpemVv ZihybmRpc19wYWNrZXQpOwogCi0JCXR4ZCA9IGhuX3R4ZGVzY19nZXQoc2MpOwotCQlpZiAodHhk ID09IE5VTEwpIHsKLQkJCXNjLT5obl9ub190eGRlc2NzKys7Ci0JCQlJRl9QUkVQRU5EKCZpZnAt PmlmX3NuZCwgbV9oZWFkKTsKLQkJCWF0b21pY19zZXRfaW50KCZpZnAtPmlmX2Rydl9mbGFncywg SUZGX0RSVl9PQUNUSVZFKTsKLQkJCWJyZWFrOwotCQl9CisJcm5kaXNfbXNnX3NpemUgPSBSTkRJ U19NRVNTQUdFX1NJWkUocm5kaXNfcGFja2V0KTsKIAotCQlwYWNrZXQgPSAmdHhkLT5uZXR2c2Nf cGt0OwotCQlwYWNrZXQtPmlzX2RhdGFfcGt0ID0gVFJVRTsKLQkJLyogSW5pdGlhbGl6ZSBpdCBm cm9tIHRoZSBtYnVmICovCi0JCXBhY2tldC0+dG90X2RhdGFfYnVmX2xlbiA9IG1faGVhZC0+bV9w a3RoZHIubGVuOworCWlmIChtX2hlYWQtPm1fZmxhZ3MgJiBNX1ZMQU5UQUcpIHsKKwkJbmRpc184 MDIxcV9pbmZvICpycHBpX3ZsYW5faW5mbzsKIAotCQkvKgotCQkgKiBleHRlbnNpb24gcG9pbnRz IHRvIHRoZSBhcmVhIHJlc2VydmVkIGZvciB0aGUKLQkJICogcm5kaXNfZmlsdGVyX3BhY2tldCwg d2hpY2ggaXMgcGxhY2VkIGp1c3QgYWZ0ZXIKLQkJICogdGhlIG5ldHZzY19wYWNrZXQgKGFuZCBy cHBpIHN0cnVjdCwgaWYgcHJlc2VudDsKLQkJICogbGVuZ3RoIGlzIHVwZGF0ZWQgbGF0ZXIpLgot CQkgKi8KLQkJcm5kaXNfbWVzZyA9IHR4ZC0+cm5kaXNfbXNnOwotCQkvKiBYWFggbm90IG5lY2Vz c2FyeSAqLwotCQltZW1zZXQocm5kaXNfbWVzZywgMCwgSE5fUk5ESVNfTVNHX0xFTik7Ci0JCXJu ZGlzX21lc2ctPm5kaXNfbXNnX3R5cGUgPSBSRU1PVEVfTkRJU19QQUNLRVRfTVNHOworCQlybmRp c19tc2dfc2l6ZSArPSBSTkRJU19WTEFOX1BQSV9TSVpFOworCQlycHBpID0gaHZfc2V0X3JwcGlf ZGF0YShybmRpc19tZXNnLCBSTkRJU19WTEFOX1BQSV9TSVpFLAorCQkgICAgaWVlZV84MDIxcV9p bmZvKTsKIAotCQlybmRpc19wa3QgPSAmcm5kaXNfbWVzZy0+bXNnLnBhY2tldDsKLQkJcm5kaXNf cGt0LT5kYXRhX29mZnNldCA9IHNpemVvZihybmRpc19wYWNrZXQpOwotCQlybmRpc19wa3QtPmRh dGFfbGVuZ3RoID0gcGFja2V0LT50b3RfZGF0YV9idWZfbGVuOwotCQlybmRpc19wa3QtPnBlcl9w a3RfaW5mb19vZmZzZXQgPSBzaXplb2Yocm5kaXNfcGFja2V0KTsKKwkJcnBwaV92bGFuX2luZm8g PSAobmRpc184MDIxcV9pbmZvICopKCh1aW50OF90ICopcnBwaSArCisJCSAgICBycHBpLT5wZXJf cGFja2V0X2luZm9fb2Zmc2V0KTsKKwkJcnBwaV92bGFuX2luZm8tPnUxLnMxLnZsYW5faWQgPQor CQkgICAgbV9oZWFkLT5tX3BrdGhkci5ldGhlcl92dGFnICYgMHhmZmY7CisJfQogCi0JCXJuZGlz X21zZ19zaXplID0gUk5ESVNfTUVTU0FHRV9TSVpFKHJuZGlzX3BhY2tldCk7CisJaWYgKG1faGVh ZC0+bV9wa3RoZHIuY3N1bV9mbGFncyAmIENTVU1fVFNPKSB7CisJCXJuZGlzX3RjcF90c29faW5m byAqdHNvX2luZm87CQorCQlzdHJ1Y3QgZXRoZXJfdmxhbl9oZWFkZXIgKmVoOworCQlpbnQgZXRo ZXJfbGVuOwogCiAJCS8qCi0JCSAqIElmIHRoZSBIeXBlci1WIGluZnJhc3RydWN0dXJlIG5lZWRz IHRvIGVtYmVkIGEgVkxBTiB0YWcsCi0JCSAqIGluaXRpYWxpemUgbmV0dnNjX3BhY2tldCBhbmQg cnBwaSBzdHJ1Y3QgdmFsdWVzIGFzIG5lZWRlZC4KKwkJICogWFhYIG5lZWQgbV9wdWxsdXAgYW5k IHVzZSBtdG9kbwogCQkgKi8KLQkJaWYgKG1faGVhZC0+bV9mbGFncyAmIE1fVkxBTlRBRykgewot CQkJLyoKLQkJCSAqIHNldCB1cCBzb21lIGFkZGl0aW9uYWwgZmllbGRzIHNvIHRoZSBIeXBlci1W IGluZnJhc3RydWN0dXJlIHdpbGwgc3R1ZmYgdGhlIFZMQU4gdGFnCi0JCQkgKiBpbnRvIHRoZSBm cmFtZS4KLQkJCSAqLwotCQkJcm5kaXNfbXNnX3NpemUgKz0gUk5ESVNfVkxBTl9QUElfU0laRTsK LQotCQkJcnBwaSA9IGh2X3NldF9ycHBpX2RhdGEocm5kaXNfbWVzZywgUk5ESVNfVkxBTl9QUElf U0laRSwKLQkJCSAgICBpZWVlXzgwMjFxX2luZm8pOwotCQkKLQkJCS8qIFZMQU4gaW5mbyBpbW1l ZGlhdGVseSBmb2xsb3dzIHJwcGkgc3RydWN0ICovCi0JCQlycHBpX3ZsYW5faW5mbyA9IChuZGlz XzgwMjFxX2luZm8gKikoKGNoYXIqKXJwcGkgKyAKLQkJCSAgICBycHBpLT5wZXJfcGFja2V0X2lu Zm9fb2Zmc2V0KTsKLQkJCS8qIEZyZWVCU0QgZG9lcyBub3Qgc3VwcG9ydCBDRkkgb3IgcHJpb3Jp dHkgKi8KLQkJCXJwcGlfdmxhbl9pbmZvLT51MS5zMS52bGFuX2lkID0KLQkJCSAgICBtX2hlYWQt Pm1fcGt0aGRyLmV0aGVyX3Z0YWcgJiAweGZmZjsKLQkJfQotCi0JCWlmIChtX2hlYWQtPm1fcGt0 aGRyLmNzdW1fZmxhZ3MgJiBDU1VNX1RTTykgewotCQkJc3RydWN0IGV0aGVyX3ZsYW5faGVhZGVy ICplaDsKLQkJCWludCBldGhlcl9sZW47Ci0KLQkJCWVoID0gbXRvZChtX2hlYWQsIHN0cnVjdCBl dGhlcl92bGFuX2hlYWRlciopOwotCQkJaWYgKGVoLT5ldmxfZW5jYXBfcHJvdG8gPT0gaHRvbnMo RVRIRVJUWVBFX1ZMQU4pKSB7Ci0JCQkJZXRoZXJfbGVuID0gRVRIRVJfSERSX0xFTiArCi0JCQkJ ICAgIEVUSEVSX1ZMQU5fRU5DQVBfTEVOOwotCQkJfSBlbHNlIHsKLQkJCQlldGhlcl9sZW4gPSBF VEhFUl9IRFJfTEVOOwotCQkJfQorCQllaCA9IG10b2QobV9oZWFkLCBzdHJ1Y3QgZXRoZXJfdmxh bl9oZWFkZXIqKTsKKwkJaWYgKGVoLT5ldmxfZW5jYXBfcHJvdG8gPT0gaHRvbnMoRVRIRVJUWVBF X1ZMQU4pKQorCQkJZXRoZXJfbGVuID0gRVRIRVJfSERSX0xFTiArIEVUSEVSX1ZMQU5fRU5DQVBf TEVOOworCQllbHNlCisJCQlldGhlcl9sZW4gPSBFVEhFUl9IRFJfTEVOOwogCi0JCQlybmRpc19t c2dfc2l6ZSArPSBSTkRJU19UU09fUFBJX1NJWkU7Ci0JCQlycHBpID0gaHZfc2V0X3JwcGlfZGF0 YShybmRpc19tZXNnLCBSTkRJU19UU09fUFBJX1NJWkUsCi0JCQkgICAgdGNwX2xhcmdlX3NlbmRf aW5mbyk7CisJCXJuZGlzX21zZ19zaXplICs9IFJORElTX1RTT19QUElfU0laRTsKKwkJcnBwaSA9 IGh2X3NldF9ycHBpX2RhdGEocm5kaXNfbWVzZywgUk5ESVNfVFNPX1BQSV9TSVpFLAorCQkgICAg dGNwX2xhcmdlX3NlbmRfaW5mbyk7CiAKLQkJCXRzb19pbmZvID0gKHJuZGlzX3RjcF90c29faW5m byAqKSgoY2hhciAqKXJwcGkgKwotCQkJICAgIHJwcGktPnBlcl9wYWNrZXRfaW5mb19vZmZzZXQp OwotCQkJdHNvX2luZm8tPmxzb192Ml94bWl0LnR5cGUgPQotCQkJICAgIFJORElTX1RDUF9MQVJH RV9TRU5EX09GRkxPQURfVjJfVFlQRTsKKwkJdHNvX2luZm8gPSAocm5kaXNfdGNwX3Rzb19pbmZv ICopKCh1aW50OF90ICopcnBwaSArCisJCSAgICBycHBpLT5wZXJfcGFja2V0X2luZm9fb2Zmc2V0 KTsKKwkJdHNvX2luZm8tPmxzb192Ml94bWl0LnR5cGUgPQorCQkgICAgUk5ESVNfVENQX0xBUkdF X1NFTkRfT0ZGTE9BRF9WMl9UWVBFOwogCiAjaWZkZWYgSU5FVAotCQkJaWYgKG1faGVhZC0+bV9w a3RoZHIuY3N1bV9mbGFncyAmIENTVU1fSVBfVFNPKSB7Ci0JCQkJc3RydWN0IGlwICppcCA9Ci0J CQkJICAgIChzdHJ1Y3QgaXAgKikobV9oZWFkLT5tX2RhdGEgKyBldGhlcl9sZW4pOwotCQkJCXVu c2lnbmVkIGxvbmcgaXBoX2xlbiA9IGlwLT5pcF9obCA8PCAyOwotCQkJCXN0cnVjdCB0Y3BoZHIg KnRoID0KLQkJCQkgICAgKHN0cnVjdCB0Y3BoZHIgKikoKGNhZGRyX3QpaXAgKyBpcGhfbGVuKTsK LQkJCQotCQkJCXRzb19pbmZvLT5sc29fdjJfeG1pdC5pcF92ZXJzaW9uID0KLQkJCQkgICAgUk5E SVNfVENQX0xBUkdFX1NFTkRfT0ZGTE9BRF9JUFY0OwotCQkJCWlwLT5pcF9sZW4gPSAwOwotCQkJ CWlwLT5pcF9zdW0gPSAwOwotCQkJCi0JCQkJdGgtPnRoX3N1bSA9IGluX3BzZXVkbyhpcC0+aXBf c3JjLnNfYWRkciwKLQkJCQkgICAgaXAtPmlwX2RzdC5zX2FkZHIsIGh0b25zKElQUFJPVE9fVENQ KSk7Ci0JCQl9CisJCWlmIChtX2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBDU1VNX0lQX1RT TykgeworCQkJc3RydWN0IGlwICppcCA9CisJCQkgICAgKHN0cnVjdCBpcCAqKShtX2hlYWQtPm1f ZGF0YSArIGV0aGVyX2xlbik7CisJCQl1bnNpZ25lZCBsb25nIGlwaF9sZW4gPSBpcC0+aXBfaGwg PDwgMjsKKwkJCXN0cnVjdCB0Y3BoZHIgKnRoID0KKwkJCSAgICAoc3RydWN0IHRjcGhkciAqKSgo Y2FkZHJfdClpcCArIGlwaF9sZW4pOworCisJCQl0c29faW5mby0+bHNvX3YyX3htaXQuaXBfdmVy c2lvbiA9CisJCQkgICAgUk5ESVNfVENQX0xBUkdFX1NFTkRfT0ZGTE9BRF9JUFY0OworCQkJaXAt PmlwX2xlbiA9IDA7CisJCQlpcC0+aXBfc3VtID0gMDsKKworCQkJdGgtPnRoX3N1bSA9IGluX3Bz ZXVkbyhpcC0+aXBfc3JjLnNfYWRkciwKKwkJCSAgICBpcC0+aXBfZHN0LnNfYWRkciwgaHRvbnMo SVBQUk9UT19UQ1ApKTsKKwkJfQogI2VuZGlmCiAjaWYgZGVmaW5lZChJTkVUNikgJiYgZGVmaW5l ZChJTkVUKQotCQkJZWxzZQorCQllbHNlCiAjZW5kaWYKICNpZmRlZiBJTkVUNgotCQkJewotCQkJ CXN0cnVjdCBpcDZfaGRyICppcDYgPSAoc3RydWN0IGlwNl9oZHIgKikKLQkJCQkgICAgKG1faGVh ZC0+bV9kYXRhICsgZXRoZXJfbGVuKTsKLQkJCQlzdHJ1Y3QgdGNwaGRyICp0aCA9IChzdHJ1Y3Qg dGNwaGRyICopKGlwNiArIDEpOwotCi0JCQkJdHNvX2luZm8tPmxzb192Ml94bWl0LmlwX3ZlcnNp b24gPQotCQkJCSAgICBSTkRJU19UQ1BfTEFSR0VfU0VORF9PRkZMT0FEX0lQVjY7Ci0JCQkJaXA2 LT5pcDZfcGxlbiA9IDA7Ci0JCQkJdGgtPnRoX3N1bSA9Ci0JCQkJICAgIGluNl9ja3N1bV9wc2V1 ZG8oaXA2LCAwLCBJUFBST1RPX1RDUCwgMCk7Ci0JCQl9CisJCXsKKwkJCXN0cnVjdCBpcDZfaGRy ICppcDYgPSAoc3RydWN0IGlwNl9oZHIgKikKKwkJCSAgICAobV9oZWFkLT5tX2RhdGEgKyBldGhl cl9sZW4pOworCQkJc3RydWN0IHRjcGhkciAqdGggPSAoc3RydWN0IHRjcGhkciAqKShpcDYgKyAx KTsKKworCQkJdHNvX2luZm8tPmxzb192Ml94bWl0LmlwX3ZlcnNpb24gPQorCQkJICAgIFJORElT X1RDUF9MQVJHRV9TRU5EX09GRkxPQURfSVBWNjsKKwkJCWlwNi0+aXA2X3BsZW4gPSAwOworCQkJ dGgtPnRoX3N1bSA9IGluNl9ja3N1bV9wc2V1ZG8oaXA2LCAwLCBJUFBST1RPX1RDUCwgMCk7CisJ CX0KICNlbmRpZgotCQkJdHNvX2luZm8tPmxzb192Ml94bWl0LnRjcF9oZWFkZXJfb2Zmc2V0ID0g MDsKLQkJCXRzb19pbmZvLT5sc29fdjJfeG1pdC5tc3MgPSBtX2hlYWQtPm1fcGt0aGRyLnRzb19z ZWdzejsKLQkJfSBlbHNlIGlmIChtX2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBzYy0+aG5f Y3N1bV9hc3Npc3QpIHsKLQkJCXJuZGlzX21zZ19zaXplICs9IFJORElTX0NTVU1fUFBJX1NJWkU7 Ci0JCQlycHBpID0gaHZfc2V0X3JwcGlfZGF0YShybmRpc19tZXNnLCBSTkRJU19DU1VNX1BQSV9T SVpFLAotCQkJICAgIHRjcGlwX2Noa3N1bV9pbmZvKTsKLQkJCWNzdW1faW5mbyA9IChybmRpc190 Y3BfaXBfY3N1bV9pbmZvICopKChjaGFyKilycHBpICsKLQkJCSAgICBycHBpLT5wZXJfcGFja2V0 X2luZm9fb2Zmc2V0KTsKLQotCQkJY3N1bV9pbmZvLT54bWl0LmlzX2lwdjQgPSAxOwotCQkJaWYg KG1faGVhZC0+bV9wa3RoZHIuY3N1bV9mbGFncyAmIENTVU1fSVApCi0JCQkJY3N1bV9pbmZvLT54 bWl0LmlwX2hlYWRlcl9jc3VtID0gMTsKLQotCQkJaWYgKG1faGVhZC0+bV9wa3RoZHIuY3N1bV9m bGFncyAmIENTVU1fVENQKSB7Ci0JCQkJY3N1bV9pbmZvLT54bWl0LnRjcF9jc3VtID0gMTsKLQkJ CQljc3VtX2luZm8tPnhtaXQudGNwX2hlYWRlcl9vZmZzZXQgPSAwOwotCQkJfSBlbHNlIGlmICht X2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBDU1VNX1VEUCkgewotCQkJCWNzdW1faW5mby0+ eG1pdC51ZHBfY3N1bSA9IDE7Ci0JCQl9CisJCXRzb19pbmZvLT5sc29fdjJfeG1pdC50Y3BfaGVh ZGVyX29mZnNldCA9IDA7CisJCXRzb19pbmZvLT5sc29fdjJfeG1pdC5tc3MgPSBtX2hlYWQtPm1f cGt0aGRyLnRzb19zZWdzejsKKwl9IGVsc2UgaWYgKG1faGVhZC0+bV9wa3RoZHIuY3N1bV9mbGFn cyAmIHNjLT5obl9jc3VtX2Fzc2lzdCkgeworCQlybmRpc190Y3BfaXBfY3N1bV9pbmZvICpjc3Vt X2luZm87CisKKwkJcm5kaXNfbXNnX3NpemUgKz0gUk5ESVNfQ1NVTV9QUElfU0laRTsKKwkJcnBw aSA9IGh2X3NldF9ycHBpX2RhdGEocm5kaXNfbWVzZywgUk5ESVNfQ1NVTV9QUElfU0laRSwKKwkJ ICAgIHRjcGlwX2Noa3N1bV9pbmZvKTsKKwkJY3N1bV9pbmZvID0gKHJuZGlzX3RjcF9pcF9jc3Vt X2luZm8gKikoKHVpbnQ4X3QgKilycHBpICsKKwkJICAgIHJwcGktPnBlcl9wYWNrZXRfaW5mb19v ZmZzZXQpOworCisJCWNzdW1faW5mby0+eG1pdC5pc19pcHY0ID0gMTsKKwkJaWYgKG1faGVhZC0+ bV9wa3RoZHIuY3N1bV9mbGFncyAmIENTVU1fSVApCisJCQljc3VtX2luZm8tPnhtaXQuaXBfaGVh ZGVyX2NzdW0gPSAxOworCisJCWlmIChtX2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBDU1VN X1RDUCkgeworCQkJY3N1bV9pbmZvLT54bWl0LnRjcF9jc3VtID0gMTsKKwkJCWNzdW1faW5mby0+ eG1pdC50Y3BfaGVhZGVyX29mZnNldCA9IDA7CisJCX0gZWxzZSBpZiAobV9oZWFkLT5tX3BrdGhk ci5jc3VtX2ZsYWdzICYgQ1NVTV9VRFApIHsKKwkJCWNzdW1faW5mby0+eG1pdC51ZHBfY3N1bSA9 IDE7CiAJCX0KKwl9CiAKLQkJcm5kaXNfbWVzZy0+bXNnX2xlbiA9IHBhY2tldC0+dG90X2RhdGFf YnVmX2xlbiArIHJuZGlzX21zZ19zaXplOwotCQlwYWNrZXQtPnRvdF9kYXRhX2J1Zl9sZW4gPSBy bmRpc19tZXNnLT5tc2dfbGVuOwotCi0JCS8qIHNlbmQgcGFja2V0IHdpdGggc2VuZCBidWZmZXIg Ki8KLQkJaWYgKHBhY2tldC0+dG90X2RhdGFfYnVmX2xlbiA8IHNjLT5obl90eF9jaGltbmV5X3Np emUpIHsKLQkJCXVpbnQzMl90IHNlbmRfYnVmX3NlY3Rpb25faWR4OwotCi0JCQlzZW5kX2J1Zl9z ZWN0aW9uX2lkeCA9Ci0JCQkgICAgaHZfbnZfZ2V0X25leHRfc2VuZF9zZWN0aW9uKG5ldF9kZXYp OwotCQkJaWYgKHNlbmRfYnVmX3NlY3Rpb25faWR4ICE9Ci0JCQkgICAgTlZTUF8xX0NISU1ORVlf U0VORF9JTlZBTElEX1NFQ1RJT05fSU5ERVgpIHsKLQkJCQl1aW50OF90ICpkZXN0ID0gKCh1aW50 OF90ICopbmV0X2Rldi0+c2VuZF9idWYgKwotCQkJCSAgICAoc2VuZF9idWZfc2VjdGlvbl9pZHgg KgotCQkJCSAgICAgbmV0X2Rldi0+c2VuZF9zZWN0aW9uX3NpemUpKTsKLQotCQkJCW1lbWNweShk ZXN0LCBybmRpc19tZXNnLCBybmRpc19tc2dfc2l6ZSk7Ci0JCQkJZGVzdCArPSBybmRpc19tc2df c2l6ZTsKLQotCQkJCW1fY29weWRhdGEobV9oZWFkLCAwLCBtX2hlYWQtPm1fcGt0aGRyLmxlbiwK LQkJCQkgICAgZGVzdCk7Ci0KLQkJCQlwYWNrZXQtPnNlbmRfYnVmX3NlY3Rpb25faWR4ID0KLQkJ CQkgICAgc2VuZF9idWZfc2VjdGlvbl9pZHg7Ci0JCQkJcGFja2V0LT5zZW5kX2J1Zl9zZWN0aW9u X3NpemUgPQotCQkJCSAgICBwYWNrZXQtPnRvdF9kYXRhX2J1Zl9sZW47Ci0JCQkJcGFja2V0LT5w YWdlX2J1Zl9jb3VudCA9IDA7Ci0JCQkJc2MtPmhuX3R4X2NoaW1uZXkrKzsKLQkJCQlnb3RvIGRv X3NlbmQ7Ci0JCQl9CisJcm5kaXNfbWVzZy0+bXNnX2xlbiA9IHBhY2tldC0+dG90X2RhdGFfYnVm X2xlbiArIHJuZGlzX21zZ19zaXplOworCXBhY2tldC0+dG90X2RhdGFfYnVmX2xlbiA9IHJuZGlz X21lc2ctPm1zZ19sZW47CisKKwkvKgorCSAqIENoaW1uZXkgc2VuZCwgaWYgdGhlIHBhY2tldCBj b3VsZCBmaXQgaW50byBvbmUgY2hpbW5leSBidWZmZXIuCisJICovCisJaWYgKHBhY2tldC0+dG90 X2RhdGFfYnVmX2xlbiA8IHNjLT5obl90eF9jaGltbmV5X3NpemUpIHsKKwkJbmV0dnNjX2RldiAq bmV0X2RldiA9IHNjLT5uZXRfZGV2OworCQl1aW50MzJfdCBzZW5kX2J1Zl9zZWN0aW9uX2lkeDsK KworCQlzZW5kX2J1Zl9zZWN0aW9uX2lkeCA9CisJCSAgICBodl9udl9nZXRfbmV4dF9zZW5kX3Nl Y3Rpb24obmV0X2Rldik7CisJCWlmIChzZW5kX2J1Zl9zZWN0aW9uX2lkeCAhPQorCQkgICAgTlZT UF8xX0NISU1ORVlfU0VORF9JTlZBTElEX1NFQ1RJT05fSU5ERVgpIHsKKwkJCXVpbnQ4X3QgKmRl c3QgPSAoKHVpbnQ4X3QgKiluZXRfZGV2LT5zZW5kX2J1ZiArCisJCQkgICAgKHNlbmRfYnVmX3Nl Y3Rpb25faWR4ICoKKwkJCSAgICAgbmV0X2Rldi0+c2VuZF9zZWN0aW9uX3NpemUpKTsKKworCQkJ bWVtY3B5KGRlc3QsIHJuZGlzX21lc2csIHJuZGlzX21zZ19zaXplKTsKKwkJCWRlc3QgKz0gcm5k aXNfbXNnX3NpemU7CisJCQltX2NvcHlkYXRhKG1faGVhZCwgMCwgbV9oZWFkLT5tX3BrdGhkci5s ZW4sIGRlc3QpOworCisJCQlwYWNrZXQtPnNlbmRfYnVmX3NlY3Rpb25faWR4ID0gc2VuZF9idWZf c2VjdGlvbl9pZHg7CisJCQlwYWNrZXQtPnNlbmRfYnVmX3NlY3Rpb25fc2l6ZSA9CisJCQkgICAg cGFja2V0LT50b3RfZGF0YV9idWZfbGVuOworCQkJcGFja2V0LT5wYWdlX2J1Zl9jb3VudCA9IDA7 CisJCQlzYy0+aG5fdHhfY2hpbW5leSsrOworCQkJZ290byBkb25lOwogCQl9CisJfQogCi0JCWVy cm9yID0gaG5fdHhkZXNjX2RtYW1hcF9sb2FkKHNjLCB0eGQsICZtX2hlYWQsIHNlZ3MsICZuc2Vn cyk7Ci0JCWlmIChlcnJvcikgewotCQkJaW50IGZyZWVkOworCWVycm9yID0gaG5fdHhkZXNjX2Rt YW1hcF9sb2FkKHNjLCB0eGQsICZtX2hlYWQsIHNlZ3MsICZuc2Vncyk7CisJaWYgKGVycm9yKSB7 CisJCWludCBmcmVlZDsKIAotCQkJLyoKLQkJCSAqIFRoaXMgbWJ1ZiBpcyBub3QgbGlua2VkIHcv IHRoZSB0eGQgeWV0LCBzbyBmcmVlCi0JCQkgKiBpdCBub3cuCi0JCQkgKi8KLQkJCW1fZnJlZW0o bV9oZWFkKTsKLQkJCWZyZWVkID0gaG5fdHhkZXNjX3B1dChzYywgdHhkKTsKLQkJCUtBU1NFUlQo ZnJlZWQgIT0gMCwKLQkJCSAgICAoImZhaWwgdG8gZnJlZSB0eGQgdXBvbiB0eGRtYSBlcnJvciIp KTsKKwkJLyoKKwkJICogVGhpcyBtYnVmIGlzIG5vdCBsaW5rZWQgdy8gdGhlIHR4ZCB5ZXQsIHNv IGZyZWUgaXQgbm93LgorCQkgKi8KKwkJbV9mcmVlbShtX2hlYWQpOworCQkqbV9oZWFkMCA9IE5V TEw7CiAKLQkJCXNjLT5obl90eGRtYV9mYWlsZWQrKzsKLQkJCWlmX2luY19jb3VudGVyKGlmcCwg SUZDT1VOVEVSX09FUlJPUlMsIDEpOwotCQkJY29udGludWU7Ci0JCX0KKwkJZnJlZWQgPSBobl90 eGRlc2NfcHV0KHNjLCB0eGQpOworCQlLQVNTRVJUKGZyZWVkICE9IDAsCisJCSAgICAoImZhaWwg dG8gZnJlZSB0eGQgdXBvbiB0eGRtYSBlcnJvciIpKTsKIAotCQlwYWNrZXQtPnBhZ2VfYnVmX2Nv dW50ID0gbnNlZ3MgKwotCQkgICAgSFZfUkZfTlVNX1RYX1JFU0VSVkVEX1BBR0VfQlVGUzsKKwkJ c2MtPmhuX3R4ZG1hX2ZhaWxlZCsrOworCQlpZl9pbmNfY291bnRlcihzYy0+aG5faWZwLCBJRkNP VU5URVJfT0VSUk9SUywgMSk7CisJCXJldHVybiBlcnJvcjsKKwl9CisJKm1faGVhZDAgPSBtX2hl YWQ7CiAKLQkJLyogc2VuZCBwYWNrZXQgd2l0aCBwYWdlIGJ1ZmZlciAqLwotCQlwYWNrZXQtPnBh Z2VfYnVmZmVyc1swXS5wZm4gPSBhdG9wKHR4ZC0+cm5kaXNfbXNnX3BhZGRyKTsKLQkJcGFja2V0 LT5wYWdlX2J1ZmZlcnNbMF0ub2Zmc2V0ID0KLQkJICAgIHR4ZC0+cm5kaXNfbXNnX3BhZGRyICYg UEFHRV9NQVNLOwotCQlwYWNrZXQtPnBhZ2VfYnVmZmVyc1swXS5sZW5ndGggPSBybmRpc19tc2df c2l6ZTsKKwlwYWNrZXQtPnBhZ2VfYnVmX2NvdW50ID0gbnNlZ3MgKyBIVl9SRl9OVU1fVFhfUkVT RVJWRURfUEFHRV9CVUZTOwogCi0JCS8qCi0JCSAqIEZpbGwgdGhlIHBhZ2UgYnVmZmVycyB3aXRo IG1idWYgaW5mbyBzdGFydGluZyBhdCBpbmRleAotCQkgKiBIVl9SRl9OVU1fVFhfUkVTRVJWRURf UEFHRV9CVUZTLgotCQkgKi8KLQkJZm9yIChpID0gMDsgaSA8IG5zZWdzOyArK2kpIHsKLQkJCWh2 X3ZtYnVzX3BhZ2VfYnVmZmVyICpwYiA9ICZwYWNrZXQtPnBhZ2VfYnVmZmVyc1sKLQkJCSAgICBp ICsgSFZfUkZfTlVNX1RYX1JFU0VSVkVEX1BBR0VfQlVGU107CisJLyogc2VuZCBwYWNrZXQgd2l0 aCBwYWdlIGJ1ZmZlciAqLworCXBhY2tldC0+cGFnZV9idWZmZXJzWzBdLnBmbiA9IGF0b3AodHhk LT5ybmRpc19tc2dfcGFkZHIpOworCXBhY2tldC0+cGFnZV9idWZmZXJzWzBdLm9mZnNldCA9IHR4 ZC0+cm5kaXNfbXNnX3BhZGRyICYgUEFHRV9NQVNLOworCXBhY2tldC0+cGFnZV9idWZmZXJzWzBd Lmxlbmd0aCA9IHJuZGlzX21zZ19zaXplOwogCi0JCQlwYi0+cGZuID0gYXRvcChzZWdzW2ldLmRz X2FkZHIpOwotCQkJcGItPm9mZnNldCA9IHNlZ3NbaV0uZHNfYWRkciAmIFBBR0VfTUFTSzsKLQkJ CXBiLT5sZW5ndGggPSBzZWdzW2ldLmRzX2xlbjsKLQkJfQorCS8qCisJICogRmlsbCB0aGUgcGFn ZSBidWZmZXJzIHdpdGggbWJ1ZiBpbmZvIHN0YXJ0aW5nIGF0IGluZGV4CisJICogSFZfUkZfTlVN X1RYX1JFU0VSVkVEX1BBR0VfQlVGUy4KKwkgKi8KKwlmb3IgKGkgPSAwOyBpIDwgbnNlZ3M7ICsr aSkgeworCQlodl92bWJ1c19wYWdlX2J1ZmZlciAqcGIgPSAmcGFja2V0LT5wYWdlX2J1ZmZlcnNb CisJCSAgICBpICsgSFZfUkZfTlVNX1RYX1JFU0VSVkVEX1BBR0VfQlVGU107CisKKwkJcGItPnBm biA9IGF0b3Aoc2Vnc1tpXS5kc19hZGRyKTsKKwkJcGItPm9mZnNldCA9IHNlZ3NbaV0uZHNfYWRk ciAmIFBBR0VfTUFTSzsKKwkJcGItPmxlbmd0aCA9IHNlZ3NbaV0uZHNfbGVuOworCX0KIAotCQlw YWNrZXQtPnNlbmRfYnVmX3NlY3Rpb25faWR4ID0gCi0JCSAgICBOVlNQXzFfQ0hJTU5FWV9TRU5E X0lOVkFMSURfU0VDVElPTl9JTkRFWDsKLQkJcGFja2V0LT5zZW5kX2J1Zl9zZWN0aW9uX3NpemUg PSAwOworCXBhY2tldC0+c2VuZF9idWZfc2VjdGlvbl9pZHggPQorCSAgICBOVlNQXzFfQ0hJTU5F WV9TRU5EX0lOVkFMSURfU0VDVElPTl9JTkRFWDsKKwlwYWNrZXQtPnNlbmRfYnVmX3NlY3Rpb25f c2l6ZSA9IDA7Citkb25lOgorCXR4ZC0+bSA9IG1faGVhZDsKIAotZG9fc2VuZDoKLQkJdHhkLT5t ID0gbV9oZWFkOworCS8qIFNldCB0aGUgY29tcGxldGlvbiByb3V0aW5lICovCisJcGFja2V0LT5j b21wbC5zZW5kLm9uX3NlbmRfY29tcGxldGlvbiA9IG5ldHZzY194bWl0X2NvbXBsZXRpb247CisJ cGFja2V0LT5jb21wbC5zZW5kLnNlbmRfY29tcGxldGlvbl9jb250ZXh0ID0gcGFja2V0OworCXBh Y2tldC0+Y29tcGwuc2VuZC5zZW5kX2NvbXBsZXRpb25fdGlkID0gKHVpbnQ2NF90KSh1aW50cHRy X3QpdHhkOwogCi0JCS8qIFNldCB0aGUgY29tcGxldGlvbiByb3V0aW5lICovCi0JCXBhY2tldC0+ Y29tcGwuc2VuZC5vbl9zZW5kX2NvbXBsZXRpb24gPSBuZXR2c2NfeG1pdF9jb21wbGV0aW9uOwot CQlwYWNrZXQtPmNvbXBsLnNlbmQuc2VuZF9jb21wbGV0aW9uX2NvbnRleHQgPSBwYWNrZXQ7Ci0J CXBhY2tldC0+Y29tcGwuc2VuZC5zZW5kX2NvbXBsZXRpb25fdGlkID0KLQkJICAgICh1aW50NjRf dCkodWludHB0cl90KXR4ZDsKKwlyZXR1cm4gMDsKK30KKworLyoKKyAqIFN0YXJ0IGEgdHJhbnNt aXQgb2Ygb25lIG9yIG1vcmUgcGFja2V0cworICovCitzdGF0aWMgaW50Citobl9zdGFydF9sb2Nr ZWQoc3RydWN0IGlmbmV0ICppZnAsIGludCBsZW4pCit7CisJc3RydWN0IGhuX3NvZnRjICpzYyA9 IGlmcC0+aWZfc29mdGM7CisJc3RydWN0IGh2X2RldmljZSAqZGV2aWNlX2N0eCA9IHZtYnVzX2dl dF9kZXZjdHgoc2MtPmhuX2Rldik7CisKKwlpZiAoKGlmcC0+aWZfZHJ2X2ZsYWdzICYgKElGRl9E UlZfUlVOTklORyB8IElGRl9EUlZfT0FDVElWRSkpICE9CisJICAgIElGRl9EUlZfUlVOTklORykK KwkJcmV0dXJuIDA7CiAKKwl3aGlsZSAoIUlGUV9EUlZfSVNfRU1QVFkoJmlmcC0+aWZfc25kKSkg eworCQlpbnQgZXJyb3IsIHNlbmRfZmFpbGVkID0gMDsKKwkJc3RydWN0IGhuX3R4ZGVzYyAqdHhk OworCQlzdHJ1Y3QgbWJ1ZiAqbV9oZWFkOworCisJCUlGUV9EUlZfREVRVUVVRSgmaWZwLT5pZl9z bmQsIG1faGVhZCk7CisJCWlmIChtX2hlYWQgPT0gTlVMTCkKKwkJCWJyZWFrOworCisJCWlmIChs ZW4gPiAwICYmIG1faGVhZC0+bV9wa3RoZHIubGVuID4gbGVuKSB7CisJCQkvKgorCQkJICogVGhp cyBzZW5kaW5nIGNvdWxkIGJlIHRpbWUgY29uc3VtaW5nOyBsZXQgY2FsbGVycworCQkJICogZGlz cGF0Y2ggdGhpcyBwYWNrZXQgc2VuZGluZyAoYW5kIHNlbmRpbmcgb2YgYW55CisJCQkgKiBmb2xs b3dpbmcgdXAgcGFja2V0cykgdG8gdHggdGFza3F1ZXVlLgorCQkJICovCisJCQlJRl9QUkVQRU5E KCZpZnAtPmlmX3NuZCwgbV9oZWFkKTsKKwkJCXJldHVybiAxOworCQl9CisKKwkJdHhkID0gaG5f dHhkZXNjX2dldChzYyk7CisJCWlmICh0eGQgPT0gTlVMTCkgeworCQkJc2MtPmhuX25vX3R4ZGVz Y3MrKzsKKwkJCUlGX1BSRVBFTkQoJmlmcC0+aWZfc25kLCBtX2hlYWQpOworCQkJYXRvbWljX3Nl dF9pbnQoJmlmcC0+aWZfZHJ2X2ZsYWdzLCBJRkZfRFJWX09BQ1RJVkUpOworCQkJYnJlYWs7CisJ CX0KKworCQllcnJvciA9IGhuX2VuY2FwKHNjLCB0eGQsICZtX2hlYWQpOworCQlpZiAoZXJyb3Ip IHsKKwkJCS8qIEJvdGggdHhkIGFuZCBtX2hlYWQgYXJlIGZyZWVkICovCisJCQljb250aW51ZTsK KwkJfQogYWdhaW46CiAJCS8qCiAJCSAqIE1ha2Ugc3VyZSB0aGF0IHR4ZCBpcyBub3QgZnJlZWQg YmVmb3JlIEVUSEVSX0JQRl9NVEFQLgogCQkgKi8KIAkJaG5fdHhkZXNjX2hvbGQodHhkKTsKLQkJ ZXJyb3IgPSBodl9udl9vbl9zZW5kKGRldmljZV9jdHgsIHBhY2tldCk7CisJCWVycm9yID0gaHZf bnZfb25fc2VuZChkZXZpY2VfY3R4LCAmdHhkLT5uZXR2c2NfcGt0KTsKIAkJaWYgKCFlcnJvcikg ewogCQkJRVRIRVJfQlBGX01UQVAoaWZwLCBtX2hlYWQpOwogCQkJaWZfaW5jX2NvdW50ZXIoaWZw LCBJRkNPVU5URVJfT1BBQ0tFVFMsIDEpOwoK --b1_8a4cb9012204fb53e8e197a24337dd20-- From owner-freebsd-net@freebsd.org Mon Feb 1 08:43:00 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88B2CA9789E for ; Mon, 1 Feb 2016 08:43:00 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 7611BA8D for ; Mon, 1 Feb 2016 08:43:00 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 72FC9106B55; Mon, 1 Feb 2016 08:43:00 +0000 (UTC) Date: Mon, 1 Feb 2016 08:43:00 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5159+325+0db17ec577dfe5ca@reviews.freebsd.org Subject: [Differential] [Request, 31 lines] D5159: hyperv/hn: Recover half of the chimney sending space Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D5159: hyperv/hn: Recover half of the chimney sending space X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk Thread-Index: MzRmNWEyNTk4ODRlNWNkNGYzNTA2Yjk4Nzc1 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_297ff6342ca4903e34619b9cb3aa961e" X-Mailman-Approved-At: Mon, 01 Feb 2016 12:11:03 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 08:43:00 -0000 --b1_297ff6342ca4903e34619b9cb3aa961e Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com created this revision. sepherosa_gmail.com added reviewers: network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com. sepherosa_gmail.com added a subscriber: freebsd-net-list. REVISION SUMMARY Due to the miss use of ffs, where ffsl should be used. And use system atomic operation instead. While I'm here, strigent chimney sending index assertion. REVISION DETAIL https://reviews.freebsd.org/D5159 AFFECTED FILES sys/dev/hyperv/netvsc/hv_net_vsc.c CHANGE DETAILS diff --git a/sys/dev/hyperv/netvsc/hv_net_vsc.c b/sys/dev/hyperv/netvsc/hv_net_vsc.c --- a/sys/dev/hyperv/netvsc/hv_net_vsc.c +++ b/sys/dev/hyperv/netvsc/hv_net_vsc.c @@ -136,15 +136,15 @@ int i; for (i = 0; i < bitsmap_words; i++) { - idx = ffs(~bitsmap[i]); + idx = ffsl(~bitsmap[i]); if (0 == idx) continue; idx--; - if (i * BITS_PER_LONG + idx >= net_dev->send_section_count) - return (ret); + KASSERT(i * BITS_PER_LONG + idx < net_dev->send_section_count, + ("invalid i %d and idx %lu", i, idx)); - if (synch_test_and_set_bit(idx, &bitsmap[i])) + if (atomic_testandset_long(&bitsmap[i], idx)) continue; ret = i * BITS_PER_LONG + idx; @@ -789,8 +789,27 @@ if (NULL != net_vsc_pkt) { if (net_vsc_pkt->send_buf_section_idx != NVSP_1_CHIMNEY_SEND_INVALID_SECTION_INDEX) { - synch_change_bit(net_vsc_pkt->send_buf_section_idx, - net_dev->send_section_bitsmap); + u_long mask; + int idx; + + idx = net_vsc_pkt->send_buf_section_idx / + BITS_PER_LONG; + KASSERT(idx < net_dev->bitsmap_words, + ("invalid section index %u", + net_vsc_pkt->send_buf_section_idx)); + mask = 1UL << + (net_vsc_pkt->send_buf_section_idx % + BITS_PER_LONG); + + KASSERT(net_dev->send_section_bitsmap[idx] & + mask, + ("index bitmap 0x%lx, section index %u, " + "bitmap idx %d, bitmask 0x%lx", + net_dev->send_section_bitsmap[idx], + net_vsc_pkt->send_buf_section_idx, + idx, mask)); + atomic_clear_long( + &net_dev->send_section_bitsmap[idx], mask); } /* Notify the layer above us */ EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com Cc: freebsd-net-list --b1_297ff6342ca4903e34619b9cb3aa961e Content-Type: text/x-patch; charset=utf-8; name="D5159.12926.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5159.12926.patch" ZGlmZiAtLWdpdCBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmMgYi9zeXMvZGV2 L2h5cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5jCi0tLSBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9o dl9uZXRfdnNjLmMKKysrIGIvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuYwpAQCAt MTM2LDE1ICsxMzYsMTUgQEAKIAlpbnQgaTsKIAogCWZvciAoaSA9IDA7IGkgPCBiaXRzbWFwX3dv cmRzOyBpKyspIHsKLQkJaWR4ID0gZmZzKH5iaXRzbWFwW2ldKTsKKwkJaWR4ID0gZmZzbCh+Yml0 c21hcFtpXSk7CiAJCWlmICgwID09IGlkeCkKIAkJCWNvbnRpbnVlOwogCiAJCWlkeC0tOwotCQlp ZiAoaSAqIEJJVFNfUEVSX0xPTkcgKyBpZHggPj0gbmV0X2Rldi0+c2VuZF9zZWN0aW9uX2NvdW50 KQotCQkJcmV0dXJuIChyZXQpOworCQlLQVNTRVJUKGkgKiBCSVRTX1BFUl9MT05HICsgaWR4IDwg bmV0X2Rldi0+c2VuZF9zZWN0aW9uX2NvdW50LAorCQkgICAgKCJpbnZhbGlkIGkgJWQgYW5kIGlk eCAlbHUiLCBpLCBpZHgpKTsKIAotCQlpZiAoc3luY2hfdGVzdF9hbmRfc2V0X2JpdChpZHgsICZi aXRzbWFwW2ldKSkKKwkJaWYgKGF0b21pY190ZXN0YW5kc2V0X2xvbmcoJmJpdHNtYXBbaV0sIGlk eCkpCiAJCQljb250aW51ZTsKIAogCQlyZXQgPSBpICogQklUU19QRVJfTE9ORyArIGlkeDsKQEAg LTc4OSw4ICs3ODksMjcgQEAKIAkJaWYgKE5VTEwgIT0gbmV0X3ZzY19wa3QpIHsKIAkJCWlmIChu ZXRfdnNjX3BrdC0+c2VuZF9idWZfc2VjdGlvbl9pZHggIT0KIAkJCSAgICBOVlNQXzFfQ0hJTU5F WV9TRU5EX0lOVkFMSURfU0VDVElPTl9JTkRFWCkgewotCQkJCXN5bmNoX2NoYW5nZV9iaXQobmV0 X3ZzY19wa3QtPnNlbmRfYnVmX3NlY3Rpb25faWR4LAotCQkJCSAgICBuZXRfZGV2LT5zZW5kX3Nl Y3Rpb25fYml0c21hcCk7CisJCQkJdV9sb25nIG1hc2s7CisJCQkJaW50IGlkeDsKKworCQkJCWlk eCA9IG5ldF92c2NfcGt0LT5zZW5kX2J1Zl9zZWN0aW9uX2lkeCAvCisJCQkJICAgIEJJVFNfUEVS X0xPTkc7CisJCQkJS0FTU0VSVChpZHggPCBuZXRfZGV2LT5iaXRzbWFwX3dvcmRzLAorCQkJCSAg ICAoImludmFsaWQgc2VjdGlvbiBpbmRleCAldSIsCisJCQkJICAgICBuZXRfdnNjX3BrdC0+c2Vu ZF9idWZfc2VjdGlvbl9pZHgpKTsKKwkJCQltYXNrID0gMVVMIDw8CisJCQkJICAgIChuZXRfdnNj X3BrdC0+c2VuZF9idWZfc2VjdGlvbl9pZHggJQorCQkJCSAgICAgQklUU19QRVJfTE9ORyk7CisK KwkJCQlLQVNTRVJUKG5ldF9kZXYtPnNlbmRfc2VjdGlvbl9iaXRzbWFwW2lkeF0gJgorCQkJCSAg ICBtYXNrLAorCQkJCSAgICAoImluZGV4IGJpdG1hcCAweCVseCwgc2VjdGlvbiBpbmRleCAldSwg IgorCQkJCSAgICAgImJpdG1hcCBpZHggJWQsIGJpdG1hc2sgMHglbHgiLAorCQkJCSAgICAgbmV0 X2Rldi0+c2VuZF9zZWN0aW9uX2JpdHNtYXBbaWR4XSwKKwkJCQkgICAgIG5ldF92c2NfcGt0LT5z ZW5kX2J1Zl9zZWN0aW9uX2lkeCwKKwkJCQkgICAgIGlkeCwgbWFzaykpOworCQkJCWF0b21pY19j bGVhcl9sb25nKAorCQkJCSAgICAmbmV0X2Rldi0+c2VuZF9zZWN0aW9uX2JpdHNtYXBbaWR4XSwg bWFzayk7CiAJCQl9CiAJCQkKIAkJCS8qIE5vdGlmeSB0aGUgbGF5ZXIgYWJvdmUgdXMgKi8KCg== --b1_297ff6342ca4903e34619b9cb3aa961e-- From owner-freebsd-net@freebsd.org Mon Feb 1 23:24:37 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB932A9774D for ; Mon, 1 Feb 2016 23:24:37 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: from mail-ig0-x22a.google.com (mail-ig0-x22a.google.com [IPv6:2607:f8b0:4001:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D6071DB0 for ; Mon, 1 Feb 2016 23:24:37 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: by mail-ig0-x22a.google.com with SMTP id mw1so45036009igb.1 for ; Mon, 01 Feb 2016 15:24:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=L+2iXhzmL5I4fY/OktCbptotLej9fyPEwdWoihFTbUA=; b=NrR0gmhQ3gKiJsbqWizync9L7S+DbxInHaP6pfVeXQFhOGHzQg9dKGG81BYlULd4iM ug9zQ0L3FzqWBdJm8/fiVh2F0jfJSTcAM0c6DntxKWWjZsX9QonBUHqkDzW+9KUtwKRG 2wLxx+tVO3ix2a/69LTJF5QZTl+A8QWbC2YP5QMeWcAHj361U7B0MHDQusJhhPtd4GiQ GE18aH8EOHcriwZ9GUXlS4ZBC/M2Kw0oRHUHMptFhGWpAJtBJgKMk08p5mKfatph/nnP diJrsy4ZuVan2Vqh65Opw2lWfC0BeAxYpVm5mQFqGgXmxZAFPqMA7JcsOo9MxjXsbkoP EHjw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rice-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=L+2iXhzmL5I4fY/OktCbptotLej9fyPEwdWoihFTbUA=; b=ZcKGZJPtW7tqZcV7kchTyYad3eeUVRixwjnWCeVzlH8xQwzvzKi6JalynXtCRlQMLu HaQ1/wsnuJA8RU3O/ubQdEblZxazqExeLuzlO+vhjUIllwWyAq5Du/Pt3msoAmO9Rx7U VEw/haTRXc++7qfrg4PzneNl/XlJay29eJCVm2TvCXKM5mjX2Z40X20S2rg7lsAlsntb F+E75x1km94cxliPNrFrjPp5ZLiibmki1DTYmDJGoAZUnWWC1BUj77t4j/U9Ns+5Xg8j cNxDAb3+L6h5kGtsfTZtYBL4kzlSik/eQQGlipICgU+MPnw8pE2/AbZVfBlI3pXB03xr t1GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=L+2iXhzmL5I4fY/OktCbptotLej9fyPEwdWoihFTbUA=; b=EUzP+FB/W527ZfG98Bd1b3YHhj/FW3Nduk6S2SrQPAhSC4yaJZtB/WMUIpfQJoovW6 254GUTrxP9Tz/lvwUFJlGfgsCfty/xthfOssQfwj7ZpMABSXgja1A+bCEkGfJbs3k2b1 FkxCxRXw3fjqv/5Bw+D3N7w/pf2WSYmAydBA27yOQehd7ZuSvv/lB3v7eUBABmVClb7v rG6rFKLdDalAuN0C03oE1tK40P3FZTxVhle/Xb+dD8Dso4AecDy8E5L3ULTlWU2eZjf2 6YXm4z2efFDBP0IE4fmNpwCDyQzMIIwHfVu4NHKzeDCMv4GtPCGUJiUs96nUP52wdRJy PRMg== X-Gm-Message-State: AG10YORj8+63NUJHEaCBYVHRLQ++i/BMWsqWj2GVYXr8iQjNrEaP9XCuQvgwiNhJIgo6uXbE2Cx6n59y4aHeYQ== MIME-Version: 1.0 X-Received: by 10.50.142.68 with SMTP id ru4mr14862591igb.54.1454369076747; Mon, 01 Feb 2016 15:24:36 -0800 (PST) Sender: sunxiaoye07@gmail.com Received: by 10.36.98.82 with HTTP; Mon, 1 Feb 2016 15:24:36 -0800 (PST) In-Reply-To: References: Date: Mon, 1 Feb 2016 17:24:36 -0600 X-Google-Sender-Auth: NYlrBqS5n5_XG7ttqybXqwKFXSs Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Xiaoye Sun To: Luigi Rizzo Cc: Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 23:24:37 -0000 Hi Luigi, Thanks for the detailed advice. With more detailed experiments, actually I found that the udp sender/receiver packet reorder issue *might* be irrelevant to the original issue I posted. However, I think we should solve the udp sender/receiver issue first. I run the experiment with more detailed log. Here is my findings. 1. I am running a netmap version available since about Oct 13rd from github (https://github.com/luigirizzo/netmap). So I think this is not the one related to the buffer allocation issue. I tried to running the newest version, however, that version causes problem when I exit the bridge program (something like kernel error which make the os crash). 2 & 3. I changed the receiver.c & bridge.c so that I can get more information (more detailed log). The reorder happens multiple times (about 10 times) within a second. Here is one example trace collected from the above two programs. (remembering that we have udp sender running on one machine; netmap bridge and udp receiver are running on another machine). There is only one pair of rings each with 512 slots (511 slot usable) on the receiver machine. =================== packet trace collected from receiver.c =================== ===== together with the slot and buf_idx of the corresponding netmap ring slots ====== [seq] [slot] [buf_idx] 8208 294 1833 *8209 295 1834* *8388 474 2013* ... (packet received in order) 8398 484 2023 *8399 485 2024* *8210 296 1835* 8211 297 1836 ... (packet received in order) ... *8222 308 1847* *8400 486 2025* *8223 309 1848* *8401 487 2026* *8224 310 1849* *8402 488 2027* *8225 311 1850* *8403 489 2028* *8226 312 1851* *8404 450 2029* *8227 313 1852* 8228 314 1853 =================================================================== As we can see that the udp receiver got packet 8210 after it got 8399, which is the first reorder. Then, the receiver got 8211 to 8222 sequentially. Then it got packet from 8223-8227 and 8400-8404 interleaved. ==================== event order seen by netmap bridge ================== get 8209 poll called *get 8210* ... ... get 8228 poll called get 8229 ... ... get 8383 poll called get 8384 ... get 8387 poll called get 8388 ... get 8393 poll called get 8394 ... *get 8399* poll called *get 8400* ... get 8404 poll called get 8405 =================================================================== As we can see, from the event ordering see by the bridge.c, all the packets are receiver in order, which means the the reorder happens when the bridge code swap the buf_idx between the nic ring(slot) and the host ring(slot). The reordered seq usually right before or after the poll function call. Best, Xiaoye On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo wrote: > On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun wrote: > > Hi Luigi, > > > > Thanks for your advice. > > I forgot to mention that I use the command "ethtool -L eth1 combined 1" > to > > set the number of rings of the nic to 1. The host also only has one > ring. > > I understand the situation where the first tx ring is full so the bridge > > will swap the packets to the second tx ring and then the host/nic might > > drain either rings. But this is not the case in the experiment. > > ok good to know that. > > So if we have ruled out multiqueue and iommu, let's look at > the internal allocator and at bridge.c > > 1. are you running the most recent version of netmap ? > Some older version (probably 1-2 years ago) had a bug > in the buffer allocator and some buffers were allocated > twice. > > 2. can you tweak your receiver.c to report some more info > on how often you get out of sequence packets, how much > out of sequence they are ? > Also it would be useful to report gaps on the increasing side > (i.e. new_seq != old_seq +1 ) > > 3. can you tweak bridge.c so that it writes into the packet > the netmap buffer indexes and slots on the rx and tx side, > so when you detect a sequence error we can figure out > where it is happening. > Ideally you could also add the sequence number detection > code in bridge.c so we can check whether the errors appear > on the input or output sides. > > cheers > luigi > > From owner-freebsd-net@freebsd.org Tue Feb 2 04:48:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A684A98733 for ; Tue, 2 Feb 2016 04:48:55 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lf0-x234.google.com (mail-lf0-x234.google.com [IPv6:2a00:1450:4010:c07::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 87E91F99 for ; Tue, 2 Feb 2016 04:48:54 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-lf0-x234.google.com with SMTP id m1so35664579lfg.0 for ; Mon, 01 Feb 2016 20:48:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=dDuBUbj8fiuD9SoHi8K+UO2Yg7IpvaZKoRfQ5ulQNZo=; b=RUtNa+yJ8K/L1dV2CU9yNe00b3JCHOiJMqWNq3Saq7tLW4vCc8GteZiUUTeltIZkLU 1Ynf3cR1u6XX5Lb69kkaGtnGFBeIa20MpZ8n3vmyE9gahPjjq747OXC20kpiUHk9Dpsc tVS9t9/f/UE5VMeMDGkkX7pOiL5O/FtseNAzwNbSV/wkzzPpxBHsU4iqFhaJC4WEMLNw 79OyZG6vFjxsXXIo2sy/f0IR5g4VYfUmx9lcgZ3/WNNOvTpTnKN+y8W097Uk0XUQt5uI I9Jm2Kuu4uXFuqmCWsO8TCgyBzjJAu+kE5vULuTQvUH34U3rCbVLddTY0Cb1AAHNWbXh o3IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=dDuBUbj8fiuD9SoHi8K+UO2Yg7IpvaZKoRfQ5ulQNZo=; b=Z60rT+/Fi8e+nR1uN1SyLOAq1jlckTt1FT9r/A7SjiwFuIY2zoa/M7laV+GP6Up++y ICdzOQxXTDLDZp4XSSxz8oTtIr8l8uvO+D9RnCl7AJbRkfILcnnUdMFcPNCFnIg5QKRV 4xldhrI9Y+1NQeuBgJBStW7ZSc2bXIuPS0JcecoIeX+ojOZDDe2eaeNSbpfTyvfhksJ+ JyvxQaaxlkkAGaBcwX3nwUZCScPMTda6zpvqcnBFGfeoFI5nH/cqHtbsHnZpajagMeMz shuQqj05yqECwBQ4ZrRS8aUSMpS0z+k35OPlMA0fUrnY2evFuaMKXBFoBPkmyz5pFOKJ YGqA== X-Gm-Message-State: AG10YOTAtNYfTOpnvs0IW4XXZ6ATDN4N2WA6shB6Xo4Bb9U4qAw9eL5uqvxmDPfq84WmMFxHm02amNNYnI8raw== MIME-Version: 1.0 X-Received: by 10.25.84.20 with SMTP id i20mr9918899lfb.7.1454388532328; Mon, 01 Feb 2016 20:48:52 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.4.232 with HTTP; Mon, 1 Feb 2016 20:48:52 -0800 (PST) In-Reply-To: References: Date: Tue, 2 Feb 2016 05:48:52 +0100 X-Google-Sender-Auth: 5jHSepP2oOzwyLQAnaeW3KN_ONk Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Luigi Rizzo To: Xiaoye Sun Cc: Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 04:48:55 -0000 Hi, there must be some wrong with your setting because slot indexes must be sequential and in your case they are not (see the jump from 295 to 474 and then back from 485 to 296, and the numerous interleavings that you are seeing later). I have no idea of the cause but typically this pattern is what you see when there are multiple input rings and not just one. Cheers Luigi On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun wrote: > Hi Luigi, > > Thanks for the detailed advice. > > With more detailed experiments, actually I found that the udp > sender/receiver packet reorder issue *might* be irrelevant to the original > issue I posted. However, I think we should solve the udp sender/receiver > issue first. > I run the experiment with more detailed log. Here is my findings. > > 1. I am running a netmap version available since about Oct 13rd from github > (https://github.com/luigirizzo/netmap). So I think this is not the one > related to the buffer allocation issue. I tried to running the newest > version, however, that version causes problem when I exit the bridge program > (something like kernel error which make the os crash). > > 2 & 3. I changed the receiver.c & bridge.c so that I can get more > information (more detailed log). > The reorder happens multiple times (about 10 times) within a second. Here is > one example trace collected from the above two programs. (remembering that > we have udp sender running on one machine; netmap bridge and udp receiver > are running on another machine). > There is only one pair of rings each with 512 slots (511 slot usable) on the > receiver machine. > > =================== packet trace collected from receiver.c > =================== > ===== together with the slot and buf_idx of the corresponding netmap ring > slots ====== > [seq] [slot] [buf_idx] > 8208 294 1833 > 8209 295 1834 > 8388 474 2013 > ... (packet received in order) > 8398 484 2023 > 8399 485 2024 > 8210 296 1835 > 8211 297 1836 > ... (packet received in order) > ... > 8222 308 1847 > 8400 486 2025 > 8223 309 1848 > 8401 487 2026 > 8224 310 1849 > 8402 488 2027 > 8225 311 1850 > 8403 489 2028 > 8226 312 1851 > 8404 450 2029 > 8227 313 1852 > 8228 314 1853 > =================================================================== > As we can see that the udp receiver got packet 8210 after it got 8399, which > is the first reorder. Then, the receiver got 8211 to 8222 sequentially. Then > it got packet from 8223-8227 and 8400-8404 interleaved. > > > ==================== event order seen by netmap bridge ================== > get 8209 > poll called > get 8210 > ... > ... > get 8228 > poll called > get 8229 > ... > ... > get 8383 > poll called > get 8384 > ... > get 8387 > poll called > get 8388 > ... > get 8393 > poll called > get 8394 > ... > get 8399 > poll called > get 8400 > ... > get 8404 > poll called > get 8405 > =================================================================== > As we can see, from the event ordering see by the bridge.c, all the packets > are receiver in order, which means the the reorder happens when the bridge > code swap the buf_idx between the nic ring(slot) and the host ring(slot). > The reordered seq usually right before or after the poll function call. > > Best, > Xiaoye > > > > > > > > > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo wrote: >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun wrote: >> > Hi Luigi, >> > >> > Thanks for your advice. >> > I forgot to mention that I use the command "ethtool -L eth1 combined 1" >> > to >> > set the number of rings of the nic to 1. The host also only has one >> > ring. >> > I understand the situation where the first tx ring is full so the bridge >> > will swap the packets to the second tx ring and then the host/nic might >> > drain either rings. But this is not the case in the experiment. >> >> ok good to know that. >> >> So if we have ruled out multiqueue and iommu, let's look at >> the internal allocator and at bridge.c >> >> 1. are you running the most recent version of netmap ? >> Some older version (probably 1-2 years ago) had a bug >> in the buffer allocator and some buffers were allocated >> twice. >> >> 2. can you tweak your receiver.c to report some more info >> on how often you get out of sequence packets, how much >> out of sequence they are ? >> Also it would be useful to report gaps on the increasing side >> (i.e. new_seq != old_seq +1 ) >> >> 3. can you tweak bridge.c so that it writes into the packet >> the netmap buffer indexes and slots on the rx and tx side, >> so when you detect a sequence error we can figure out >> where it is happening. >> Ideally you could also add the sequence number detection >> code in bridge.c so we can check whether the errors appear >> on the input or output sides. >> >> cheers >> luigi >> > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@freebsd.org Tue Feb 2 05:23:46 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01F0DA97404 for ; Tue, 2 Feb 2016 05:23:46 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: from mail-ig0-x236.google.com (mail-ig0-x236.google.com [IPv6:2607:f8b0:4001:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B0062682 for ; Tue, 2 Feb 2016 05:23:45 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: by mail-ig0-x236.google.com with SMTP id z14so52608064igp.1 for ; Mon, 01 Feb 2016 21:23:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Lvqt2tpn47PhrI56R6HKCdIwNzpq3jujfrBG2/HoLc0=; b=SNMH6UsGz6WI3bF9gDiYkxmOBeiJ2nUlGyv7Wgm1PK9/lHm6cQ39/GPR8nfRAoygdj G7OvVsl0XFi7F0/ZMKmM6+GgUbT5USpazpVnbTtxaxQ/Cgdctd4efueBZjGoHruYyS+q ONqe1h5UCBkoh859PdBFs2RlEsh+yxATcHztreT3I1RnpWYwzfHR2AVzYGZhtY9Oy08q 4aWLojFkoxUhvcOJ73Pd6LvhkAE3fCuNyX74Vc4DGzhZdFyHzVnhJGD6M/LFeHDyMBJ6 w1e0XxCjuarA0S93+SmH3UeUXZ+AXsfRABeJ+S8TFfTPjMmDQfPCxfuXTDk5MV2B/A7/ E0vw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rice-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Lvqt2tpn47PhrI56R6HKCdIwNzpq3jujfrBG2/HoLc0=; b=PCreveNRStxg4DR2xIpwcD1IbUuMQRhCNTuYyDYpxlpBn/t5QcPFMOyAA7ceWYD7ia W9Qf9yg8dbX2l9lD413FxzRw4zJrkxnncekzaDNlJejh+I5u1eva8hzasHrEVegBS6ww zr8jdMfpivQqXbm4GEIg1AHLqYDr9TI5J+GI0YfALmDU8VuhYpj9cPPESiJYn/zl8mak l76VN7tgEFTbW86XsVSNQ4fT03ZnYKgtoympS8x6kJbkSv/2vVDe0/Ta63V8q/bLwoKz Dg0WUTi0BQ2qN0svXa/aAYyhLF6A+POUndwb0yFScxsUShLKYTq3QADbXSQN089XqaDm Fr+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Lvqt2tpn47PhrI56R6HKCdIwNzpq3jujfrBG2/HoLc0=; b=HHbikJ2/xViz8CnR8p/VZ4EoJQR0oDXHpKuTpjJkGmoy1VIosuV+DDFGOS80EKRIco 6FovURAcH6L23Z1o4+9137mvwSgcmSzi5D19XWqUqUw2YT7GqHRlbf+KSmtNeS/S2mwn esqz383SV+Gsj+98t2wClt2jz+8Bz4CGwjhmyCBXAQ17R1LvuQlr4rR1E8KHXOaEhjrP +RqjnQZbzHXyolfJWOovq5aer0TAu/JQ2U1Epbni+xpFWJ/NnPKADy5BNQNZMOOzfNZ/ q4he4Y8J4gwjm0PlUqrhbC2U+yp9cC4skMYWK84uBv5O3LYFubwyMhOesTeQImpGm6KJ xr5A== X-Gm-Message-State: AG10YOQzh92Pm6oZivvurlRmgWM74NkPtXrNU7ZqH4Uc7sYmf3Q3+xyt7G7eQGoa4kWSfWGmTtbkkVp+SPCX9g== MIME-Version: 1.0 X-Received: by 10.50.110.5 with SMTP id hw5mr15590871igb.65.1454390624815; Mon, 01 Feb 2016 21:23:44 -0800 (PST) Sender: sunxiaoye07@gmail.com Received: by 10.36.98.82 with HTTP; Mon, 1 Feb 2016 21:23:44 -0800 (PST) In-Reply-To: References: Date: Mon, 1 Feb 2016 23:23:44 -0600 X-Google-Sender-Auth: IOdU6h2zMzrvaFWkFhR0Hh__rLQ Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Xiaoye Sun To: Luigi Rizzo Cc: Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 05:23:46 -0000 Hi Luigi, I have to clarify about the *jumping issue* about the slot indexes. In the bridge.c program, the slot index never jumps and it increases sequentially. In the receiver.c program, the udp packet seq jumps and I showed the slot index that each udp packet uses. So the slot index jumps together with the udp seq (at the receiver program only). There is really one ring (tx and rx) for NIC and one ring (tx and rx) for the host. I also doubt that there might be multiple tx rings for the host. It seems like that bridge program swap packet to multiple host rings and the udp recv program drains packets from these rings. But this is not the case here. The bridge program prints a line like this *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* this is printed by the following line the original program *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, pb->first_rx_ring, pb->req.nr_rx_rings);* I think this shows that there is really one NIC ring and one HOST ring. Is there another way to verify the number of ring that netmap has? Thanks! Xiaoye On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo wrote: > Hi, > there must be some wrong with your setting because > slot indexes must be sequential and in your case they > are not (see the jump from 295 to 474 and then > back from 485 to 296, and the numerous interleavings > that you are seeing later). > > I have no idea of the cause but typically this pattern > is what you see when there are multiple input rings and > not just one. > > Cheers > Luigi > > > > > On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun wrote: > > Hi Luigi, > > > > Thanks for the detailed advice. > > > > With more detailed experiments, actually I found that the udp > > sender/receiver packet reorder issue *might* be irrelevant to the > original > > issue I posted. However, I think we should solve the udp sender/receiver > > issue first. > > I run the experiment with more detailed log. Here is my findings. > > > > 1. I am running a netmap version available since about Oct 13rd from > github > > (https://github.com/luigirizzo/netmap). So I think this is not the one > > related to the buffer allocation issue. I tried to running the newest > > version, however, that version causes problem when I exit the bridge > program > > (something like kernel error which make the os crash). > > > > 2 & 3. I changed the receiver.c & bridge.c so that I can get more > > information (more detailed log). > > The reorder happens multiple times (about 10 times) within a second. > Here is > > one example trace collected from the above two programs. (remembering > that > > we have udp sender running on one machine; netmap bridge and udp receiver > > are running on another machine). > > There is only one pair of rings each with 512 slots (511 slot usable) on > the > > receiver machine. > > > > =================== packet trace collected from receiver.c > > =================== > > ===== together with the slot and buf_idx of the corresponding netmap ring > > slots ====== > > [seq] [slot] [buf_idx] > > 8208 294 1833 > > 8209 295 1834 > > 8388 474 2013 > > ... (packet received in order) > > 8398 484 2023 > > 8399 485 2024 > > 8210 296 1835 > > 8211 297 1836 > > ... (packet received in order) > > ... > > 8222 308 1847 > > 8400 486 2025 > > 8223 309 1848 > > 8401 487 2026 > > 8224 310 1849 > > 8402 488 2027 > > 8225 311 1850 > > 8403 489 2028 > > 8226 312 1851 > > 8404 450 2029 > > 8227 313 1852 > > 8228 314 1853 > > =================================================================== > > As we can see that the udp receiver got packet 8210 after it got 8399, > which > > is the first reorder. Then, the receiver got 8211 to 8222 sequentially. > Then > > it got packet from 8223-8227 and 8400-8404 interleaved. > > > > > > ==================== event order seen by netmap bridge ================== > > get 8209 > > poll called > > get 8210 > > ... > > ... > > get 8228 > > poll called > > get 8229 > > ... > > ... > > get 8383 > > poll called > > get 8384 > > ... > > get 8387 > > poll called > > get 8388 > > ... > > get 8393 > > poll called > > get 8394 > > ... > > get 8399 > > poll called > > get 8400 > > ... > > get 8404 > > poll called > > get 8405 > > =================================================================== > > As we can see, from the event ordering see by the bridge.c, all the > packets > > are receiver in order, which means the the reorder happens when the > bridge > > code swap the buf_idx between the nic ring(slot) and the host ring(slot). > > The reordered seq usually right before or after the poll function call. > > > > Best, > > Xiaoye > > > > > > > > > > > > > > > > > > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo wrote: > >> > >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun > wrote: > >> > Hi Luigi, > >> > > >> > Thanks for your advice. > >> > I forgot to mention that I use the command "ethtool -L eth1 combined > 1" > >> > to > >> > set the number of rings of the nic to 1. The host also only has one > >> > ring. > >> > I understand the situation where the first tx ring is full so the > bridge > >> > will swap the packets to the second tx ring and then the host/nic > might > >> > drain either rings. But this is not the case in the experiment. > >> > >> ok good to know that. > >> > >> So if we have ruled out multiqueue and iommu, let's look at > >> the internal allocator and at bridge.c > >> > >> 1. are you running the most recent version of netmap ? > >> Some older version (probably 1-2 years ago) had a bug > >> in the buffer allocator and some buffers were allocated > >> twice. > >> > >> 2. can you tweak your receiver.c to report some more info > >> on how often you get out of sequence packets, how much > >> out of sequence they are ? > >> Also it would be useful to report gaps on the increasing side > >> (i.e. new_seq != old_seq +1 ) > >> > >> 3. can you tweak bridge.c so that it writes into the packet > >> the netmap buffer indexes and slots on the rx and tx side, > >> so when you detect a sequence error we can figure out > >> where it is happening. > >> Ideally you could also add the sequence number detection > >> code in bridge.c so we can check whether the errors appear > >> on the input or output sides. > >> > >> cheers > >> luigi > >> > > > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2217533 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > > From owner-freebsd-net@freebsd.org Tue Feb 2 05:34:51 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41E4EA977F1 for ; Tue, 2 Feb 2016 05:34:51 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A131AAD4 for ; Tue, 2 Feb 2016 05:34:50 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-lb0-x235.google.com with SMTP id cl12so87651072lbc.1 for ; Mon, 01 Feb 2016 21:34:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=TGq+QzhqQ2A/hxvhmQshjph9jnzIKTYk2CtaJLhdG4o=; b=Vt4a7Wf79PStDGFPEFON83Px7t1k/kwXZrdFvOVnSM7iCmw5nFuGlh78WnDBxRA9Rh cwv9H+tSlotykwJY6BtsHxvXfNCSGuxQV+xN/C/MAAL3ONclPaaXBmvqEjSrbDrPSXL7 ofsdIsQ+BjIy3WThOyDdAoPNt62noa5XjiMAqSE15Y8+yMh6YWGXGdVyLudmX06sRVo7 1w9Wt7g2XPcxwZNXlS1+qWaaS218BNchmFQExt5jKjhToqvzIF8/kPRakuIozXLZXJOB YlLPgClrswa+usX9Jw4vG14LFFwSCSZosnFUWxhSmibg4FRVcrSEHm4UJ5EG/hqaDyVm +egQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=TGq+QzhqQ2A/hxvhmQshjph9jnzIKTYk2CtaJLhdG4o=; b=h0w/ZTYAxGYwPuHOhM8QCmG2Wak5edNbeKV6VjfdR7iX7D5W8lQ2Yf+D8x8kJpftEh gX8y+/tth2OJ8r1eVcljwN1DuMkf3CahcfnmO6MFUggcyzNXisqRtAxXKpJZXXKerKC3 P56WpUiT7U7USEtQc/Eg7Saly6OCi2iu/s4JTUDn5gcR+L2uSRpPGg5RX87yEUODSccp yA0X+8rWYndF/SMNo4E7REdDQcAurLIJKj1m+pXeDAjMuVFNzojb04C/DYIIL51vvHzh 8ylm6u4Rs3diuy60Vp2hNsvatInUfEVmfcxXEYpdm6LSI7lUx4E0DQaPlcZ4w78R8U0c 5g5Q== X-Gm-Message-State: AG10YOSBevrOR59hB9xRlQcIVJGijPRhmXArJy3NaxQ2nvsOWpuhJU0c3vvH1btz5EV/pFIC5vBQBH9knSgHwg== MIME-Version: 1.0 X-Received: by 10.112.126.72 with SMTP id mw8mr10224354lbb.14.1454391288571; Mon, 01 Feb 2016 21:34:48 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.4.232 with HTTP; Mon, 1 Feb 2016 21:34:48 -0800 (PST) In-Reply-To: References: Date: Tue, 2 Feb 2016 06:34:48 +0100 X-Google-Sender-Auth: FZMAk3I8UbiXoeKAfEZPMNVJ8M4 Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Luigi Rizzo To: Xiaoye Sun Cc: Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 05:34:51 -0000 On Tue, Feb 2, 2016 at 6:23 AM, Xiaoye Sun wrote: > Hi Luigi, > > I have to clarify about the *jumping issue* about the slot indexes. > In the bridge.c program, the slot index never jumps and it increases > sequentially. > In the receiver.c program, the udp packet seq jumps and I showed the slot > index that each udp packet uses. So the slot index jumps together with the > udp seq (at the receiver program only). So let me understand, is the "slot" some information written in the packet by bridge.c (referring to the rx or tx slot, I am not sure) and then read and printed by receiver.c (which gets the packet through recvfrom so there isn't really any slot index) ? Do you see any ordering inversion when the receiver gets packets through the NETMAP API (e.g. using bridge.c instead of receiver.c) ? Are you using native netmap drivers or the emulated mode ? You can check that by playing with the "admode" sysctl entry (or sysfs on linux) - try setting to 1 and 2 and see if the behaviour changes. dev.netmap.admode: 0 Controls the use of native or emulated adapter mode. 0 uses the best available option, 1 forces native and fails if not available, 2 forces emulated hence never fails. cheers luigi > > There is really one ring (tx and rx) for NIC and one ring (tx and rx) for > the host. > I also doubt that there might be multiple tx rings for the host. It seems > like that bridge program swap packet to multiple host rings and the udp recv > program drains packets from these rings. But this is not the case here. > > The bridge program prints a line like this > *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* > this is printed by the following line the original program > *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, > pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, pb->first_rx_ring, > pb->req.nr_rx_rings);* > > I think this shows that there is really one NIC ring and one HOST ring. > > Is there another way to verify the number of ring that netmap has? > > Thanks! > Xiaoye > > On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo wrote: >> >> Hi, >> there must be some wrong with your setting because >> slot indexes must be sequential and in your case they >> are not (see the jump from 295 to 474 and then >> back from 485 to 296, and the numerous interleavings >> that you are seeing later). >> >> I have no idea of the cause but typically this pattern >> is what you see when there are multiple input rings and >> not just one. >> >> Cheers >> Luigi >> >> >> >> >> On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun wrote: >> > Hi Luigi, >> > >> > Thanks for the detailed advice. >> > >> > With more detailed experiments, actually I found that the udp >> > sender/receiver packet reorder issue *might* be irrelevant to the >> > original >> > issue I posted. However, I think we should solve the udp sender/receiver >> > issue first. >> > I run the experiment with more detailed log. Here is my findings. >> > >> > 1. I am running a netmap version available since about Oct 13rd from >> > github >> > (https://github.com/luigirizzo/netmap). So I think this is not the one >> > related to the buffer allocation issue. I tried to running the newest >> > version, however, that version causes problem when I exit the bridge >> > program >> > (something like kernel error which make the os crash). >> > >> > 2 & 3. I changed the receiver.c & bridge.c so that I can get more >> > information (more detailed log). >> > The reorder happens multiple times (about 10 times) within a second. >> > Here is >> > one example trace collected from the above two programs. (remembering >> > that >> > we have udp sender running on one machine; netmap bridge and udp >> > receiver >> > are running on another machine). >> > There is only one pair of rings each with 512 slots (511 slot usable) on >> > the >> > receiver machine. >> > >> > =================== packet trace collected from receiver.c >> > =================== >> > ===== together with the slot and buf_idx of the corresponding netmap >> > ring >> > slots ====== >> > [seq] [slot] [buf_idx] >> > 8208 294 1833 >> > 8209 295 1834 >> > 8388 474 2013 >> > ... (packet received in order) >> > 8398 484 2023 >> > 8399 485 2024 >> > 8210 296 1835 >> > 8211 297 1836 >> > ... (packet received in order) >> > ... >> > 8222 308 1847 >> > 8400 486 2025 >> > 8223 309 1848 >> > 8401 487 2026 >> > 8224 310 1849 >> > 8402 488 2027 >> > 8225 311 1850 >> > 8403 489 2028 >> > 8226 312 1851 >> > 8404 450 2029 >> > 8227 313 1852 >> > 8228 314 1853 >> > =================================================================== >> > As we can see that the udp receiver got packet 8210 after it got 8399, >> > which >> > is the first reorder. Then, the receiver got 8211 to 8222 sequentially. >> > Then >> > it got packet from 8223-8227 and 8400-8404 interleaved. >> > >> > >> > ==================== event order seen by netmap bridge >> > ================== >> > get 8209 >> > poll called >> > get 8210 >> > ... >> > ... >> > get 8228 >> > poll called >> > get 8229 >> > ... >> > ... >> > get 8383 >> > poll called >> > get 8384 >> > ... >> > get 8387 >> > poll called >> > get 8388 >> > ... >> > get 8393 >> > poll called >> > get 8394 >> > ... >> > get 8399 >> > poll called >> > get 8400 >> > ... >> > get 8404 >> > poll called >> > get 8405 >> > =================================================================== >> > As we can see, from the event ordering see by the bridge.c, all the >> > packets >> > are receiver in order, which means the the reorder happens when the >> > bridge >> > code swap the buf_idx between the nic ring(slot) and the host >> > ring(slot). >> > The reordered seq usually right before or after the poll function call. >> > >> > Best, >> > Xiaoye >> > >> > >> > >> > >> > >> > >> > >> > >> > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo wrote: >> >> >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun >> >> wrote: >> >> > Hi Luigi, >> >> > >> >> > Thanks for your advice. >> >> > I forgot to mention that I use the command "ethtool -L eth1 combined >> >> > 1" >> >> > to >> >> > set the number of rings of the nic to 1. The host also only has one >> >> > ring. >> >> > I understand the situation where the first tx ring is full so the >> >> > bridge >> >> > will swap the packets to the second tx ring and then the host/nic >> >> > might >> >> > drain either rings. But this is not the case in the experiment. >> >> >> >> ok good to know that. >> >> >> >> So if we have ruled out multiqueue and iommu, let's look at >> >> the internal allocator and at bridge.c >> >> >> >> 1. are you running the most recent version of netmap ? >> >> Some older version (probably 1-2 years ago) had a bug >> >> in the buffer allocator and some buffers were allocated >> >> twice. >> >> >> >> 2. can you tweak your receiver.c to report some more info >> >> on how often you get out of sequence packets, how much >> >> out of sequence they are ? >> >> Also it would be useful to report gaps on the increasing side >> >> (i.e. new_seq != old_seq +1 ) >> >> >> >> 3. can you tweak bridge.c so that it writes into the packet >> >> the netmap buffer indexes and slots on the rx and tx side, >> >> so when you detect a sequence error we can figure out >> >> where it is happening. >> >> Ideally you could also add the sequence number detection >> >> code in bridge.c so we can check whether the errors appear >> >> on the input or output sides. >> >> >> >> cheers >> >> luigi >> >> >> > >> >> >> >> -- >> -----------------------------------------+------------------------------- >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2217533 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------- >> > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@freebsd.org Tue Feb 2 09:06:14 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 737E5A9764A for ; Tue, 2 Feb 2016 09:06:14 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 5221D92B for ; Tue, 2 Feb 2016 09:06:14 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 4DFBC107678; Tue, 2 Feb 2016 09:06:14 +0000 (UTC) Date: Tue, 2 Feb 2016 09:06:14 +0000 To: freebsd-net@freebsd.org From: "mugius.0x101.freebsd_gmail.com (Mugenga Marius)" Reply-to: D5165+325+ebe40adb3bf8170b@reviews.freebsd.org Subject: [Differential] [Request, 9 lines] D5165: [patch] dev/bwn suppressing "bwn0: unsupported rate 0" console messages Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D5165: [patch] dev/bwn suppressing "bwn0: unsupported rate 0" console messages X-Herald-Rules: <28> X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Index: ZWFmNDNmM2JhYmZjMTE4OGFlOGQ2ZmM4Y2Y1 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_76207cb7079bc6e6e2274d49843006df" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 09:06:14 -0000 --b1_76207cb7079bc6e6e2274d49843006df Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit mugius.0x101.freebsd_gmail.com created this revision. mugius.0x101.freebsd_gmail.com added a reviewer: network. mugius.0x101.freebsd_gmail.com added a subscriber: freebsd-net-list. mugius.0x101.freebsd_gmail.com set the repository for this revision to rS FreeBSD src repository. Herald added a subscriber: imp. REVISION SUMMARY Update to PR206199 REPOSITORY rS FreeBSD src repository REVISION DETAIL https://reviews.freebsd.org/D5165 AFFECTED FILES head/sys/dev/bwn/if_bwn.c CHANGE DETAILS diff --git a/head/sys/dev/bwn/if_bwn.c b/head/sys/dev/bwn/if_bwn.c --- a/head/sys/dev/bwn/if_bwn.c +++ b/head/sys/dev/bwn/if_bwn.c @@ -9467,7 +9467,7 @@ struct mbuf *mprot; unsigned int len; uint32_t macctl = 0; - int protdur, rts_rate, rts_rate_fb, ismcast, isshort, rix, type; + int protdur, rts_rate, rts_rate_fb, ismcast, isshort, nrates, type; uint16_t phyctl = 0; uint8_t rate, rate_fb; @@ -9489,11 +9489,12 @@ else if (tp->ucastrate != IEEE80211_FIXED_RATE_NONE) rate = rate_fb = tp->ucastrate; else { - rix = ieee80211_ratectl_rate(ni, NULL, 0); + ieee80211_ratectl_rate(ni, NULL, 0); + nrates = ni->ni_rates.rs_nrates; rate = ni->ni_txrate; - if (rix > 0) - rate_fb = ni->ni_rates.rs_rates[rix - 1] & + if (nrates > 0) + rate_fb = ni->ni_rates.rs_rates[nrates - 1] & IEEE80211_RATE_VAL; else rate_fb = rate; EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: mugius.0x101.freebsd_gmail.com, network Cc: imp, freebsd-net-list --b1_76207cb7079bc6e6e2274d49843006df Content-Type: text/x-patch; charset=utf-8; name="D5165.12945.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5165.12945.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9id24vaWZfYnduLmMgYi9oZWFkL3N5cy9kZXYvYndu L2lmX2J3bi5jCi0tLSBhL2hlYWQvc3lzL2Rldi9id24vaWZfYnduLmMKKysrIGIvaGVhZC9zeXMv ZGV2L2J3bi9pZl9id24uYwpAQCAtOTQ2Nyw3ICs5NDY3LDcgQEAKIAlzdHJ1Y3QgbWJ1ZiAqbXBy b3Q7CiAJdW5zaWduZWQgaW50IGxlbjsKIAl1aW50MzJfdCBtYWNjdGwgPSAwOwotCWludCBwcm90 ZHVyLCBydHNfcmF0ZSwgcnRzX3JhdGVfZmIsIGlzbWNhc3QsIGlzc2hvcnQsIHJpeCwgdHlwZTsK KwlpbnQgcHJvdGR1ciwgcnRzX3JhdGUsIHJ0c19yYXRlX2ZiLCBpc21jYXN0LCBpc3Nob3J0LCBu cmF0ZXMsIHR5cGU7CiAJdWludDE2X3QgcGh5Y3RsID0gMDsKIAl1aW50OF90IHJhdGUsIHJhdGVf ZmI7CiAKQEAgLTk0ODksMTEgKzk0ODksMTIgQEAKIAllbHNlIGlmICh0cC0+dWNhc3RyYXRlICE9 IElFRUU4MDIxMV9GSVhFRF9SQVRFX05PTkUpCiAJCXJhdGUgPSByYXRlX2ZiID0gdHAtPnVjYXN0 cmF0ZTsKIAllbHNlIHsKLQkJcml4ID0gaWVlZTgwMjExX3JhdGVjdGxfcmF0ZShuaSwgTlVMTCwg MCk7CisJCWllZWU4MDIxMV9yYXRlY3RsX3JhdGUobmksIE5VTEwsIDApOworCQlucmF0ZXMgPSBu aS0+bmlfcmF0ZXMucnNfbnJhdGVzOwogCQlyYXRlID0gbmktPm5pX3R4cmF0ZTsKIAotCQlpZiAo cml4ID4gMCkKLQkJCXJhdGVfZmIgPSBuaS0+bmlfcmF0ZXMucnNfcmF0ZXNbcml4IC0gMV0gJgor CQlpZiAobnJhdGVzID4gMCkKKwkJCXJhdGVfZmIgPSBuaS0+bmlfcmF0ZXMucnNfcmF0ZXNbbnJh dGVzIC0gMV0gJgogCQkJICAgIElFRUU4MDIxMV9SQVRFX1ZBTDsKIAkJZWxzZQogCQkJcmF0ZV9m YiA9IHJhdGU7Cgo= --b1_76207cb7079bc6e6e2274d49843006df-- From owner-freebsd-net@freebsd.org Tue Feb 2 10:22:29 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB008A98524 for ; Tue, 2 Feb 2016 10:22:29 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 98C3C1338 for ; Tue, 2 Feb 2016 10:22:29 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 965621078FA; Tue, 2 Feb 2016 10:22:29 +0000 (UTC) Date: Tue, 2 Feb 2016 10:22:29 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5166+325+a3471c38b880b49e@reviews.freebsd.org Subject: [Differential] [Request, 28 lines] D5166: hyperv/hn: Increase LRO entry count to 128 by default Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D5166: hyperv/hn: Increase LRO entry count to 128 by default X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk Thread-Index: NWRmOGI3NDFlMWM4OTg3OGUxYjZjYzIwNmVl MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_020731e7f71b5160e2241da0917756fc" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 10:22:29 -0000 --b1_020731e7f71b5160e2241da0917756fc Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com created this revision. sepherosa_gmail.com added reviewers: network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com. sepherosa_gmail.com added a subscriber: freebsd-net-list. REVISION SUMMARY hn(4) only has one RX ring, so default 8 LRO entries are too small. REVISION DETAIL https://reviews.freebsd.org/D5166 AFFECTED FILES sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -132,6 +132,8 @@ /* YYY should get it from the underlying channel */ #define HN_TX_DESC_CNT 512 +#define HN_LROENT_CNT_DEF 128 + #define HN_RNDIS_MSG_LEN \ (sizeof(rndis_msg) + \ RNDIS_VLAN_PPI_SIZE + \ @@ -232,6 +234,13 @@ static int hn_direct_tx_size = HN_DIRECT_TX_SIZE_DEF; TUNABLE_INT("dev.hn.direct_tx_size", &hn_direct_tx_size); +#if defined(INET) || defined(INET6) +#if __FreeBSD_version >= 1100095 +static int hn_lro_entry_count = HN_LROENT_CNT_DEF; +TUNABLE_INT("dev.hn.lro_entry_count", &hn_lro_entry_count); +#endif +#endif + /* * Forward declarations */ @@ -335,6 +344,11 @@ #if __FreeBSD_version >= 1100045 int tso_maxlen; #endif +#if defined(INET) || defined(INET6) +#if __FreeBSD_version >= 1100095 + int lroent_cnt; +#endif +#endif sc = device_get_softc(dev); if (sc == NULL) { @@ -417,9 +431,17 @@ } #if defined(INET) || defined(INET6) +#if __FreeBSD_version >= 1100095 + lroent_cnt = hn_lro_entry_count; + if (lroent_cnt < TCP_LRO_ENTRIES) + lroent_cnt = TCP_LRO_ENTRIES; + tcp_lro_init_args(&sc->hn_lro, ifp, lroent_cnt, 0); + device_printf(dev, "LRO: entry count %d\n", lroent_cnt); +#else tcp_lro_init(&sc->hn_lro); /* Driver private LRO settings */ sc->hn_lro.ifp = ifp; +#endif #ifdef HN_LRO_HIWAT sc->hn_lro.lro_hiwat = sc->hn_lro_hiwat; #endif @@ -547,6 +569,12 @@ SYSCTL_ADD_INT(dc_ctx, dc_child, OID_AUTO, "direct_tx_size", CTLFLAG_RD, &hn_direct_tx_size, 0, "Size of the packet for direct transmission"); +#if defined(INET) || defined(INET6) +#if __FreeBSD_version >= 1100095 + SYSCTL_ADD_INT(dc_ctx, dc_child, OID_AUTO, "lro_entry_count", + CTLFLAG_RD, &hn_lro_entry_count, 0, "LRO entry count"); +#endif +#endif } return (0); EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com Cc: freebsd-net-list --b1_020731e7f71b5160e2241da0917756fc Content-Type: text/x-patch; charset=utf-8; name="D5166.12947.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5166.12947.patch" ZGlmZiAtLWdpdCBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2c2NfZHJ2X2ZyZWVic2Qu YyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2c2NfZHJ2X2ZyZWVic2QuYwotLS0gYS9z eXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMKKysrIGIvc3lzL2Rl di9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJlZWJzZC5jCkBAIC0xMzIsNiArMTMyLDgg QEAKIC8qIFlZWSBzaG91bGQgZ2V0IGl0IGZyb20gdGhlIHVuZGVybHlpbmcgY2hhbm5lbCAqLwog I2RlZmluZSBITl9UWF9ERVNDX0NOVAkJCTUxMgogCisjZGVmaW5lIEhOX0xST0VOVF9DTlRfREVG CQkxMjgKKwogI2RlZmluZSBITl9STkRJU19NU0dfTEVOCQlcCiAgICAgKHNpemVvZihybmRpc19t c2cpICsJCVwKICAgICAgUk5ESVNfVkxBTl9QUElfU0laRSArCQlcCkBAIC0yMzIsNiArMjM0LDEz IEBACiBzdGF0aWMgaW50IGhuX2RpcmVjdF90eF9zaXplID0gSE5fRElSRUNUX1RYX1NJWkVfREVG OwogVFVOQUJMRV9JTlQoImRldi5obi5kaXJlY3RfdHhfc2l6ZSIsICZobl9kaXJlY3RfdHhfc2l6 ZSk7CiAKKyNpZiBkZWZpbmVkKElORVQpIHx8IGRlZmluZWQoSU5FVDYpCisjaWYgX19GcmVlQlNE X3ZlcnNpb24gPj0gMTEwMDA5NQorc3RhdGljIGludCBobl9scm9fZW50cnlfY291bnQgPSBITl9M Uk9FTlRfQ05UX0RFRjsKK1RVTkFCTEVfSU5UKCJkZXYuaG4ubHJvX2VudHJ5X2NvdW50IiwgJmhu X2xyb19lbnRyeV9jb3VudCk7CisjZW5kaWYKKyNlbmRpZgorCiAvKgogICogRm9yd2FyZCBkZWNs YXJhdGlvbnMKICAqLwpAQCAtMzM1LDYgKzM0NCwxMSBAQAogI2lmIF9fRnJlZUJTRF92ZXJzaW9u ID49IDExMDAwNDUKIAlpbnQgdHNvX21heGxlbjsKICNlbmRpZgorI2lmIGRlZmluZWQoSU5FVCkg fHwgZGVmaW5lZChJTkVUNikKKyNpZiBfX0ZyZWVCU0RfdmVyc2lvbiA+PSAxMTAwMDk1CisJaW50 IGxyb2VudF9jbnQ7CisjZW5kaWYKKyNlbmRpZgogCiAJc2MgPSBkZXZpY2VfZ2V0X3NvZnRjKGRl dik7CiAJaWYgKHNjID09IE5VTEwpIHsKQEAgLTQxNyw5ICs0MzEsMTcgQEAKIAl9CiAKICNpZiBk ZWZpbmVkKElORVQpIHx8IGRlZmluZWQoSU5FVDYpCisjaWYgX19GcmVlQlNEX3ZlcnNpb24gPj0g MTEwMDA5NQorCWxyb2VudF9jbnQgPSBobl9scm9fZW50cnlfY291bnQ7CisJaWYgKGxyb2VudF9j bnQgPCBUQ1BfTFJPX0VOVFJJRVMpCisJCWxyb2VudF9jbnQgPSBUQ1BfTFJPX0VOVFJJRVM7CisJ dGNwX2xyb19pbml0X2FyZ3MoJnNjLT5obl9scm8sIGlmcCwgbHJvZW50X2NudCwgMCk7CisJZGV2 aWNlX3ByaW50ZihkZXYsICJMUk86IGVudHJ5IGNvdW50ICVkXG4iLCBscm9lbnRfY250KTsKKyNl bHNlCiAJdGNwX2xyb19pbml0KCZzYy0+aG5fbHJvKTsKIAkvKiBEcml2ZXIgcHJpdmF0ZSBMUk8g c2V0dGluZ3MgKi8KIAlzYy0+aG5fbHJvLmlmcCA9IGlmcDsKKyNlbmRpZgogI2lmZGVmIEhOX0xS T19ISVdBVAogCXNjLT5obl9scm8ubHJvX2hpd2F0ID0gc2MtPmhuX2xyb19oaXdhdDsKICNlbmRp ZgpAQCAtNTQ3LDYgKzU2OSwxMiBAQAogCQlTWVNDVExfQUREX0lOVChkY19jdHgsIGRjX2NoaWxk LCBPSURfQVVUTywgImRpcmVjdF90eF9zaXplIiwKIAkJICAgIENUTEZMQUdfUkQsICZobl9kaXJl Y3RfdHhfc2l6ZSwgMCwKIAkJICAgICJTaXplIG9mIHRoZSBwYWNrZXQgZm9yIGRpcmVjdCB0cmFu c21pc3Npb24iKTsKKyNpZiBkZWZpbmVkKElORVQpIHx8IGRlZmluZWQoSU5FVDYpCisjaWYgX19G cmVlQlNEX3ZlcnNpb24gPj0gMTEwMDA5NQorCQlTWVNDVExfQUREX0lOVChkY19jdHgsIGRjX2No aWxkLCBPSURfQVVUTywgImxyb19lbnRyeV9jb3VudCIsCisJCSAgICBDVExGTEFHX1JELCAmaG5f bHJvX2VudHJ5X2NvdW50LCAwLCAiTFJPIGVudHJ5IGNvdW50Iik7CisjZW5kaWYKKyNlbmRpZgog CX0KIAogCXJldHVybiAoMCk7Cgo= --b1_020731e7f71b5160e2241da0917756fc-- From owner-freebsd-net@freebsd.org Tue Feb 2 10:27:11 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46F4EA986C5 for ; Tue, 2 Feb 2016 10:27:11 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 1F8D414CE for ; Tue, 2 Feb 2016 10:27:11 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 1B9CB107B30; Tue, 2 Feb 2016 10:27:11 +0000 (UTC) Date: Tue, 2 Feb 2016 10:27:11 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5167+325+acc9bcb5dbcc28f2@reviews.freebsd.org Subject: [Differential] [Request, 21 lines] D5167: hyperv/hn: Move LRO flush to the channel processing rollup Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D5167: hyperv/hn: Move LRO flush to the channel processing rollup X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk Thread-Index: N2NlNTE2YjI3MTBjYjEzNmM4OWIyM2FkMjc1 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_bcbac81c98c0f7d73a739f38120bf5b5" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 10:27:11 -0000 --b1_bcbac81c98c0f7d73a739f38120bf5b5 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com created this revision. sepherosa_gmail.com added reviewers: network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com. sepherosa_gmail.com added a subscriber: freebsd-net-list. REVISION SUMMARY This significantly increases LRO aggregation ratio when there are large amount of connections (improves reception performance a lot). REVISION DETAIL https://reviews.freebsd.org/D5167 AFFECTED FILES sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -764,6 +764,15 @@ netvsc_channel_rollup(struct hv_device *device_ctx) { struct hn_softc *sc = device_get_softc(device_ctx->device); +#if defined(INET) || defined(INET6) + struct lro_ctrl *lro = &sc->hn_lro; + struct lro_entry *queued; + + while ((queued = SLIST_FIRST(&lro->lro_active)) != NULL) { + SLIST_REMOVE_HEAD(&lro->lro_active, next); + tcp_lro_flush(lro, queued); + } +#endif if (!sc->hn_txeof) return; @@ -1338,18 +1347,8 @@ } void -netvsc_recv_rollup(struct hv_device *device_ctx) +netvsc_recv_rollup(struct hv_device *device_ctx __unused) { -#if defined(INET) || defined(INET6) - hn_softc_t *sc = device_get_softc(device_ctx->device); - struct lro_ctrl *lro = &sc->hn_lro; - struct lro_entry *queued; - - while ((queued = SLIST_FIRST(&lro->lro_active)) != NULL) { - SLIST_REMOVE_HEAD(&lro->lro_active, next); - tcp_lro_flush(lro, queued); - } -#endif } /* EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com Cc: freebsd-net-list --b1_bcbac81c98c0f7d73a739f38120bf5b5 Content-Type: text/x-patch; charset=utf-8; name="D5167.12948.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5167.12948.patch" ZGlmZiAtLWdpdCBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2c2NfZHJ2X2ZyZWVic2Qu YyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2c2NfZHJ2X2ZyZWVic2QuYwotLS0gYS9z eXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMKKysrIGIvc3lzL2Rl di9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJlZWJzZC5jCkBAIC03NjQsNiArNzY0LDE1 IEBACiBuZXR2c2NfY2hhbm5lbF9yb2xsdXAoc3RydWN0IGh2X2RldmljZSAqZGV2aWNlX2N0eCkK IHsKIAlzdHJ1Y3QgaG5fc29mdGMgKnNjID0gZGV2aWNlX2dldF9zb2Z0YyhkZXZpY2VfY3R4LT5k ZXZpY2UpOworI2lmIGRlZmluZWQoSU5FVCkgfHwgZGVmaW5lZChJTkVUNikKKwlzdHJ1Y3QgbHJv X2N0cmwgKmxybyA9ICZzYy0+aG5fbHJvOworCXN0cnVjdCBscm9fZW50cnkgKnF1ZXVlZDsKKwor CXdoaWxlICgocXVldWVkID0gU0xJU1RfRklSU1QoJmxyby0+bHJvX2FjdGl2ZSkpICE9IE5VTEwp IHsKKwkJU0xJU1RfUkVNT1ZFX0hFQUQoJmxyby0+bHJvX2FjdGl2ZSwgbmV4dCk7CisJCXRjcF9s cm9fZmx1c2gobHJvLCBxdWV1ZWQpOworCX0KKyNlbmRpZgogCiAJaWYgKCFzYy0+aG5fdHhlb2Yp CiAJCXJldHVybjsKQEAgLTEzMzgsMTggKzEzNDcsOCBAQAogfQogCiB2b2lkCi1uZXR2c2NfcmVj dl9yb2xsdXAoc3RydWN0IGh2X2RldmljZSAqZGV2aWNlX2N0eCkKK25ldHZzY19yZWN2X3JvbGx1 cChzdHJ1Y3QgaHZfZGV2aWNlICpkZXZpY2VfY3R4IF9fdW51c2VkKQogewotI2lmIGRlZmluZWQo SU5FVCkgfHwgZGVmaW5lZChJTkVUNikKLQlobl9zb2Z0Y190ICpzYyA9IGRldmljZV9nZXRfc29m dGMoZGV2aWNlX2N0eC0+ZGV2aWNlKTsKLQlzdHJ1Y3QgbHJvX2N0cmwgKmxybyA9ICZzYy0+aG5f bHJvOwotCXN0cnVjdCBscm9fZW50cnkgKnF1ZXVlZDsKLQotCXdoaWxlICgocXVldWVkID0gU0xJ U1RfRklSU1QoJmxyby0+bHJvX2FjdGl2ZSkpICE9IE5VTEwpIHsKLQkJU0xJU1RfUkVNT1ZFX0hF QUQoJmxyby0+bHJvX2FjdGl2ZSwgbmV4dCk7Ci0JCXRjcF9scm9fZmx1c2gobHJvLCBxdWV1ZWQp OwotCX0KLSNlbmRpZgogfQogCiAvKgoK --b1_bcbac81c98c0f7d73a739f38120bf5b5-- From owner-freebsd-net@freebsd.org Tue Feb 2 21:48:33 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C67AA9954E for ; Tue, 2 Feb 2016 21:48:33 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: from mail-io0-x229.google.com (mail-io0-x229.google.com [IPv6:2607:f8b0:4001:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 29FA71C5E for ; Tue, 2 Feb 2016 21:48:33 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: by mail-io0-x229.google.com with SMTP id 9so33785661iom.1 for ; Tue, 02 Feb 2016 13:48:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=2YTNkEA/7CguaNMef5ETWUGNKVz0L1G4YXEfPkR9cSc=; b=lyX0VtB2EIyyEtEPdtKVmdcjhld+vna+jyEV0IVE0Ub+qQSMk3tFKsSfJ03IFe3qq4 QDXV3eEBoXbdw1/aGU8Bz4le40qZVJAzw0g4nlxtNVGFbps9DmQyTYh79pYo5mSNaJ0N 1R66ts254w9PyM0ph60InRv3Bw/AG1t6hvv/ZSEUtjUDTGI0W8rz0vg91C5DR+BvVqHN zFDstca6gxnEkJu0w0TeHCk4MhBU0QewgTN0zJUpsm7WjFgYBxNJWu0SHWKtK4ey9aVN ZAxcrZ/ub8+4gQiEq7xaV9Xr8GVxrnWQMxYuxMO6Zl97XnM9kUpscyvr6fC23fntGn7D Rppg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rice-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=2YTNkEA/7CguaNMef5ETWUGNKVz0L1G4YXEfPkR9cSc=; b=GdexqBDYLMt1XWRuQHa/MvF6qhpjArpU4fWjBm+TQS0x+hesCe7fgSgJIaO+yig8Kq jiN6T90UMqOn2R7z+UHXRXK4qfd+03GK5CpUqJbWmP1ZAWy5qS5WG/waC2nB/3/K0/ON wesN8GGPNEOJHUqYhLC1IhKZI3AgFjK+EEcDCob8AvfZ7B72WjcT0i/L6dI4KUGJu79X XwEgVzT6qFwhBtPfOzvSQSqS7nzzvaDW+U+ekBWjxdmXP2WwKsPTjsozOrUGB7UgfMFx N4IXzpmBy2JsvMxNwjb/GsV8l1S+fT60HV3oTTJOyuoYN3F12khuAQhvluyRJJmBH2UW 0PEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=2YTNkEA/7CguaNMef5ETWUGNKVz0L1G4YXEfPkR9cSc=; b=j6IGNKmCdpy3UPSkqRMp5zo8CRZ4em84CrUHlRUdTdTRU5DlDbfHUwOTwX7vBoxSec Nr3JzoX2vSzCan9uBo/RCqeS4TxwByB49ayqAwgWcFh8Yq+wCQlnYq0PGuWQsWNaTwMP MaZdTS8b+sD3Sa2mxRv4eaS+chCvzfZFaIyMgCBug3Dv1OdtU/qU+czxo9lIC3ll8Qvu wPQtkUO5Mli4jjUuwdTxxgcAWFmjiTUaR8/9Jado63To0xwyJXZbBYd4JqiSR5dW6TcH vrLjGlAlu4t0umdoaKNdtIeg2chXsQp3KDDBVOT/M8aS/znkKrqtOcdneqBh0QskEIq+ vhKg== X-Gm-Message-State: AG10YOS45EtXGzzbzKvXr35DFkbQhyTVFMorj5QJmcCyx7OG1eepM6gCh11SRv+lJ984ZeJKVGwbjwIAMTDJPQ== MIME-Version: 1.0 X-Received: by 10.107.137.100 with SMTP id l97mr14699427iod.110.1454449712619; Tue, 02 Feb 2016 13:48:32 -0800 (PST) Sender: sunxiaoye07@gmail.com Received: by 10.36.98.82 with HTTP; Tue, 2 Feb 2016 13:48:32 -0800 (PST) In-Reply-To: References: Date: Tue, 2 Feb 2016 15:48:32 -0600 X-Google-Sender-Auth: 4h7tGoSYfN3BFG8pwcmev59i3Xg Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Xiaoye Sun To: Luigi Rizzo Cc: Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Feb 2016 21:48:33 -0000 On Mon, Feb 1, 2016 at 11:34 PM, Luigi Rizzo wrote: > On Tue, Feb 2, 2016 at 6:23 AM, Xiaoye Sun wrote: > > Hi Luigi, > > > > I have to clarify about the *jumping issue* about the slot indexes. > > In the bridge.c program, the slot index never jumps and it increases > > sequentially. > > In the receiver.c program, the udp packet seq jumps and I showed the slot > > index that each udp packet uses. So the slot index jumps together with > the > > udp seq (at the receiver program only). > > So let me understand, is the "slot" some information written > in the packet by bridge.c (referring to the rx or tx slot, > I am not sure) and then read and printed by receiver.c > (which gets the packet through recvfrom so there isn't > really any slot index) ? > > It works in the other way: The bridge.c checks the seq numbers of the udp packets in netmap slots (in nic rx ring) before the swap; then it records the seq number, slot number(both rx and tx (tx indexes were not shown in the previous email since they all look correct)) and buf_idx (rx and tx). The bridge.c does not change anything in the buffer and it knows the slot and buf_idx that a packet uses. Please refer to the added code in *process_rings* function http://www.owlnet.rice.edu/~xs6/bridge.c The receiver.c checks the seq numbers only and print out the seq numbers it receive sequentially. With these information, I manually match the seq number I got from receiver.c and the seq number I got from bridge.c. So we know what is the seq order the receiver sees and which slot a packet uses when bridge.c swaps the buf_idxs. Do you see any ordering inversion when the receiver > gets packets through the NETMAP API (e.g. using bridge.c > instead of receiver.c) ? > > There is no ordering inversion seen by bridge.c (As I said in the previous paragraph, the bridge.c checks the seq number and I did not see any order inversion in THIS simple experiment (In my multicast protocol (mentioned in the first email), there is ordering inversion. But let us solve the simple bridge.c's problem first. I think they are two relatively independent issues.)). > Are you using native netmap drivers or the emulated mode ? > You can check that by playing with the "admode" sysctl entry > (or sysfs on linux) - try setting to 1 and 2 and see if > the behaviour changes. > > dev.netmap.admode: 0 > Controls the use of native or emulated adapter mode. > 0 uses the best available option, > 1 forces native and fails if not available, > 2 forces emulated hence never fails. > > I was using admode 0. I changed the admode to 1 and 2 using the command like *echo 1 > /sys/module/netmap/parameters/admode* and restart the bridge program. The behavior keeps the same. > cheers > luigi > > > > > There is really one ring (tx and rx) for NIC and one ring (tx and rx) for > > the host. > > I also doubt that there might be multiple tx rings for the host. It seems > > like that bridge program swap packet to multiple host rings and the udp > recv > > program drains packets from these rings. But this is not the case here. > > > > The bridge program prints a line like this > > *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* > > this is printed by the following line the original program > > *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, > > pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, > pb->first_rx_ring, > > pb->req.nr_rx_rings);* > > > > I think this shows that there is really one NIC ring and one HOST ring. > > > > Is there another way to verify the number of ring that netmap has? > > > > Thanks! > > Xiaoye > > > > On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo wrote: > >> > >> Hi, > >> there must be some wrong with your setting because > >> slot indexes must be sequential and in your case they > >> are not (see the jump from 295 to 474 and then > >> back from 485 to 296, and the numerous interleavings > >> that you are seeing later). > >> > >> I have no idea of the cause but typically this pattern > >> is what you see when there are multiple input rings and > >> not just one. > >> > >> Cheers > >> Luigi > >> > >> > >> > >> > >> On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun > wrote: > >> > Hi Luigi, > >> > > >> > Thanks for the detailed advice. > >> > > >> > With more detailed experiments, actually I found that the udp > >> > sender/receiver packet reorder issue *might* be irrelevant to the > >> > original > >> > issue I posted. However, I think we should solve the udp > sender/receiver > >> > issue first. > >> > I run the experiment with more detailed log. Here is my findings. > >> > > >> > 1. I am running a netmap version available since about Oct 13rd from > >> > github > >> > (https://github.com/luigirizzo/netmap). So I think this is not the > one > >> > related to the buffer allocation issue. I tried to running the newest > >> > version, however, that version causes problem when I exit the bridge > >> > program > >> > (something like kernel error which make the os crash). > >> > > >> > 2 & 3. I changed the receiver.c & bridge.c so that I can get more > >> > information (more detailed log). > >> > The reorder happens multiple times (about 10 times) within a second. > >> > Here is > >> > one example trace collected from the above two programs. (remembering > >> > that > >> > we have udp sender running on one machine; netmap bridge and udp > >> > receiver > >> > are running on another machine). > >> > There is only one pair of rings each with 512 slots (511 slot usable) > on > >> > the > >> > receiver machine. > >> > > >> > =================== packet trace collected from receiver.c > >> > =================== > >> > ===== together with the slot and buf_idx of the corresponding netmap > >> > ring > >> > slots ====== > >> > [seq] [slot] [buf_idx] > >> > 8208 294 1833 > >> > 8209 295 1834 > >> > 8388 474 2013 > >> > ... (packet received in order) > >> > 8398 484 2023 > >> > 8399 485 2024 > >> > 8210 296 1835 > >> > 8211 297 1836 > >> > ... (packet received in order) > >> > ... > >> > 8222 308 1847 > >> > 8400 486 2025 > >> > 8223 309 1848 > >> > 8401 487 2026 > >> > 8224 310 1849 > >> > 8402 488 2027 > >> > 8225 311 1850 > >> > 8403 489 2028 > >> > 8226 312 1851 > >> > 8404 450 2029 > >> > 8227 313 1852 > >> > 8228 314 1853 > >> > =================================================================== > >> > As we can see that the udp receiver got packet 8210 after it got 8399, > >> > which > >> > is the first reorder. Then, the receiver got 8211 to 8222 > sequentially. > >> > Then > >> > it got packet from 8223-8227 and 8400-8404 interleaved. > >> > > >> > > >> > ==================== event order seen by netmap bridge > >> > ================== > >> > get 8209 > >> > poll called > >> > get 8210 > >> > ... > >> > ... > >> > get 8228 > >> > poll called > >> > get 8229 > >> > ... > >> > ... > >> > get 8383 > >> > poll called > >> > get 8384 > >> > ... > >> > get 8387 > >> > poll called > >> > get 8388 > >> > ... > >> > get 8393 > >> > poll called > >> > get 8394 > >> > ... > >> > get 8399 > >> > poll called > >> > get 8400 > >> > ... > >> > get 8404 > >> > poll called > >> > get 8405 > >> > =================================================================== > >> > As we can see, from the event ordering see by the bridge.c, all the > >> > packets > >> > are receiver in order, which means the the reorder happens when the > >> > bridge > >> > code swap the buf_idx between the nic ring(slot) and the host > >> > ring(slot). > >> > The reordered seq usually right before or after the poll function > call. > >> > > >> > Best, > >> > Xiaoye > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo > wrote: > >> >> > >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun > >> >> wrote: > >> >> > Hi Luigi, > >> >> > > >> >> > Thanks for your advice. > >> >> > I forgot to mention that I use the command "ethtool -L eth1 > combined > >> >> > 1" > >> >> > to > >> >> > set the number of rings of the nic to 1. The host also only has > one > >> >> > ring. > >> >> > I understand the situation where the first tx ring is full so the > >> >> > bridge > >> >> > will swap the packets to the second tx ring and then the host/nic > >> >> > might > >> >> > drain either rings. But this is not the case in the experiment. > >> >> > >> >> ok good to know that. > >> >> > >> >> So if we have ruled out multiqueue and iommu, let's look at > >> >> the internal allocator and at bridge.c > >> >> > >> >> 1. are you running the most recent version of netmap ? > >> >> Some older version (probably 1-2 years ago) had a bug > >> >> in the buffer allocator and some buffers were allocated > >> >> twice. > >> >> > >> >> 2. can you tweak your receiver.c to report some more info > >> >> on how often you get out of sequence packets, how much > >> >> out of sequence they are ? > >> >> Also it would be useful to report gaps on the increasing side > >> >> (i.e. new_seq != old_seq +1 ) > >> >> > >> >> 3. can you tweak bridge.c so that it writes into the packet > >> >> the netmap buffer indexes and slots on the rx and tx side, > >> >> so when you detect a sequence error we can figure out > >> >> where it is happening. > >> >> Ideally you could also add the sequence number detection > >> >> code in bridge.c so we can check whether the errors appear > >> >> on the input or output sides. > >> >> > >> >> cheers > >> >> luigi > >> >> > >> > > >> > >> > >> > >> -- > >> > -----------------------------------------+------------------------------- > >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > dell'Informazione > >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > >> TEL +39-050-2217533 . via Diotisalvi 2 > >> Mobile +39-338-6809875 . 56122 PISA (Italy) > >> > -----------------------------------------+------------------------------- > >> > > > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2217533 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > > From owner-freebsd-net@freebsd.org Wed Feb 3 08:12:14 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B07F0A9A462 for ; Wed, 3 Feb 2016 08:12:14 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lf0-x22a.google.com (mail-lf0-x22a.google.com [IPv6:2a00:1450:4010:c07::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 213211FA for ; Wed, 3 Feb 2016 08:12:14 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-lf0-x22a.google.com with SMTP id l143so8377961lfe.2 for ; Wed, 03 Feb 2016 00:12:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Lui4h5tBJvIOLVhqbdbYM37hZMtsWu29QlVpwJ516pE=; b=v9kWmapK4xjJBm1sMb1L1LQHGEMVjX+Mry0KypqFyhdaKBkWXNUMrPzHAQben9CABn 927yAx7Chl9mVvchLLvSStruIp9NXvdYD/jbdOV6yUgLEvKxD4tMIR4dEQBjxO8RMv9H 0tFvWjo4pBPJG9NFin5FHJJlAJ6Z/zUaqy71fk6F7KFAAoKWmyaX23XA4UC1hrRqawHe 5F89abBlLz1cfrRsjpKzogLwMWs/KjKMpFKCNkJ2Laof3AXITaE+drPRrLOa/iEyBglv bts7Snb4iqeymJatWEsLQBuCt6vApgIYVpdAcuDH0aU/vU4OHa2R3qwLzyrqiiTyqfTJ vT/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Lui4h5tBJvIOLVhqbdbYM37hZMtsWu29QlVpwJ516pE=; b=j/H1mWRLnZVSaHpEyL8X/bsRmlG7rNlnfh0GQb/tYkECZamFbTsmE6ZY+GutwH/J15 mHEIpCIXiMzqVjhvpY2UyotrQydgbvIfdU3obro2egiN7oqqp3cfwPszwurManiKuIL9 O5neXC8s9pxjAqtlm4kDSfnrd3xubFuLubSFhvTLi90LgL7A20YRQOPY30aPOamgqCLS HCKwKeZC37zOq8LRQNNQvebUBOPzHz8xOSSYQWl0vd+w4QtY6xCsZzQkKXbLvEI3VPuE yZARCLXLIslyxv5i44zXzHaxTXufqUH0+BxrbZ6vRQjRRQGzAJqrkKP9+/QLLFRPQfdf Hp3Q== X-Gm-Message-State: AG10YOT2W3AUWrOezBfXeYu8Fob/Tt228iGCfiYumsGt+Hz5CIJUB7jx9YM7U2nhYmMv3VTG9En/NcxH+QHKdA== MIME-Version: 1.0 X-Received: by 10.25.158.136 with SMTP id h130mr104929lfe.136.1454487132189; Wed, 03 Feb 2016 00:12:12 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.4.232 with HTTP; Wed, 3 Feb 2016 00:12:11 -0800 (PST) In-Reply-To: References: Date: Wed, 3 Feb 2016 09:12:11 +0100 X-Google-Sender-Auth: puLV5n36Yqzt4hIForH70dL2QYY Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Luigi Rizzo To: Xiaoye Sun Cc: Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 08:12:14 -0000 On Tue, Feb 2, 2016 at 10:48 PM, Xiaoye Sun wrote: > > > On Mon, Feb 1, 2016 at 11:34 PM, Luigi Rizzo wrote: >> >> On Tue, Feb 2, 2016 at 6:23 AM, Xiaoye Sun wrote: >> > Hi Luigi, >> > >> > I have to clarify about the *jumping issue* about the slot indexes. >> > In the bridge.c program, the slot index never jumps and it increases >> > sequentially. >> > In the receiver.c program, the udp packet seq jumps and I showed the >> > slot >> > index that each udp packet uses. So the slot index jumps together with >> > the >> > udp seq (at the receiver program only). >> >> So let me understand, is the "slot" some information written >> in the packet by bridge.c (referring to the rx or tx slot, >> I am not sure) and then read and printed by receiver.c >> (which gets the packet through recvfrom so there isn't >> really any slot index) ? >> > It works in the other way: > The bridge.c checks the seq numbers of the udp packets in netmap slots (in > nic rx ring) before the swap; then it records the seq number, slot > number(both rx and tx (tx indexes were not shown in the previous email since > they all look correct)) and buf_idx (rx and tx). The bridge.c does not > change anything in the buffer and it knows the slot and buf_idx that a > packet uses. Please refer to the added code in *process_rings* function > http://www.owlnet.rice.edu/~xs6/bridge.c > The receiver.c checks the seq numbers only and print out the seq numbers it > receive sequentially. > With these information, I manually match the seq number I got from > receiver.c and the seq number I got from bridge.c. So we know what is the > seq order the receiver sees and which slot a packet uses when bridge.c swaps > the buf_idxs. > >> Do you see any ordering inversion when the receiver >> gets packets through the NETMAP API (e.g. using bridge.c >> instead of receiver.c) ? >> > There is no ordering inversion seen by bridge.c (As I said in the previous > paragraph, the bridge.c checks the seq number and I did not see any order > inversion in THIS simple experiment (In my multicast protocol (mentioned in > the first email), there is ordering inversion. But let us solve the simple > bridge.c's problem first. I think they are two relatively independent > issues.)). Sorry there was a misunderstanding. I wanted you to check the following setup: [1: send.c] ->- [2: bridge.c] ->- [3: XYZ] where in XYZ you replace your receiver.c with some netmap-based receiver (it could be pkt-gen in rx mode, or possibly even another instance of bridge.c where you connect the output port to a vale switch so traffic is dropped), and then in XYZ print the content of the packets. >From your previous report we know that node 2: sees packets in order, and node 3: sees packets out of order. However, if the problem were due to bridge.c sending the old buffer and not the new one, you'd see not only reordering but also replication of packets. The fact that you see only the reordering in 3: makes me think that the problem is in that node, and it could be the network stack in 3: that does something strange. So if you can run something netmap based in 3: and make sure there is only one queue to read from, we could at least figure out what is going on. cheers luigi is that > >> >> Are you using native netmap drivers or the emulated mode ? >> You can check that by playing with the "admode" sysctl entry >> (or sysfs on linux) - try setting to 1 and 2 and see if >> the behaviour changes. >> >> dev.netmap.admode: 0 >> Controls the use of native or emulated adapter mode. >> 0 uses the best available option, >> 1 forces native and fails if not available, >> 2 forces emulated hence never fails. >> > I was using admode 0. I changed the admode to 1 and 2 using the command like > *echo 1 > /sys/module/netmap/parameters/admode* and restart the bridge > program. The behavior keeps the same. > >> >> cheers >> luigi >> >> > >> > There is really one ring (tx and rx) for NIC and one ring (tx and rx) >> > for >> > the host. >> > I also doubt that there might be multiple tx rings for the host. It >> > seems >> > like that bridge program swap packet to multiple host rings and the udp >> > recv >> > program drains packets from these rings. But this is not the case here. >> > >> > The bridge program prints a line like this >> > *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* >> > this is printed by the following line the original program >> > *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, >> > pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, >> > pb->first_rx_ring, >> > pb->req.nr_rx_rings);* >> > >> > I think this shows that there is really one NIC ring and one HOST ring. >> > >> > Is there another way to verify the number of ring that netmap has? >> > >> > Thanks! >> > Xiaoye >> > >> > On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo wrote: >> >> >> >> Hi, >> >> there must be some wrong with your setting because >> >> slot indexes must be sequential and in your case they >> >> are not (see the jump from 295 to 474 and then >> >> back from 485 to 296, and the numerous interleavings >> >> that you are seeing later). >> >> >> >> I have no idea of the cause but typically this pattern >> >> is what you see when there are multiple input rings and >> >> not just one. >> >> >> >> Cheers >> >> Luigi >> >> >> >> >> >> >> >> >> >> On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun >> >> wrote: >> >> > Hi Luigi, >> >> > >> >> > Thanks for the detailed advice. >> >> > >> >> > With more detailed experiments, actually I found that the udp >> >> > sender/receiver packet reorder issue *might* be irrelevant to the >> >> > original >> >> > issue I posted. However, I think we should solve the udp >> >> > sender/receiver >> >> > issue first. >> >> > I run the experiment with more detailed log. Here is my findings. >> >> > >> >> > 1. I am running a netmap version available since about Oct 13rd from >> >> > github >> >> > (https://github.com/luigirizzo/netmap). So I think this is not the >> >> > one >> >> > related to the buffer allocation issue. I tried to running the newest >> >> > version, however, that version causes problem when I exit the bridge >> >> > program >> >> > (something like kernel error which make the os crash). >> >> > >> >> > 2 & 3. I changed the receiver.c & bridge.c so that I can get more >> >> > information (more detailed log). >> >> > The reorder happens multiple times (about 10 times) within a second. >> >> > Here is >> >> > one example trace collected from the above two programs. (remembering >> >> > that >> >> > we have udp sender running on one machine; netmap bridge and udp >> >> > receiver >> >> > are running on another machine). >> >> > There is only one pair of rings each with 512 slots (511 slot usable) >> >> > on >> >> > the >> >> > receiver machine. >> >> > >> >> > =================== packet trace collected from receiver.c >> >> > =================== >> >> > ===== together with the slot and buf_idx of the corresponding netmap >> >> > ring >> >> > slots ====== >> >> > [seq] [slot] [buf_idx] >> >> > 8208 294 1833 >> >> > 8209 295 1834 >> >> > 8388 474 2013 >> >> > ... (packet received in order) >> >> > 8398 484 2023 >> >> > 8399 485 2024 >> >> > 8210 296 1835 >> >> > 8211 297 1836 >> >> > ... (packet received in order) >> >> > ... >> >> > 8222 308 1847 >> >> > 8400 486 2025 >> >> > 8223 309 1848 >> >> > 8401 487 2026 >> >> > 8224 310 1849 >> >> > 8402 488 2027 >> >> > 8225 311 1850 >> >> > 8403 489 2028 >> >> > 8226 312 1851 >> >> > 8404 450 2029 >> >> > 8227 313 1852 >> >> > 8228 314 1853 >> >> > =================================================================== >> >> > As we can see that the udp receiver got packet 8210 after it got >> >> > 8399, >> >> > which >> >> > is the first reorder. Then, the receiver got 8211 to 8222 >> >> > sequentially. >> >> > Then >> >> > it got packet from 8223-8227 and 8400-8404 interleaved. >> >> > >> >> > >> >> > ==================== event order seen by netmap bridge >> >> > ================== >> >> > get 8209 >> >> > poll called >> >> > get 8210 >> >> > ... >> >> > ... >> >> > get 8228 >> >> > poll called >> >> > get 8229 >> >> > ... >> >> > ... >> >> > get 8383 >> >> > poll called >> >> > get 8384 >> >> > ... >> >> > get 8387 >> >> > poll called >> >> > get 8388 >> >> > ... >> >> > get 8393 >> >> > poll called >> >> > get 8394 >> >> > ... >> >> > get 8399 >> >> > poll called >> >> > get 8400 >> >> > ... >> >> > get 8404 >> >> > poll called >> >> > get 8405 >> >> > =================================================================== >> >> > As we can see, from the event ordering see by the bridge.c, all the >> >> > packets >> >> > are receiver in order, which means the the reorder happens when the >> >> > bridge >> >> > code swap the buf_idx between the nic ring(slot) and the host >> >> > ring(slot). >> >> > The reordered seq usually right before or after the poll function >> >> > call. >> >> > >> >> > Best, >> >> > Xiaoye >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo >> >> > wrote: >> >> >> >> >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun >> >> >> wrote: >> >> >> > Hi Luigi, >> >> >> > >> >> >> > Thanks for your advice. >> >> >> > I forgot to mention that I use the command "ethtool -L eth1 >> >> >> > combined >> >> >> > 1" >> >> >> > to >> >> >> > set the number of rings of the nic to 1. The host also only has >> >> >> > one >> >> >> > ring. >> >> >> > I understand the situation where the first tx ring is full so the >> >> >> > bridge >> >> >> > will swap the packets to the second tx ring and then the host/nic >> >> >> > might >> >> >> > drain either rings. But this is not the case in the experiment. >> >> >> >> >> >> ok good to know that. >> >> >> >> >> >> So if we have ruled out multiqueue and iommu, let's look at >> >> >> the internal allocator and at bridge.c >> >> >> >> >> >> 1. are you running the most recent version of netmap ? >> >> >> Some older version (probably 1-2 years ago) had a bug >> >> >> in the buffer allocator and some buffers were allocated >> >> >> twice. >> >> >> >> >> >> 2. can you tweak your receiver.c to report some more info >> >> >> on how often you get out of sequence packets, how much >> >> >> out of sequence they are ? >> >> >> Also it would be useful to report gaps on the increasing side >> >> >> (i.e. new_seq != old_seq +1 ) >> >> >> >> >> >> 3. can you tweak bridge.c so that it writes into the packet >> >> >> the netmap buffer indexes and slots on the rx and tx side, >> >> >> so when you detect a sequence error we can figure out >> >> >> where it is happening. >> >> >> Ideally you could also add the sequence number detection >> >> >> code in bridge.c so we can check whether the errors appear >> >> >> on the input or output sides. >> >> >> >> >> >> cheers >> >> >> luigi >> >> >> >> >> > >> >> >> >> >> >> >> >> -- >> >> >> >> -----------------------------------------+------------------------------- >> >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >> >> dell'Informazione >> >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> >> TEL +39-050-2217533 . via Diotisalvi 2 >> >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> >> >> >> -----------------------------------------+------------------------------- >> >> >> > >> >> >> >> -- >> -----------------------------------------+------------------------------- >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2217533 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------- >> > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@freebsd.org Wed Feb 3 08:42:38 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9324CA9AD4A for ; Wed, 3 Feb 2016 08:42:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 835D21C3 for ; Wed, 3 Feb 2016 08:42:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u138gc1g072126 for ; Wed, 3 Feb 2016 08:42:38 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206876] ifconfig(8): Inconsistent behavior when creating and giving custom name to interface at the same time Date: Wed, 03 Feb 2016 08:42:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: easy, needs-patch, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? X-Bugzilla-Changed-Fields: assigned_to keywords flagtypes.name cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 08:42:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206876 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Keywords| |easy, needs-patch, needs-qa Flags| |mfc-stable10? CC| |koobs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Feb 3 13:38:43 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69546A993DF; Wed, 3 Feb 2016 13:38:43 +0000 (UTC) (envelope-from wolfgang.meyer@hob.de) Received: from hobex19.hob.de (hobex29.hob.de [212.185.199.32]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D086B13DD; Wed, 3 Feb 2016 13:38:38 +0000 (UTC) (envelope-from wolfgang.meyer@hob.de) Received: from HOBEX22.hob.de (172.22.1.4) by hobex29.hob.de (172.25.1.32) with Microsoft SMTP Server (TLS) id 14.2.347.0; Wed, 3 Feb 2016 14:37:10 +0100 Received: from HOBEX11.hob.de ([fe80::b99f:1c54:7122:49b4]) by HOBEX22.hob.de ([::1]) with mapi id 14.02.0387.000; Wed, 3 Feb 2016 14:37:22 +0100 From: "Meyer, Wolfgang" To: "'freebsd-net@FreeBSD.org'" CC: "'freebsd-performance@FreeBSD.org'" Subject: ixgbe: Network performance tuning (#TCP connections) Thread-Topic: ixgbe: Network performance tuning (#TCP connections) Thread-Index: AdFefq3DO+J8ScSLR4uotZFUcfP67w== Date: Wed, 3 Feb 2016 13:37:21 +0000 Message-ID: Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.24.71.140] old-x-esetresult: clean, is OK old-x-esetid: 46BD7B39450636371DFD21 x-esetresult: clean, is OK x-esetid: 46BD7B39450636371DFD21 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 13:38:43 -0000 Hello, we are evaluating network performance on a DELL-Server (PowerEdge R930 with= 4 Sockets, hw.model: Intel(R) Xeon(R) CPU E7-8891 v3 @ 2.80GHz) with 10 Gb= E-Cards. We use programs that on server side accepts connections on a IP-ad= dress+port from the client side and after establishing the connection data = is sent in turns between server and client in a predefined pattern (server = side sends more data than client side) with sleeps in between the send phas= es. The test set-up is chosen in such way that every client process initiat= es 500 connections handled in threads and on the server side each process r= epresenting an IP/Port pair also handles 500 connections in threads. The number of connections is then increased and the overall network througp= ut is observed using nload. On FreeBSD (on server side) roughly at 50,000 c= onnections errors begin to occur and the overall throughput won't increase = further with more connections. With Linux on the server side it is possible= to establish more than 120,000 connections and at 50,000 connections the o= verall throughput ist double that of FreeBSD with the same sending pattern.= Furthermore system load on FreeBSD is much higher with 50 % system usage o= n each core and 80 % interrupt usage on the 8 cores handling the interrupt = queues for the NIC. In comparison Linux has <10 % system usage, <10 % user = usage and about 15 % interrupt usage on the 16 cores handling the network i= nterrupts for 50,000 connections. Varying the numbers for the NIC interrupt queues won't change the performan= ce (rather worsens the situation). Disabling Hyperthreading (utilising 40 c= ores) degrades the performance. Increasing MAXCPU to utilise all 80 cores w= on't improve compared to 64 cores, atkbd and uart had to be disabled to avo= id kernel panics with increased MAXCPU (thanks to Andre Oppermann for inves= tigating this). Initiallly the tests were made on 10.2 Release, later I swi= tched to 10 Stable (later with ixgbe driver version 3.1.0) but that didn't = change the numbers. Some sysctl configurables were modified along the network performance guide= lines found on the net (e.g. https://calomel.org/freebsd_network_tuning.htm= l, https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html, ht= tps://pleiades.ucsc.edu/hyades/FreeBSD_Network_Tuning) but most of them did= n't have any measuarable impact. Final sysctl.conf and loader.conf settings= see below. Actually the only tunables that provided any improvement were i= dentified to be hw.ix.txd, and hw.ix.rxd that were reduced (!) to the minim= um value of 64 and hw.ix.tx_process_limit and hw.ix.rx_process_limit that w= ere set to -1. Any ideas what tunables might be changed to get a higher number of TCP conn= ections (it's not a question of the overall throughput as changing the send= ing pattern allows me to fully utilise the 10Gb bandwidth)? How can I deter= mine where the kernel is spending its time that causes the high CPU load? A= ny pointers are highly appreciated, I can't believe that there is such a bl= atant difference in network performance compared to Linux. Regards, Wolfgang : cc_htcp_load=3D"YES" hw.ix.txd=3D"64" hw.ix.rxd=3D"64" hw.ix.tx_process_limit=3D"-1" hw.ix.rx_process_limit=3D"-1" hw.ix.num_queues=3D"8" #hw.ix.enable_aim=3D"0" #hw.ix.max_interrupt_rate=3D"31250" #net.isr.maxthreads=3D"16" : kern.ipc.soacceptqueue=3D1024 kern.ipc.maxsockbuf=3D16777216 net.inet.tcp.sendbuf_max=3D16777216 net.inet.tcp.recvbuf_max=3D16777216 net.inet.tcp.tso=3D0 net.inet.tcp.mssdflt=3D1460 net.inet.tcp.minmss=3D1300 net.inet.tcp.nolocaltimewait=3D1 net.inet.tcp.syncache.rexmtlimit=3D0 #net.inet.tcp.syncookies=3D0 net.inet.tcp.drop_synfin=3D1 net.inet.tcp.fast_finwait2_recycle=3D1 net.inet.tcp.icmp_may_rst=3D0 net.inet.tcp.msl=3D5000 net.inet.tcp.path_mtu_discovery=3D0 net.inet.tcp.blackhole=3D1 net.inet.udp.blackhole=3D1 net.inet.tcp.cc.algorithm=3Dhtcp net.inet.tcp.cc.htcp.adaptive_backoff=3D1 net.inet.tcp.cc.htcp.rtt_scaling=3D1 net.inet.ip.forwarding=3D1 net.inet.ip.fastforwarding=3D1 net.inet.ip.rtexpire=3D1 net.inet.ip.rtminexpire=3D1 ________________________________ Follow HOB: - HOB: http://www.hob.de/redirect/hob.html - Xing: http://www.hob.de/redirect/xing.html - LinkedIn: http://www.hob.de/redirect/linkedin.html - HOBLink Mobile: http://www.hob.de/redirect/hoblinkmobile.html - Facebook: http://www.hob.de/redirect/facebook.html - Twitter: http://www.hob.de/redirect/twitter.html - YouTube: http://www.hob.de/redirect/youtube.html - E-Mail: http://www.hob.de/redirect/mail.html HOB GmbH & Co. KG Schwadermuehlstr. 3 D-90556 Cadolzburg Geschaeftsfuehrung: Klaus Brandstaetter, Zoran Adamovic AG Fuerth, HRA 5180 Steuer-Nr. 218/163/00107 USt-ID-Nr. DE 132747002 Komplementaerin HOB electronic Beteiligungs GmbH AG Fuerth, HRB 3416 From owner-freebsd-net@freebsd.org Wed Feb 3 16:02:59 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D27FA9AAF1 for ; Wed, 3 Feb 2016 16:02:59 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 65224120C for ; Wed, 3 Feb 2016 16:02:59 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 61B3D10660C; Wed, 3 Feb 2016 16:02:59 +0000 (UTC) Date: Wed, 3 Feb 2016 16:02:59 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5175+325+b48f3d05eeb00f73@reviews.freebsd.org Subject: [Differential] [Request, 19 lines] D5175: hyperv/hn: Add an option to always do transmission scheduling Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D5175: hyperv/hn: Add an option to always do transmission scheduling X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Index: OGFmNjgzYWFmZmVhMjQ0MTIwYzQ1OWJjZDg2 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_19267c547ac6fd0909d37cf4c2a7da02" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 16:02:59 -0000 --b1_19267c547ac6fd0909d37cf4c2a7da02 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com created this revision. sepherosa_gmail.com added reviewers: network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com. sepherosa_gmail.com added subscribers: freebsd-net-list, freebsd-virtualization-list. REVISION SUMMARY It is off by default. This eases more experiment on hn(4). REVISION DETAIL https://reviews.freebsd.org/D5175 AFFECTED FILES sys/dev/hyperv/netvsc/hv_net_vsc.h sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -534,6 +534,10 @@ SYSCTL_ADD_INT(ctx, child, OID_AUTO, "direct_tx_size", CTLFLAG_RW, &sc->hn_direct_tx_size, 0, "Size of the packet for direct transmission"); + SYSCTL_ADD_INT(ctx, child, OID_AUTO, "sched_tx", + CTLFLAG_RW, &sc->hn_sched_tx, 0, + "Always schedule transmission " + "instead of doing direct transmission"); if (unit == 0) { struct sysctl_ctx_list *dc_ctx; @@ -1602,26 +1606,31 @@ static void hn_start(struct ifnet *ifp) { - hn_softc_t *sc; + struct hn_softc *sc = ifp->if_softc; + + if (sc->hn_sched_tx) + goto do_sched; - sc = ifp->if_softc; if (NV_TRYLOCK(sc)) { int sched; sched = hn_start_locked(ifp, sc->hn_direct_tx_size); NV_UNLOCK(sc); if (!sched) return; } +do_sched: taskqueue_enqueue_fast(sc->hn_tx_taskq, &sc->hn_start_task); } static void hn_start_txeof(struct ifnet *ifp) { - hn_softc_t *sc; + struct hn_softc *sc = ifp->if_softc; + + if (sc->hn_sched_tx) + goto do_sched; - sc = ifp->if_softc; if (NV_TRYLOCK(sc)) { int sched; @@ -1633,6 +1642,7 @@ &sc->hn_start_task); } } else { +do_sched: /* * Release the OACTIVE earlier, with the hope, that * others could catch up. The task will clear the diff --git a/sys/dev/hyperv/netvsc/hv_net_vsc.h b/sys/dev/hyperv/netvsc/hv_net_vsc.h --- a/sys/dev/hyperv/netvsc/hv_net_vsc.h +++ b/sys/dev/hyperv/netvsc/hv_net_vsc.h @@ -1023,6 +1023,7 @@ int hn_txdesc_avail; int hn_txeof; + int hn_sched_tx; int hn_direct_tx_size; struct taskqueue *hn_tx_taskq; struct task hn_start_task; EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com Cc: freebsd-virtualization-list, freebsd-net-list --b1_19267c547ac6fd0909d37cf4c2a7da02 Content-Type: text/x-patch; charset=utf-8; name="D5175.12968.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5175.12968.patch" ZGlmZiAtLWdpdCBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2c2NfZHJ2X2ZyZWVic2Qu YyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2c2NfZHJ2X2ZyZWVic2QuYwotLS0gYS9z eXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMKKysrIGIvc3lzL2Rl di9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJlZWJzZC5jCkBAIC01MzQsNiArNTM0LDEw IEBACiAJU1lTQ1RMX0FERF9JTlQoY3R4LCBjaGlsZCwgT0lEX0FVVE8sICJkaXJlY3RfdHhfc2l6 ZSIsCiAJICAgIENUTEZMQUdfUlcsICZzYy0+aG5fZGlyZWN0X3R4X3NpemUsIDAsCiAJICAgICJT aXplIG9mIHRoZSBwYWNrZXQgZm9yIGRpcmVjdCB0cmFuc21pc3Npb24iKTsKKwlTWVNDVExfQURE X0lOVChjdHgsIGNoaWxkLCBPSURfQVVUTywgInNjaGVkX3R4IiwKKwkgICAgQ1RMRkxBR19SVywg JnNjLT5obl9zY2hlZF90eCwgMCwKKwkgICAgIkFsd2F5cyBzY2hlZHVsZSB0cmFuc21pc3Npb24g IgorCSAgICAiaW5zdGVhZCBvZiBkb2luZyBkaXJlY3QgdHJhbnNtaXNzaW9uIik7CiAKIAlpZiAo dW5pdCA9PSAwKSB7CiAJCXN0cnVjdCBzeXNjdGxfY3R4X2xpc3QgKmRjX2N0eDsKQEAgLTE2MDIs MjYgKzE2MDYsMzEgQEAKIHN0YXRpYyB2b2lkCiBobl9zdGFydChzdHJ1Y3QgaWZuZXQgKmlmcCkK IHsKLQlobl9zb2Z0Y190ICpzYzsKKwlzdHJ1Y3QgaG5fc29mdGMgKnNjID0gaWZwLT5pZl9zb2Z0 YzsKKworCWlmIChzYy0+aG5fc2NoZWRfdHgpCisJCWdvdG8gZG9fc2NoZWQ7CiAKLQlzYyA9IGlm cC0+aWZfc29mdGM7CiAJaWYgKE5WX1RSWUxPQ0soc2MpKSB7CiAJCWludCBzY2hlZDsKIAogCQlz Y2hlZCA9IGhuX3N0YXJ0X2xvY2tlZChpZnAsIHNjLT5obl9kaXJlY3RfdHhfc2l6ZSk7CiAJCU5W X1VOTE9DSyhzYyk7CiAJCWlmICghc2NoZWQpCiAJCQlyZXR1cm47CiAJfQorZG9fc2NoZWQ6CiAJ dGFza3F1ZXVlX2VucXVldWVfZmFzdChzYy0+aG5fdHhfdGFza3EsICZzYy0+aG5fc3RhcnRfdGFz ayk7CiB9CiAKIHN0YXRpYyB2b2lkCiBobl9zdGFydF90eGVvZihzdHJ1Y3QgaWZuZXQgKmlmcCkK IHsKLQlobl9zb2Z0Y190ICpzYzsKKwlzdHJ1Y3QgaG5fc29mdGMgKnNjID0gaWZwLT5pZl9zb2Z0 YzsKKworCWlmIChzYy0+aG5fc2NoZWRfdHgpCisJCWdvdG8gZG9fc2NoZWQ7CiAKLQlzYyA9IGlm cC0+aWZfc29mdGM7CiAJaWYgKE5WX1RSWUxPQ0soc2MpKSB7CiAJCWludCBzY2hlZDsKIApAQCAt MTYzMyw2ICsxNjQyLDcgQEAKIAkJCSAgICAmc2MtPmhuX3N0YXJ0X3Rhc2spOwogCQl9CiAJfSBl bHNlIHsKK2RvX3NjaGVkOgogCQkvKgogCQkgKiBSZWxlYXNlIHRoZSBPQUNUSVZFIGVhcmxpZXIs IHdpdGggdGhlIGhvcGUsIHRoYXQKIAkJICogb3RoZXJzIGNvdWxkIGNhdGNoIHVwLiAgVGhlIHRh c2sgd2lsbCBjbGVhciB0aGUKZGlmZiAtLWdpdCBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9u ZXRfdnNjLmggYi9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5oCi0tLSBhL3N5cy9k ZXYvaHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmgKKysrIGIvc3lzL2Rldi9oeXBlcnYvbmV0dnNj L2h2X25ldF92c2MuaApAQCAtMTAyMyw2ICsxMDIzLDcgQEAKIAlpbnQJCWhuX3R4ZGVzY19hdmFp bDsKIAlpbnQJCWhuX3R4ZW9mOwogCisJaW50CQlobl9zY2hlZF90eDsKIAlpbnQJCWhuX2RpcmVj dF90eF9zaXplOwogCXN0cnVjdCB0YXNrcXVldWUgKmhuX3R4X3Rhc2txOwogCXN0cnVjdCB0YXNr CWhuX3N0YXJ0X3Rhc2s7Cgo= --b1_19267c547ac6fd0909d37cf4c2a7da02-- From owner-freebsd-net@freebsd.org Wed Feb 3 17:19:59 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63EC4A99AB8 for ; Wed, 3 Feb 2016 17:19:59 +0000 (UTC) (envelope-from wolfgang.meyer@hob.de) Received: from hobex19.hob.de (hobex19.hob.de [212.185.199.31]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hobex19", Issuer "hobex19" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CA27EBEE; Wed, 3 Feb 2016 17:19:58 +0000 (UTC) (envelope-from wolfgang.meyer@hob.de) Received: from HOBEX22.hob.de (172.22.1.4) by hobex19.hob.de (172.25.1.31) with Microsoft SMTP Server (TLS) id 14.2.347.0; Wed, 3 Feb 2016 18:18:24 +0100 Received: from HOBEX11.hob.de ([fe80::b99f:1c54:7122:49b4]) by HOBEX22.hob.de ([::1]) with mapi id 14.02.0387.000; Wed, 3 Feb 2016 18:18:40 +0100 From: "Meyer, Wolfgang" To: 'Allan Jude' CC: "'freebsd-net@FreeBSD.org'" Subject: RE: ixgbe: Network performance tuning (#TCP connections) Thread-Topic: ixgbe: Network performance tuning (#TCP connections) Thread-Index: AdFefq3DO+J8ScSLR4uotZFUcfP67wAE1XmAAAUBoXA= Date: Wed, 3 Feb 2016 17:18:40 +0000 Message-ID: References: <56B2216B.6080404@freebsd.org> In-Reply-To: <56B2216B.6080404@freebsd.org> Accept-Language: de-DE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.24.71.140] old-x-esetresult: clean, is OK old-x-esetid: 46BD7B39450636371DFD24 x-esetresult: clean, is OK x-esetid: 46BD7B39450636371DFD24 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 17:19:59 -0000 > -----Original Message----- > From: Allan Jude [mailto:allanjude@freebsd.org] > Sent: Mittwoch, 3. Februar 2016 16:49 > To: Meyer, Wolfgang > Subject: Re: ixgbe: Network performance tuning (#TCP connections) > > On 2016-02-03 08:37, Meyer, Wolfgang wrote: > > Hello, > > > > we are evaluating network performance on a DELL-Server (PowerEdge > R930 with 4 Sockets, hw.model: Intel(R) Xeon(R) CPU E7-8891 v3 @ 2.80GHz) > with 10 GbE-Cards. We use programs that on server side accepts connection= s > on a IP-address+port from the client side and after establishing the > connection data is sent in turns between server and client in a predefine= d > pattern (server side sends more data than client side) with sleeps in > between the send phases. The test set-up is chosen in such way that every > client process initiates 500 connections handled in threads and on the se= rver > side each process representing an IP/Port pair also handles 500 connectio= ns > in threads. > > > > The number of connections is then increased and the overall network > througput is observed using nload. On FreeBSD (on server side) roughly at > 50,000 connections errors begin to occur and the overall throughput won't > increase further with more connections. With Linux on the server side it = is > possible to establish more than 120,000 connections and at 50,000 > connections the overall throughput ist double that of FreeBSD with the sa= me > sending pattern. Furthermore system load on FreeBSD is much higher with > 50 % system usage on each core and 80 % interrupt usage on the 8 cores > handling the interrupt queues for the NIC. In comparison Linux has <10 % > system usage, <10 % user usage and about 15 % interrupt usage on the 16 > cores handling the network interrupts for 50,000 connections. > > > > Varying the numbers for the NIC interrupt queues won't change the > performance (rather worsens the situation). Disabling Hyperthreading > (utilising 40 cores) degrades the performance. Increasing MAXCPU to utili= se > all 80 cores won't improve compared to 64 cores, atkbd and uart had to be > disabled to avoid kernel panics with increased MAXCPU (thanks to Andre > Oppermann for investigating this). Initiallly the tests were made on 10.2 > Release, later I switched to 10 Stable (later with ixgbe driver version 3= .1.0) > but that didn't change the numbers. > > > > Some sysctl configurables were modified along the network performance > guidelines found on the net (e.g. > https://calomel.org/freebsd_network_tuning.html, > https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html, > https://pleiades.ucsc.edu/hyades/FreeBSD_Network_Tuning) but most of > them didn't have any measuarable impact. Final sysctl.conf and loader.con= f > settings see below. Actually the only tunables that provided any > improvement were identified to be hw.ix.txd, and hw.ix.rxd that were > reduced (!) to the minimum value of 64 and hw.ix.tx_process_limit and > hw.ix.rx_process_limit that were set to -1. > > > > Any ideas what tunables might be changed to get a higher number of TCP > connections (it's not a question of the overall throughput as changing th= e > sending pattern allows me to fully utilise the 10Gb bandwidth)? How can I > determine where the kernel is spending its time that causes the high CPU > load? Any pointers are highly appreciated, I can't believe that there is = such a > blatant difference in network performance compared to Linux. > > > > Regards, > > Wolfgang > > > > : > > cc_htcp_load=3D"YES" > > hw.ix.txd=3D"64" > > hw.ix.rxd=3D"64" > > hw.ix.tx_process_limit=3D"-1" > > hw.ix.rx_process_limit=3D"-1" > > hw.ix.num_queues=3D"8" > > #hw.ix.enable_aim=3D"0" > > #hw.ix.max_interrupt_rate=3D"31250" > > > > #net.isr.maxthreads=3D"16" > > > > : > > kern.ipc.soacceptqueue=3D1024 > > > > kern.ipc.maxsockbuf=3D16777216 > > net.inet.tcp.sendbuf_max=3D16777216 > > net.inet.tcp.recvbuf_max=3D16777216 > > > > net.inet.tcp.tso=3D0 > > net.inet.tcp.mssdflt=3D1460 > > net.inet.tcp.minmss=3D1300 > > > > net.inet.tcp.nolocaltimewait=3D1 > > net.inet.tcp.syncache.rexmtlimit=3D0 > > > > #net.inet.tcp.syncookies=3D0 > > net.inet.tcp.drop_synfin=3D1 > > net.inet.tcp.fast_finwait2_recycle=3D1 > > > > net.inet.tcp.icmp_may_rst=3D0 > > net.inet.tcp.msl=3D5000 > > net.inet.tcp.path_mtu_discovery=3D0 > > net.inet.tcp.blackhole=3D1 > > net.inet.udp.blackhole=3D1 > > > > net.inet.tcp.cc.algorithm=3Dhtcp > > net.inet.tcp.cc.htcp.adaptive_backoff=3D1 > > net.inet.tcp.cc.htcp.rtt_scaling=3D1 > > > > net.inet.ip.forwarding=3D1 > > net.inet.ip.fastforwarding=3D1 > > net.inet.ip.rtexpire=3D1 > > net.inet.ip.rtminexpire=3D1 > > > > > > > > > > ________________________________ > > > > Follow HOB: > > > > - HOB: http://www.hob.de/redirect/hob.html > > - Xing: http://www.hob.de/redirect/xing.html > > - LinkedIn: http://www.hob.de/redirect/linkedin.html > > - HOBLink Mobile: http://www.hob.de/redirect/hoblinkmobile.html > > - Facebook: http://www.hob.de/redirect/facebook.html > > - Twitter: http://www.hob.de/redirect/twitter.html > > - YouTube: http://www.hob.de/redirect/youtube.html > > - E-Mail: http://www.hob.de/redirect/mail.html > > > > > > HOB GmbH & Co. KG > > Schwadermuehlstr. 3 > > D-90556 Cadolzburg > > > > Geschaeftsfuehrung: Klaus Brandstaetter, Zoran Adamovic > > > > AG Fuerth, HRA 5180 > > Steuer-Nr. 218/163/00107 > > USt-ID-Nr. DE 132747002 > > > > Komplementaerin HOB electronic Beteiligungs GmbH AG Fuerth, HRB 3416 > > _______________________________________________ > > freebsd-performance@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-performance > > To unsubscribe, send any mail to "freebsd-performance- > unsubscribe@freebsd.org" > > > > Is there a reason you are disabling TSO? I would expect that to have a > significant impact on throughput. It is usually only disabled on a router= , to > reduce latency on forwarded packets. > > net.inet.tcp.tso=3D1 > > Also, please provide the list with the output of 'ifconfig ix0' > > -- > Allan Jude This sysctl was one of the knobs I found in the internet that should be con= sidered to improve network performance. Actually it won't change the number= s either way (just tested again). ifconfig ix1: ix1: flags=3D8843 metric 0 mtu 9000 options=3D8407bb ether a0:36:9f:7c:9e:52 inet 172.31.1.1 netmask 0xffffff00 broadcast 172.31.1.255 inet 172.31.2.1 netmask 0xffffff00 broadcast 172.31.2.255 inet 172.31.3.1 netmask 0xffffff00 broadcast 172.31.3.255 inet 172.31.4.1 netmask 0xffffff00 broadcast 172.31.4.255 nd6 options=3D29 media: Ethernet autoselect (10Gbase-T ) status: active Regards, Wolfgang Meyer ________________________________ Follow HOB: - HOB: http://www.hob.de/redirect/hob.html - Xing: http://www.hob.de/redirect/xing.html - LinkedIn: http://www.hob.de/redirect/linkedin.html - HOBLink Mobile: http://www.hob.de/redirect/hoblinkmobile.html - Facebook: http://www.hob.de/redirect/facebook.html - Twitter: http://www.hob.de/redirect/twitter.html - YouTube: http://www.hob.de/redirect/youtube.html - E-Mail: http://www.hob.de/redirect/mail.html HOB GmbH & Co. KG Schwadermuehlstr. 3 D-90556 Cadolzburg Geschaeftsfuehrung: Klaus Brandstaetter, Zoran Adamovic AG Fuerth, HRA 5180 Steuer-Nr. 218/163/00107 USt-ID-Nr. DE 132747002 Komplementaerin HOB electronic Beteiligungs GmbH AG Fuerth, HRB 3416 From owner-freebsd-net@freebsd.org Wed Feb 3 21:09:59 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 973C1A9A8A4 for ; Wed, 3 Feb 2016 21:09:59 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 845C91D53 for ; Wed, 3 Feb 2016 21:09:59 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 8457D106DB8; Wed, 3 Feb 2016 21:09:59 +0000 (UTC) Date: Wed, 3 Feb 2016 21:09:59 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5175+325+b48f3d05eeb00f73@reviews.freebsd.org Subject: [Differential] [Accepted] D5175: hyperv/hn: Add an option to always do transmission scheduling Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , Thread-Topic: D5175: hyperv/hn: Add an option to always do transmission scheduling X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: OGFmNjgzYWFmZmVhMjQ0MTIwYzQ1OWJjZDg2IFaybKc= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-Mailman-Approved-At: Wed, 03 Feb 2016 21:32:39 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 21:09:59 -0000 adrian accepted this revision. adrian added a comment. This revision has a positive review. Fine by me! REVISION DETAIL https://reviews.freebsd.org/D5175 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Wed Feb 3 21:34:05 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2AE12A99579; Wed, 3 Feb 2016 21:34:05 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ig0-x229.google.com (mail-ig0-x229.google.com [IPv6:2607:f8b0:4001:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E9A83D25; Wed, 3 Feb 2016 21:34:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-ig0-x229.google.com with SMTP id rs20so13533920igc.0; Wed, 03 Feb 2016 13:34:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=iIS/1D5ntVZgOmhLD0yJjosaDNv7pIm8Sp8h4rmub7I=; b=0yX5rpoUOujET51YvqUQ7qj7GMFadteSaLa3mP5mNYbMtLkRU45IvOzqy5ZyDszJ6f 2tRRSVgZHQuofdLW+pSVkb7tiBOLGI6zZQAPg9CnLuiXHIuQq6u99BtD5n0QEyqs+c4E TbX6hRwafILEN9JhHoZtIw6tHMl6KM+pGBiUrS402t+FRg1x42YnZrQU0iPgprxGTVUi iAdkqifSjHN76FD5lMBduK6TJzClc7U069aSjPAp6kIkPb+3zqwMjZcv80MFfXolMkb1 /veHb5mcJieH2222VRedc2uV6c6SjFd2Ytm9ErsxJJO0rItnLg1lnRALvUkoI9Rbqr+z A71w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=iIS/1D5ntVZgOmhLD0yJjosaDNv7pIm8Sp8h4rmub7I=; b=U6wQm/DRGfW1S8CS1UY/ar00wH8lOWEImY5vaWA/bLS57uxipfxB20j23vSZYXCNnv +PrQSBzK+lkmAyjn3al4Ft3kV6eSIpGuaRd+cqz8ED9bbWtLtuAMw5y+TlXlJyY7Oeq7 Xj2omjhc0P5fpzkGpukaTb4Y1ZDeSRQA2SFzonPfq6bOMFRv8xgScQs4az57L2RckP7Z tdcKp6LfML/6rBSBodQuPQF6JVAnCaAk3rO5L53sqNMZVmetR+HmUMimvDc6ig6oY/GO ME1O04FBdNtuEXBx1ZpIV2VWoioSZoNnf7cqN1V8nPN7QoKxyIapebuKie+TL3W8Htle rV5Q== X-Gm-Message-State: AG10YOTgbHq9yimE33TkUWDypNSuPtp8RJlSzuc3n1wUIm7e57Qqb0fPuxjnzYkh7605Yn1TKwZ9WhfmKwOKLQ== MIME-Version: 1.0 X-Received: by 10.50.93.36 with SMTP id cr4mr5566578igb.22.1454535243969; Wed, 03 Feb 2016 13:34:03 -0800 (PST) Received: by 10.36.14.19 with HTTP; Wed, 3 Feb 2016 13:34:03 -0800 (PST) In-Reply-To: References: Date: Wed, 3 Feb 2016 13:34:03 -0800 Message-ID: Subject: Re: ixgbe: Network performance tuning (#TCP connections) From: Adrian Chadd To: "Meyer, Wolfgang" Cc: "freebsd-net@FreeBSD.org" , "freebsd-performance@FreeBSD.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 21:34:05 -0000 hi, can you share your testing program source? -a On 3 February 2016 at 05:37, Meyer, Wolfgang wrote: > Hello, > > we are evaluating network performance on a DELL-Server (PowerEdge R930 wi= th 4 Sockets, hw.model: Intel(R) Xeon(R) CPU E7-8891 v3 @ 2.80GHz) with 10 = GbE-Cards. We use programs that on server side accepts connections on a IP-= address+port from the client side and after establishing the connection dat= a is sent in turns between server and client in a predefined pattern (serve= r side sends more data than client side) with sleeps in between the send ph= ases. The test set-up is chosen in such way that every client process initi= ates 500 connections handled in threads and on the server side each process= representing an IP/Port pair also handles 500 connections in threads. > > The number of connections is then increased and the overall network throu= gput is observed using nload. On FreeBSD (on server side) roughly at 50,000= connections errors begin to occur and the overall throughput won't increas= e further with more connections. With Linux on the server side it is possib= le to establish more than 120,000 connections and at 50,000 connections the= overall throughput ist double that of FreeBSD with the same sending patter= n. Furthermore system load on FreeBSD is much higher with 50 % system usage= on each core and 80 % interrupt usage on the 8 cores handling the interrup= t queues for the NIC. In comparison Linux has <10 % system usage, <10 % use= r usage and about 15 % interrupt usage on the 16 cores handling the network= interrupts for 50,000 connections. > > Varying the numbers for the NIC interrupt queues won't change the perform= ance (rather worsens the situation). Disabling Hyperthreading (utilising 40= cores) degrades the performance. Increasing MAXCPU to utilise all 80 cores= won't improve compared to 64 cores, atkbd and uart had to be disabled to a= void kernel panics with increased MAXCPU (thanks to Andre Oppermann for inv= estigating this). Initiallly the tests were made on 10.2 Release, later I s= witched to 10 Stable (later with ixgbe driver version 3.1.0) but that didn'= t change the numbers. > > Some sysctl configurables were modified along the network performance gui= delines found on the net (e.g. https://calomel.org/freebsd_network_tuning.h= tml, https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html, = https://pleiades.ucsc.edu/hyades/FreeBSD_Network_Tuning) but most of them d= idn't have any measuarable impact. Final sysctl.conf and loader.conf settin= gs see below. Actually the only tunables that provided any improvement were= identified to be hw.ix.txd, and hw.ix.rxd that were reduced (!) to the min= imum value of 64 and hw.ix.tx_process_limit and hw.ix.rx_process_limit that= were set to -1. > > Any ideas what tunables might be changed to get a higher number of TCP co= nnections (it's not a question of the overall throughput as changing the se= nding pattern allows me to fully utilise the 10Gb bandwidth)? How can I det= ermine where the kernel is spending its time that causes the high CPU load?= Any pointers are highly appreciated, I can't believe that there is such a = blatant difference in network performance compared to Linux. > > Regards, > Wolfgang > > : > cc_htcp_load=3D"YES" > hw.ix.txd=3D"64" > hw.ix.rxd=3D"64" > hw.ix.tx_process_limit=3D"-1" > hw.ix.rx_process_limit=3D"-1" > hw.ix.num_queues=3D"8" > #hw.ix.enable_aim=3D"0" > #hw.ix.max_interrupt_rate=3D"31250" > > #net.isr.maxthreads=3D"16" > > : > kern.ipc.soacceptqueue=3D1024 > > kern.ipc.maxsockbuf=3D16777216 > net.inet.tcp.sendbuf_max=3D16777216 > net.inet.tcp.recvbuf_max=3D16777216 > > net.inet.tcp.tso=3D0 > net.inet.tcp.mssdflt=3D1460 > net.inet.tcp.minmss=3D1300 > > net.inet.tcp.nolocaltimewait=3D1 > net.inet.tcp.syncache.rexmtlimit=3D0 > > #net.inet.tcp.syncookies=3D0 > net.inet.tcp.drop_synfin=3D1 > net.inet.tcp.fast_finwait2_recycle=3D1 > > net.inet.tcp.icmp_may_rst=3D0 > net.inet.tcp.msl=3D5000 > net.inet.tcp.path_mtu_discovery=3D0 > net.inet.tcp.blackhole=3D1 > net.inet.udp.blackhole=3D1 > > net.inet.tcp.cc.algorithm=3Dhtcp > net.inet.tcp.cc.htcp.adaptive_backoff=3D1 > net.inet.tcp.cc.htcp.rtt_scaling=3D1 > > net.inet.ip.forwarding=3D1 > net.inet.ip.fastforwarding=3D1 > net.inet.ip.rtexpire=3D1 > net.inet.ip.rtminexpire=3D1 > > > > > ________________________________ > > Follow HOB: > > - HOB: http://www.hob.de/redirect/hob.html > - Xing: http://www.hob.de/redirect/xing.html > - LinkedIn: http://www.hob.de/redirect/linkedin.html > - HOBLink Mobile: http://www.hob.de/redirect/hoblinkmobile.html > - Facebook: http://www.hob.de/redirect/facebook.html > - Twitter: http://www.hob.de/redirect/twitter.html > - YouTube: http://www.hob.de/redirect/youtube.html > - E-Mail: http://www.hob.de/redirect/mail.html > > > HOB GmbH & Co. KG > Schwadermuehlstr. 3 > D-90556 Cadolzburg > > Geschaeftsfuehrung: Klaus Brandstaetter, Zoran Adamovic > > AG Fuerth, HRA 5180 > Steuer-Nr. 218/163/00107 > USt-ID-Nr. DE 132747002 > > Komplementaerin HOB electronic Beteiligungs GmbH > AG Fuerth, HRB 3416 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Wed Feb 3 23:15:22 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11954A9B939; Wed, 3 Feb 2016 23:15:22 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mx1.shrew.net (mx1.shrew.net [38.97.5.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DAFAA1C03; Wed, 3 Feb 2016 23:15:21 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx1.shrew.net (8.14.7/8.14.7) with ESMTP id u13MsL71064482; Wed, 3 Feb 2016 16:54:21 -0600 (CST) (envelope-from mgrooms@shrew.net) Received: from [10.16.48.132] (67-198-50-4.static.grandenetworks.net [67.198.50.4]) by mail.shrew.net (Postfix) with ESMTPSA id C270218C848; Wed, 3 Feb 2016 16:54:15 -0600 (CST) Message-ID: <56B285B0.8010306@shrew.net> Date: Wed, 03 Feb 2016 16:56:48 -0600 From: Matthew Grooms User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org, freebsd-net@freebsd.org Subject: 10.2-RELEASE-p12 pf+GRE crashing Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mx1.shrew.net [10.24.10.10]); Wed, 03 Feb 2016 16:54:21 -0600 (CST) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Feb 2016 23:15:22 -0000 All, I recently upgraded a pair of 10.0-RELEASE firewalls in the hope that I could avoid the local patching required to keep it up and running. Unfortunately, it crashes whenever I reload my pf firewall rule set. If I remove the GRE tunnel configurations from rc.conf, it happily reloads the rule set all day long. The kernel config is mostly GENERIC with the following additions ... # Packet Filter device pf # PF OpenBSD packet-filter firewall device pflog # Logging support interface for PF device pfsync # Synchronization interface for PF device carp # Common Address Redundancy Protocol # IPsec device crypto device enc options IPSEC The crash is easy to reproduce as pfctl -f /etc/pf.conf does it every time. I should also mention that I tried with and without the following additional commits applied, but get the same result ... https://svnweb.freebsd.org/base?view=revision&revision=272695 https://svnweb.freebsd.org/base?view=revision&revision=288529 I'm also a bit confused as to why these patches haven't made it into 10 STABLE yet. The former doesn't mention an MFC and the latter has an MFC of 1 week, but was never done. In any case, here is the output from kgdb ... [root@fw2 /var/crash]# kgdb /boot/kernel/kernel vmcore.3 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x4a4 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff809c07f4 stack pointer = 0x28:0xfffffe0000233b30 frame pointer = 0x28:0xfffffe0000233b40 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1990 (pfctl) trap number = 12 panic: page fault cpuid = 1 KDB: stack backtrace: #0 0xffffffff808048c0 at kdb_backtrace+0x60 #1 0xffffffff807c8596 at vpanic+0x126 #2 0xffffffff807c8463 at panic+0x43 #3 0xffffffff80bdc10b at trap_fatal+0x36b #4 0xffffffff80bdc40d at trap_pfault+0x2ed #5 0xffffffff80bdbaaa at trap+0x47a #6 0xffffffff80bc1fa2 at calltrap+0x8 #7 0xffffffff809a91f4 at pf_empty_pool+0x44 #8 0xffffffff809ab3e5 at pfioctl+0x805 #9 0xffffffff806b5659 at devfs_ioctl_f+0x139 #10 0xffffffff8081b805 at kern_ioctl+0x255 #11 0xffffffff8081b500 at sys_ioctl+0x140 #12 0xffffffff80bdca27 at amd64_syscall+0x357 #13 0xffffffff80bc228b at Xfast_syscall+0xfb Uptime: 1m1s Dumping 156 out of 2025 MB:..11%..21%..31%..42%..52%..62%..72%..83%..93% Reading symbols from /boot/kernel/if_lagg.ko.symbols...done. Loaded symbols for /boot/kernel/if_lagg.ko.symbols Reading symbols from /boot/kernel/if_gre.ko.symbols...done. Loaded symbols for /boot/kernel/if_gre.ko.symbols #0 doadump (textdump=) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h Any help in resolving this would be greatly appreciated. -Matthew From owner-freebsd-net@freebsd.org Thu Feb 4 00:47:36 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5AB76A9B96F; Thu, 4 Feb 2016 00:47:36 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mx1.shrew.net (mx1.shrew.net [38.97.5.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2EA9E114A; Thu, 4 Feb 2016 00:47:35 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx1.shrew.net (8.14.7/8.14.7) with ESMTP id u140ivuS065863; Wed, 3 Feb 2016 18:44:57 -0600 (CST) (envelope-from mgrooms@shrew.net) Received: from [10.16.48.132] (67-198-50-4.static.grandenetworks.net [67.198.50.4]) by mail.shrew.net (Postfix) with ESMTPSA id E6C0018C85D; Wed, 3 Feb 2016 18:44:51 -0600 (CST) Message-ID: <56B29FA0.4080000@shrew.net> Date: Wed, 03 Feb 2016 18:47:28 -0600 From: Matthew Grooms User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org, freebsd-net@freebsd.org Subject: Re: 10.2-RELEASE-p12 pf+GRE crashing References: <56B285B0.8010306@shrew.net> In-Reply-To: <56B285B0.8010306@shrew.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mx1.shrew.net [10.24.10.10]); Wed, 03 Feb 2016 18:44:57 -0600 (CST) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 00:47:36 -0000 On 2/3/2016 4:56 PM, Matthew Grooms wrote: > All, > > I recently upgraded a pair of 10.0-RELEASE firewalls in the hope that > I could avoid the local patching required to keep it up and running. > Unfortunately, it crashes whenever I reload my pf firewall rule set. > If I remove the GRE tunnel configurations from rc.conf, it happily > reloads the rule set all day long. The kernel config is mostly GENERIC > with the following additions ... > > # Packet Filter > device pf # PF OpenBSD packet-filter firewall > device pflog # Logging support interface for PF > device pfsync # Synchronization interface for PF > device carp # Common Address Redundancy Protocol > > # IPsec > device crypto > device enc > options IPSEC > > The crash is easy to reproduce as pfctl -f /etc/pf.conf does it every > time. I should also mention that I tried with and without the > following additional commits applied, but get the same result ... > > https://svnweb.freebsd.org/base?view=revision&revision=272695 > https://svnweb.freebsd.org/base?view=revision&revision=288529 > > I'm also a bit confused as to why these patches haven't made it into > 10 STABLE yet. The former doesn't mention an MFC and the latter has an > MFC of 1 week, but was never done. In any case, here is the output > from kgdb ... This turned out to be another issue that was patched in head but not back ported to stable. I can't explain why it didn't get tripped when GRE tunnels were disabled. With the patch applied, I can reload my rule sets again without crashing ... https://svnweb.freebsd.org/base?view=revision&revision=264689 (kgdb) bt #0 doadump (textdump=) at pcpu.h:219 #1 0xffffffff807c81f2 in kern_reboot (howto=260) at ../../../kern/kern_shutdown.c:451 #2 0xffffffff807c85d5 in vpanic (fmt=, ap=) at ../../../kern/kern_shutdown.c:758 #3 0xffffffff807c8463 in panic (fmt=0x0) at ../../../kern/kern_shutdown.c:687 #4 0xffffffff80bdc10b in trap_fatal (frame=, eva=) at ../../../amd64/amd64/trap.c:851 #5 0xffffffff80bdc40d in trap_pfault (frame=0xfffffe0000233a80, usermode=) at ../../../amd64/amd64/trap.c:674 #6 0xffffffff80bdbaaa in trap (frame=0xfffffe0000233a80) at ../../../amd64/amd64/trap.c:440 #7 0xffffffff80bc1fa2 in calltrap () at ../../../amd64/amd64/exception.S:236 #8 0xffffffff809c07f4 in pfr_detach_table (kt=0x0) at ../../../netpfil/pf/pf_table.c:2047 #9 0xffffffff809a91f4 in pf_empty_pool (poola=0xffffffff813c3d68) at ../../../netpfil/pf/pf_ioctl.c:354 #10 0xffffffff809ab3e5 in pfioctl (dev=, cmd=, addr=0xfffff8005eaf6800 "", flags=, td=) at ../../../netpfil/pf/pf_ioctl.c:2189 #11 0xffffffff806b5659 in devfs_ioctl_f (fp=0xfffff8000a2927d0, com=3295691827, data=0xfffff8005eaf6800, cred=, td=0xfffff8000a25f000) at ../../../fs/devfs/devfs_vnops.c:785 #12 0xffffffff8081b805 in kern_ioctl (td=0xfffff8000a25f000, fd=, com=2) at file.h:320 #13 0xffffffff8081b500 in sys_ioctl (td=0xfffff8000a25f000, uap=0xfffffe0000234b40) at ../../../kern/sys_generic.c:718 #14 0xffffffff80bdca27 in amd64_syscall (td=0xfffff8000a25f000, traced=0) at subr_syscall.c:134 #15 0xffffffff80bc228b in Xfast_syscall () at ../../../amd64/amd64/exception.S:396 #16 0x0000000800dd9fda in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal -Matthew From owner-freebsd-net@freebsd.org Thu Feb 4 00:45:26 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8B22A9B8CF for ; Thu, 4 Feb 2016 00:45:26 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id D585210C0 for ; Thu, 4 Feb 2016 00:45:26 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id D1823106226; Thu, 4 Feb 2016 00:45:26 +0000 (UTC) Date: Thu, 4 Feb 2016 00:45:26 +0000 To: freebsd-net@freebsd.org From: "honzhan_microsoft.com (hongjiangzhang)" Reply-to: D5175+325+b48f3d05eeb00f73@reviews.freebsd.org Subject: [Differential] [Accepted] D5175: hyperv/hn: Add an option to always do transmission scheduling Message-ID: <88fe64d570d8f7f9cf4cdb972056d226@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , Thread-Topic: D5175: hyperv/hn: Add an option to always do transmission scheduling X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: OGFmNjgzYWFmZmVhMjQ0MTIwYzQ1OWJjZDg2IFaynyY= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-Mailman-Approved-At: Thu, 04 Feb 2016 00:53:51 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 00:45:27 -0000 honzhan_microsoft.com accepted this revision. honzhan_microsoft.com added a comment. Looks ok to me. REVISION DETAIL https://reviews.freebsd.org/D5175 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, howard0su_gmail.com, adrian, network, honzhan_microsoft.com Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 03:42:03 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC6A9A99796 for ; Thu, 4 Feb 2016 03:42:03 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id CAF6FD74 for ; Thu, 4 Feb 2016 03:42:03 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id C727A106EE0; Thu, 4 Feb 2016 03:42:03 +0000 (UTC) Date: Thu, 4 Feb 2016 03:42:03 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5166+325+a3471c38b880b49e@reviews.freebsd.org Subject: [Differential] [Accepted] D5166: hyperv/hn: Increase LRO entry count to 128 by default Message-ID: <65b78aef7961b607bc8159be083be6fc@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5166: hyperv/hn: Increase LRO entry count to 128 by default X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NWRmOGI3NDFlMWM4OTg3OGUxYjZjYzIwNmVlIFayyIs= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 03:42:04 -0000 adrian accepted this revision. This revision has a positive review. REVISION DETAIL https://reviews.freebsd.org/D5166 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 03:42:32 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE894A997FF for ; Thu, 4 Feb 2016 03:42:32 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id BDD60F2B for ; Thu, 4 Feb 2016 03:42:32 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id BB358106079; Thu, 4 Feb 2016 03:42:32 +0000 (UTC) Date: Thu, 4 Feb 2016 03:42:32 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5098+325+4e694d74d281089c@reviews.freebsd.org Subject: [Differential] [Accepted] D5098: hyperv/hn: Reorganize TX csum offloading Message-ID: <2183fbd5a290c08fd7875067055b6358@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , Thread-Topic: D5098: hyperv/hn: Reorganize TX csum offloading X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: MWY5NTZkMGRiYzcxOTg0MWVmNWQ1NWNiZWRhIFayyKg= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 03:42:32 -0000 adrian accepted this revision. adrian added a comment. Good! REVISION DETAIL https://reviews.freebsd.org/D5098 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, howard0su_gmail.com, honzhan_microsoft.com, adrian, network Cc: freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 03:43:51 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1D03A998BA for ; Thu, 4 Feb 2016 03:43:51 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id A0DA51024 for ; Thu, 4 Feb 2016 03:43:51 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 9EBD310611E; Thu, 4 Feb 2016 03:43:51 +0000 (UTC) Date: Thu, 4 Feb 2016 03:43:51 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5103+325+9f7cd214eb126a32@reviews.freebsd.org Subject: [Differential] [Accepted] D5103: hyperv/hn: Add sysctl to trust host side UDP and IP csum verification Message-ID: <0aebef0336db0e289427bc43dc3a8122@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , Thread-Topic: D5103: hyperv/hn: Add sysctl to trust host side UDP and IP csum verification X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: YTc1N2JhYmJkMzBkNmVlOWEyYjZiYzZjY2FjIFayyPc= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 03:43:51 -0000 adrian accepted this revision. adrian added a comment. So this is okay, but we put the non-unit bits under 'hw', not 'dev'. Ie, it'd be dev.XXX.0.stuff, and hw.XXX.stuff. So I'll okay this, but we should eventually shuffle them back to 'hw'. REVISION DETAIL https://reviews.freebsd.org/D5103 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, howard0su_gmail.com, honzhan_microsoft.com, adrian, network Cc: freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 03:44:33 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31F39A99954 for ; Thu, 4 Feb 2016 03:44:33 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 20FB2110C for ; Thu, 4 Feb 2016 03:44:33 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 1E52010618F; Thu, 4 Feb 2016 03:44:33 +0000 (UTC) Date: Thu, 4 Feb 2016 03:44:33 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5158+325+84c30785c1b48e01@reviews.freebsd.org Subject: [Differential] [Accepted] D5158: hyperv/hn: Factor out hn_encap from hn_start_locked() Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5158: hyperv/hn: Factor out hn_encap from hn_start_locked() X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: MjFmOWM1NjYwZmMxOTMyMjJlM2E1YTQ1MjllIFayySE= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 03:44:33 -0000 adrian accepted this revision. This revision has a positive review. REVISION DETAIL https://reviews.freebsd.org/D5158 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 03:44:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FA04A99991 for ; Thu, 4 Feb 2016 03:44:55 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 8E5E511D5 for ; Thu, 4 Feb 2016 03:44:55 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 89FD6106205; Thu, 4 Feb 2016 03:44:55 +0000 (UTC) Date: Thu, 4 Feb 2016 03:44:55 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5159+325+0db17ec577dfe5ca@reviews.freebsd.org Subject: [Differential] [Accepted] D5159: hyperv/hn: Recover half of the chimney sending space Message-ID: <2d870d2160028043526009c0555aec53@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5159: hyperv/hn: Recover half of the chimney sending space X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: MzRmNWEyNTk4ODRlNWNkNGYzNTA2Yjk4Nzc1IFayyTc= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 03:44:55 -0000 adrian accepted this revision. This revision has a positive review. REVISION DETAIL https://reviews.freebsd.org/D5159 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 03:45:22 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56431A99A08 for ; Thu, 4 Feb 2016 03:45:22 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 4575312A5 for ; Thu, 4 Feb 2016 03:45:22 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 4250A10626F; Thu, 4 Feb 2016 03:45:22 +0000 (UTC) Date: Thu, 4 Feb 2016 03:45:22 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5167+325+acc9bcb5dbcc28f2@reviews.freebsd.org Subject: [Differential] [Accepted] D5167: hyperv/hn: Move LRO flush to the channel processing rollup Message-ID: <4dc62f46364c6fb08ae5eb226a2a08b4@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5167: hyperv/hn: Move LRO flush to the channel processing rollup X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: N2NlNTE2YjI3MTBjYjEzNmM4OWIyM2FkMjc1IFayyVI= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 03:45:22 -0000 adrian accepted this revision. This revision has a positive review. REVISION DETAIL https://reviews.freebsd.org/D5167 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 08:44:09 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBB59A9B1AC for ; Thu, 4 Feb 2016 08:44:09 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id C50761442 for ; Thu, 4 Feb 2016 08:44:09 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id C25E510755D; Thu, 4 Feb 2016 08:44:09 +0000 (UTC) Date: Thu, 4 Feb 2016 08:44:09 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Request, 117 lines] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdm MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_77fc80858801b21443ab2a903cf15bcc" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 08:44:10 -0000 --b1_77fc80858801b21443ab2a903cf15bcc Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com created this revision. sepherosa_gmail.com added reviewers: network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, gallatin, hselasky, np. sepherosa_gmail.com added subscribers: freebsd-net-list, freebsd-virtualization-list. Herald added a reviewer: transport. REVISION SUMMARY It's append_cnt based. Unless the network driver sets these two limits, its an NO-OP. For hn(4): - Set TCP ACK append limit to 1, i.e. aggregate 2 ACKs at most. Aggregate anything more than 2 hurts TCP sending performance in hyperv. This significantly improves the TCP sending performance when the number of concurrent connetion is low (2~8). And greatly stabilize the TCP sending performance in other cases. - Set TCP data segments append limit to 25. Without this limitation, hn(4) could aggregate ~45 TCP data segments for each connection (even at 64 or more connections) before dispatching them to socket code; large aggregation slows down ACK sending and eventually hurts/destabilizes TCP reception performance. This setting stabilizes and improves TCP reception performance for >4 concurrent connections significantly. Make them sysctls so they could be adjusted. REVISION DETAIL https://reviews.freebsd.org/D5185 AFFECTED FILES sys/dev/hyperv/netvsc/hv_net_vsc.h sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c sys/netinet/tcp_lro.c sys/netinet/tcp_lro.h EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, transport, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, gallatin, hselasky, np Cc: freebsd-virtualization-list, freebsd-net-list --b1_77fc80858801b21443ab2a903cf15bcc Content-Type: text/x-patch; charset=utf-8; name="D5185.12995.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5185.12995.patch" ZGlmZiAtLWdpdCBhL3N5cy9uZXRpbmV0L3RjcF9scm8uaCBiL3N5cy9uZXRpbmV0L3RjcF9scm8u aAotLS0gYS9zeXMvbmV0aW5ldC90Y3BfbHJvLmgKKysrIGIvc3lzL25ldGluZXQvdGNwX2xyby5o CkBAIC05MSw2ICs5MSw4IEBACiAJdW5zaWduZWQJbHJvX2NudDsKIAl1bnNpZ25lZAlscm9fbWJ1 Zl9jb3VudDsKIAl1bnNpZ25lZAlscm9fbWJ1Zl9tYXg7CisJdW5zaWduZWQgc2hvcnQJbHJvX2Fj a19hcHBlbmRfbGltOworCXVuc2lnbmVkIHNob3J0CWxyb19kYXRhX2FwcGVuZF9saW07CiAKIAlz dHJ1Y3QgbHJvX2hlYWQJbHJvX2FjdGl2ZTsKIAlzdHJ1Y3QgbHJvX2hlYWQJbHJvX2ZyZWU7CmRp ZmYgLS1naXQgYS9zeXMvbmV0aW5ldC90Y3BfbHJvLmMgYi9zeXMvbmV0aW5ldC90Y3BfbHJvLmMK LS0tIGEvc3lzL25ldGluZXQvdGNwX2xyby5jCisrKyBiL3N5cy9uZXRpbmV0L3RjcF9scm8uYwpA QCAtODgsNiArODgsOCBAQAogCWxjLT5scm9fbWJ1Zl9tYXggPSBscm9fbWJ1ZnM7CiAJbGMtPmxy b19jbnQgPSBscm9fZW50cmllczsKIAlsYy0+aWZwID0gaWZwOworCWxjLT5scm9fYWNrX2FwcGVu ZF9saW0gPSAwOworCWxjLT5scm9fZGF0YV9hcHBlbmRfbGltID0gMDsKIAlTTElTVF9JTklUKCZs Yy0+bHJvX2ZyZWUpOwogCVNMSVNUX0lOSVQoJmxjLT5scm9fYWN0aXZlKTsKIApAQCAtNjQ2LDYg KzY0OCwxNiBAQAogCiAJCWlmICh0Y3BfZGF0YV9sZW4gPT0gMCkgewogCQkJbV9mcmVlbShtKTsK KwkJCS8qCisJCQkgKiBGbHVzaCB0aGlzIExSTyBlbnRyeSwgaWYgdGhpcyBBQ0sgc2hvdWxkCisJ CQkgKiBub3QgYmUgZnVydGhlciBkZWxheWVkLgorCQkJICovCisJCQlpZiAobGMtPmxyb19hY2tf YXBwZW5kX2xpbSAmJgorCQkJICAgIGxlLT5hcHBlbmRfY250ID49IGxjLT5scm9fYWNrX2FwcGVu ZF9saW0pIHsKKwkJCQlTTElTVF9SRU1PVkUoJmxjLT5scm9fYWN0aXZlLCBsZSwgbHJvX2VudHJ5 LAorCQkJCSAgICBuZXh0KTsKKwkJCQl0Y3BfbHJvX2ZsdXNoKGxjLCBsZSk7CisJCQl9CiAJCQly ZXR1cm4gKDApOwogCQl9CiAKQEAgLTY2NCw5ICs2NzYsMTIgQEAKIAogCQkvKgogCQkgKiBJZiBh IHBvc3NpYmxlIG5leHQgZnVsbCBsZW5ndGggcGFja2V0IHdvdWxkIGNhdXNlIGFuCi0JCSAqIG92 ZXJmbG93LCBwcm8tYWN0aXZlbHkgZmx1c2ggbm93LgorCQkgKiBvdmVyZmxvdywgcHJvLWFjdGl2 ZWx5IGZsdXNoIG5vdy4gIEFuZCBpZiB3ZSBhcmUgYXNrZWQKKwkJICogdG8gbGltaXQgdGhlIGRh dGEgYWdncmVnYXRlLCBmbHVzaCB0aGlzIExSTyBlbnRyeSBub3cuCiAJCSAqLwotCQlpZiAobGUt PnBfbGVuID4gKDY1NTM1IC0gbGMtPmlmcC0+aWZfbXR1KSkgeworCQlpZiAobGUtPnBfbGVuID4g KDY1NTM1IC0gbGMtPmlmcC0+aWZfbXR1KSB8fAorCQkgICAgKGxjLT5scm9fZGF0YV9hcHBlbmRf bGltICYmCisJCSAgICAgbGUtPmFwcGVuZF9jbnQgPj0gbGMtPmxyb19kYXRhX2FwcGVuZF9saW0p KSB7CiAJCQlTTElTVF9SRU1PVkUoJmxjLT5scm9fYWN0aXZlLCBsZSwgbHJvX2VudHJ5LCBuZXh0 KTsKIAkJCXRjcF9scm9fZmx1c2gobGMsIGxlKTsKIAkJfSBlbHNlCmRpZmYgLS1naXQgYS9zeXMv ZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMgYi9zeXMvZGV2L2h5cGVy di9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMKLS0tIGEvc3lzL2Rldi9oeXBlcnYvbmV0 dnNjL2h2X25ldHZzY19kcnZfZnJlZWJzZC5jCisrKyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9o dl9uZXR2c2NfZHJ2X2ZyZWVic2QuYwpAQCAtMTc2LDE0ICsxNzYsOCBAQAogI2RlZmluZSBITl9D U1VNX0FTU0lTVF9XSU44CShDU1VNX1RDUCkKICNkZWZpbmUgSE5fQ1NVTV9BU1NJU1QJCShDU1VN X0lQIHwgQ1NVTV9VRFAgfCBDU1VNX1RDUCkKIAotLyogWFhYIG1vdmUgdG8gbmV0aW5ldC90Y3Bf bHJvLmggKi8KLSNkZWZpbmUgSE5fTFJPX0hJV0FUX01BWAkJCQk2NTUzNQotI2RlZmluZSBITl9M Uk9fSElXQVRfREVGCQkJCUhOX0xST19ISVdBVF9NQVgKLS8qIFlZWSAyKk1UVSBpcyBhIGJpdCBy b3VnaCwgYnV0IHNob3VsZCBiZSBnb29kIGVub3VnaC4gKi8KLSNkZWZpbmUgSE5fTFJPX0hJV0FU X01UVUxJTShpZnApCQkJKDIgKiAoaWZwKS0+aWZfbXR1KQotI2RlZmluZSBITl9MUk9fSElXQVRf SVNWQUxJRChzYywgaGl3YXQpCQkJXAotICAgICgoaGl3YXQpID49IEhOX0xST19ISVdBVF9NVFVM SU0oKHNjKS0+aG5faWZwKSB8fAlcCi0gICAgIChoaXdhdCkgPD0gSE5fTFJPX0hJV0FUX01BWCkK KyNkZWZpbmUgSE5fTFJPX0FDS19BUFBFTkRfTElNCTEKKyNkZWZpbmUgSE5fTFJPX0RBVEFfQVBQ RU5EX0xJTQkyNQogCiAvKgogICogQmUgYXdhcmUgdGhhdCB0aGlzIHNsZWVwYWJsZSBtdXRleCB3 aWxsIGV4aGliaXQgV0lUTkVTUyBlcnJvcnMgd2hlbgpAQCAtMjUzLDI3ICsyNDcsMTYgQEAKIHN0 YXRpYyB2b2lkIGhuX3N0YXJ0X3R4ZW9mKHN0cnVjdCBpZm5ldCAqaWZwKTsKIHN0YXRpYyBpbnQg aG5faWZtZWRpYV91cGQoc3RydWN0IGlmbmV0ICppZnApOwogc3RhdGljIHZvaWQgaG5faWZtZWRp YV9zdHMoc3RydWN0IGlmbmV0ICppZnAsIHN0cnVjdCBpZm1lZGlhcmVxICppZm1yKTsKLSNpZmRl ZiBITl9MUk9fSElXQVQKLXN0YXRpYyBpbnQgaG5fbHJvX2hpd2F0X3N5c2N0bChTWVNDVExfSEFO RExFUl9BUkdTKTsKLSNlbmRpZgogc3RhdGljIGludCBobl90cnVzdF9oY3N1bV9zeXNjdGwoU1lT Q1RMX0hBTkRMRVJfQVJHUyk7CiBzdGF0aWMgaW50IGhuX3R4X2NoaW1uZXlfc2l6ZV9zeXNjdGwo U1lTQ1RMX0hBTkRMRVJfQVJHUyk7CitzdGF0aWMgaW50IGhuX2xyb19hcHBlbmRfbGltX3N5c2N0 bChTWVNDVExfSEFORExFUl9BUkdTKTsKIHN0YXRpYyBpbnQgaG5fY2hlY2tfaXBsZW4oY29uc3Qg c3RydWN0IG1idWYgKiwgaW50KTsKIHN0YXRpYyBpbnQgaG5fY3JlYXRlX3R4X3Jpbmcoc3RydWN0 IGhuX3NvZnRjICpzYyk7CiBzdGF0aWMgdm9pZCBobl9kZXN0cm95X3R4X3Jpbmcoc3RydWN0IGhu X3NvZnRjICpzYyk7CiBzdGF0aWMgdm9pZCBobl9zdGFydF90YXNrZnVuYyh2b2lkICp4c2MsIGlu dCBwZW5kaW5nKTsKIHN0YXRpYyB2b2lkIGhuX3R4ZW9mX3Rhc2tmdW5jKHZvaWQgKnhzYywgaW50 IHBlbmRpbmcpOwogc3RhdGljIGludCBobl9lbmNhcChzdHJ1Y3QgaG5fc29mdGMgKiwgc3RydWN0 IGhuX3R4ZGVzYyAqLCBzdHJ1Y3QgbWJ1ZiAqKik7CiAKLXN0YXRpYyBfX2lubGluZSB2b2lkCi1o bl9zZXRfbHJvX2hpd2F0KHN0cnVjdCBobl9zb2Z0YyAqc2MsIGludCBoaXdhdCkKLXsKLQlzYy0+ aG5fbHJvX2hpd2F0ID0gaGl3YXQ7Ci0jaWZkZWYgSE5fTFJPX0hJV0FUCi0Jc2MtPmhuX2xyby5s cm9faGl3YXQgPSBzYy0+aG5fbHJvX2hpd2F0OwotI2VuZGlmCi19Ci0KIHN0YXRpYyBpbnQKIGhu X2lmbWVkaWFfdXBkKHN0cnVjdCBpZm5ldCAqaWZwIF9fdW51c2VkKQogewpAQCAtMzU4LDcgKzM0 MSw2IEBACiAJYnplcm8oc2MsIHNpemVvZihobl9zb2Z0Y190KSk7CiAJc2MtPmhuX3VuaXQgPSB1 bml0OwogCXNjLT5obl9kZXYgPSBkZXY7Ci0Jc2MtPmhuX2xyb19oaXdhdCA9IEhOX0xST19ISVdB VF9ERUY7CiAJc2MtPmhuX2RpcmVjdF90eF9zaXplID0gaG5fZGlyZWN0X3R4X3NpemU7CiAJaWYg KGhuX3RydXN0X2hvc3R0Y3ApCiAJCXNjLT5obl90cnVzdF9oY3N1bSB8PSBITl9UUlVTVF9IQ1NV TV9UQ1A7CkBAIC00NDIsOSArNDI0LDggQEAKIAkvKiBEcml2ZXIgcHJpdmF0ZSBMUk8gc2V0dGlu Z3MgKi8KIAlzYy0+aG5fbHJvLmlmcCA9IGlmcDsKICNlbmRpZgotI2lmZGVmIEhOX0xST19ISVdB VAotCXNjLT5obl9scm8ubHJvX2hpd2F0ID0gc2MtPmhuX2xyb19oaXdhdDsKLSNlbmRpZgorCXNj LT5obl9scm8ubHJvX2Fja19hcHBlbmRfbGltID0gSE5fTFJPX0FDS19BUFBFTkRfTElNOworCXNj LT5obl9scm8ubHJvX2RhdGFfYXBwZW5kX2xpbSA9IEhOX0xST19EQVRBX0FQUEVORF9MSU07CiAj ZW5kaWYJLyogSU5FVCB8fCBJTkVUNiAqLwogCiAjaWYgX19GcmVlQlNEX3ZlcnNpb24gPj0gMTEw MDA0NQpAQCAtNDcxLDYgKzQ1MiwxMyBAQAogCSAgICBobl90eF9jaGltbmV5X3NpemUgPCBzYy0+ aG5fdHhfY2hpbW5leV9tYXgpCiAJCXNjLT5obl90eF9jaGltbmV5X3NpemUgPSBobl90eF9jaGlt bmV5X3NpemU7CiAKKwkvKgorCSAqIEFsd2F5cyBzY2hlZHVsZSB0cmFuc21pc3Npb24gaW5zdGVh ZCBvZiB0cnlpbmcKKwkgKiB0byBkbyBkaXJlY3QgdHJhbnNtaXNzaW9uLiAgVGhpcyBvbmUgZ2l2 ZXMgdGhlCisJICogYmVzdCBwZXJmb3JtYW5jZSBzbyBmYXIuCisJICovCisJc2MtPmhuX3NjaGVk X3R4ID0gMTsKKwogCWN0eCA9IGRldmljZV9nZXRfc3lzY3RsX2N0eChkZXYpOwogCWNoaWxkID0g U1lTQ1RMX0NISUxEUkVOKGRldmljZV9nZXRfc3lzY3RsX3RyZWUoZGV2KSk7CiAKQEAgLTQ4MCwx MSArNDY4LDYgQEAKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9scm8ubHJvX2ZsdXNoZWQsIDAs ICJMUk8gZmx1c2hlZCIpOwogCVNZU0NUTF9BRERfVUxPTkcoY3R4LCBjaGlsZCwgT0lEX0FVVE8s ICJscm9fdHJpZWQiLAogCSAgICBDVExGTEFHX1JXLCAmc2MtPmhuX2xyb190cmllZCwgIiMgb2Yg TFJPIHRyaWVzIik7Ci0jaWZkZWYgSE5fTFJPX0hJV0FUCi0JU1lTQ1RMX0FERF9QUk9DKGN0eCwg Y2hpbGQsIE9JRF9BVVRPLCAibHJvX2hpd2F0IiwKLQkgICAgQ1RMVFlQRV9JTlQgfCBDVExGTEFH X1JXLCBzYywgMCwgaG5fbHJvX2hpd2F0X3N5c2N0bCwKLQkgICAgIkkiLCAiTFJPIGhpZ2ggd2F0 ZXJtYXJrIik7Ci0jZW5kaWYKIAlTWVNDVExfQUREX1BST0MoY3R4LCBjaGlsZCwgT0lEX0FVVE8s ICJ0cnVzdF9ob3N0dGNwIiwKIAkgICAgQ1RMVFlQRV9JTlQgfCBDVExGTEFHX1JXLCBzYywgSE5f VFJVU1RfSENTVU1fVENQLAogCSAgICBobl90cnVzdF9oY3N1bV9zeXNjdGwsICJJIiwKQEAgLTUz OCw2ICs1MjEsMTQgQEAKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9zY2hlZF90eCwgMCwKIAkg ICAgIkFsd2F5cyBzY2hlZHVsZSB0cmFuc21pc3Npb24gIgogCSAgICAiaW5zdGVhZCBvZiBkb2lu ZyBkaXJlY3QgdHJhbnNtaXNzaW9uIik7CisJU1lTQ1RMX0FERF9QUk9DKGN0eCwgY2hpbGQsIE9J RF9BVVRPLCAibHJvX2Fja19hcHBlbmRfbGltIiwKKwkgICAgQ1RMVFlQRV9JTlQgfCBDVExGTEFH X1JXLCAmc2MtPmhuX2xyby5scm9fYWNrX2FwcGVuZF9saW0sCisJICAgIDAsIGhuX2xyb19hcHBl bmRfbGltX3N5c2N0bCwKKwkgICAgIkkiLCAiTFJPIEFDSyBhcHBlbmRpbmcgbGltaXRhdGlvbiIp OworCVNZU0NUTF9BRERfUFJPQyhjdHgsIGNoaWxkLCBPSURfQVVUTywgImxyb19kYXRhX2FwcGVu ZF9saW0iLAorCSAgICBDVExUWVBFX0lOVCB8IENUTEZMQUdfUlcsICZzYy0+aG5fbHJvLmxyb19k YXRhX2FwcGVuZF9saW0sCisJICAgIDAsIGhuX2xyb19hcHBlbmRfbGltX3N5c2N0bCwKKwkgICAg IkkiLCAiTFJPIGRhdGEgc2VnbWVudHMgYXBwZW5kaW5nIGxpbWl0YXRpb24iKTsKIAogCWlmICh1 bml0ID09IDApIHsKIAkJc3RydWN0IHN5c2N0bF9jdHhfbGlzdCAqZGNfY3R4OwpAQCAtMTQxMCwx MyArMTQwMSw2IEBACiAKIAkJLyogT2J0YWluIGFuZCByZWNvcmQgcmVxdWVzdGVkIE1UVSAqLwog CQlpZnAtPmlmX210dSA9IGlmci0+aWZyX210dTsKLQkJLyoKLQkJICogTWFrZSBzdXJlIHRoYXQg TFJPIGhpZ2ggd2F0ZXJtYXJrIGlzIHN0aWxsIHZhbGlkLAotCQkgKiBhZnRlciBNVFUgY2hhbmdl ICh0aGUgMipNVFUgbGltaXQpLgotCQkgKi8KLQkJaWYgKCFITl9MUk9fSElXQVRfSVNWQUxJRChz Yywgc2MtPmhuX2xyb19oaXdhdCkpCi0JCQlobl9zZXRfbHJvX2hpd2F0KHNjLCBITl9MUk9fSElX QVRfTVRVTElNKGlmcCkpOwotCiAJCWRvIHsKIAkJCU5WX0xPQ0soc2MpOwogCQkJaWYgKCFzYy0+ dGVtcF91bnVzYWJsZSkgewpAQCAtMTcyMiwyNyArMTcwNiw2IEBACiB9CiAjZW5kaWYKIAotI2lm ZGVmIEhOX0xST19ISVdBVAotc3RhdGljIGludAotaG5fbHJvX2hpd2F0X3N5c2N0bChTWVNDVExf SEFORExFUl9BUkdTKQotewotCXN0cnVjdCBobl9zb2Z0YyAqc2MgPSBhcmcxOwotCWludCBoaXdh dCwgZXJyb3I7Ci0KLQloaXdhdCA9IHNjLT5obl9scm9faGl3YXQ7Ci0JZXJyb3IgPSBzeXNjdGxf aGFuZGxlX2ludChvaWRwLCAmaGl3YXQsIDAsIHJlcSk7Ci0JaWYgKGVycm9yIHx8IHJlcS0+bmV3 cHRyID09IE5VTEwpCi0JCXJldHVybiBlcnJvcjsKLQotCWlmICghSE5fTFJPX0hJV0FUX0lTVkFM SUQoc2MsIGhpd2F0KSkKLQkJcmV0dXJuIEVJTlZBTDsKLQotCWlmIChzYy0+aG5fbHJvX2hpd2F0 ICE9IGhpd2F0KQotCQlobl9zZXRfbHJvX2hpd2F0KHNjLCBoaXdhdCk7Ci0JcmV0dXJuIDA7Ci19 Ci0jZW5kaWYJLyogSE5fTFJPX0hJV0FUICovCi0KIHN0YXRpYyBpbnQKIGhuX3RydXN0X2hjc3Vt X3N5c2N0bChTWVNDVExfSEFORExFUl9BUkdTKQogewpAQCAtMTc4Nyw2ICsxNzUwLDI0IEBACiB9 CiAKIHN0YXRpYyBpbnQKK2huX2xyb19hcHBlbmRfbGltX3N5c2N0bChTWVNDVExfSEFORExFUl9B UkdTKQoreworCXVuc2lnbmVkIHNob3J0ICpscm9fbGltID0gYXJnMTsKKwlpbnQgbGltLCBlcnJv cjsKKworCWxpbSA9ICpscm9fbGltOworCWVycm9yID0gc3lzY3RsX2hhbmRsZV9pbnQob2lkcCwg JmxpbSwgMCwgcmVxKTsKKwlpZiAoZXJyb3IgfHwgcmVxLT5uZXdwdHIgPT0gTlVMTCkKKwkJcmV0 dXJuIGVycm9yOworCisJaWYgKGxpbSA8IDAgfHwgbGltID4gNjU1MzUpCisJCXJldHVybiBFSU5W QUw7CisKKwkqbHJvX2xpbSA9IGxpbTsKKwlyZXR1cm4gMDsKK30KKworc3RhdGljIGludAogaG5f Y2hlY2tfaXBsZW4oY29uc3Qgc3RydWN0IG1idWYgKm0sIGludCBob2ZmKQogewogCWNvbnN0IHN0 cnVjdCBpcCAqaXA7CmRpZmYgLS1naXQgYS9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0X3Zz Yy5oIGIvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuaAotLS0gYS9zeXMvZGV2L2h5 cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5oCisrKyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9u ZXRfdnNjLmgKQEAgLTEwMzAsNyArMTAzMCw2IEBACiAJc3RydWN0IHRhc2sJaG5fdHhlb2ZfdGFz azsKIAogCXN0cnVjdCBscm9fY3RybAlobl9scm87Ci0JaW50CQlobl9scm9faGl3YXQ7CiAKIAkv KiBUcnVzdCBjc3VtIHZlcmlmaWNhdGlvbiBvbiBob3N0IHNpZGUgKi8KIAlpbnQJCWhuX3RydXN0 X2hjc3VtOwkvKiBITl9UUlVTVF9IQ1NVTV8gKi8KCg== --b1_77fc80858801b21443ab2a903cf15bcc-- From owner-freebsd-net@freebsd.org Thu Feb 4 08:46:58 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DAC31A9B3AD for ; Thu, 4 Feb 2016 08:46:58 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id C4E4E1812 for ; Thu, 4 Feb 2016 08:46:58 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id BFDAD1076C2; Thu, 4 Feb 2016 08:46:58 +0000 (UTC) Date: Thu, 4 Feb 2016 08:46:58 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Updated] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <6459f2659e2fe7441077a4f49775d048@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFazEAI= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 08:46:58 -0000 sepherosa_gmail.com updated the summary for this revision. REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, gallatin, hselasky, np, transport Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 09:09:18 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9A4BA9BDFB; Thu, 4 Feb 2016 09:09:18 +0000 (UTC) (envelope-from remy.nonnenmacher@boostedge.com) Received: from fr-exchange.activnetworks.com (fr-exchange.activnetworks.com [62.210.235.130]) by mx1.freebsd.org (Postfix) with ESMTP id 67C9EA51; Thu, 4 Feb 2016 09:09:17 +0000 (UTC) (envelope-from remy.nonnenmacher@boostedge.com) Received: from rn.activnetworks.com ([192.168.1.100] RDNS failed) by fr-exchange.activnetworks.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 4 Feb 2016 09:39:39 +0100 Subject: Re: ixgbe: Network performance tuning (#TCP connections) To: "Meyer, Wolfgang" , "'freebsd-net@FreeBSD.org'" References: Cc: "'freebsd-performance@FreeBSD.org'" From: Remy Nonnenmacher Message-ID: <56B314F7.3060502@activnetworks.com> Date: Thu, 4 Feb 2016 10:08:07 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 04 Feb 2016 08:39:39.0571 (UTC) FILETIME=[9604B830:01D15F27] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 09:09:18 -0000 On 02/03/16 14:37, Meyer, Wolfgang wrote: > Hello, > > we are evaluating network performance on a DELL-Server (PowerEdge R930 with 4 Sockets, hw.model: Intel(R) Xeon(R) CPU E7-8891 v3 @ 2.80GHz) with 10 GbE-Cards. We use programs that on server side accepts connections on a IP-address+port from the client side and after establishing the connection data is sent in turns between server and client in a predefined pattern (server side sends more data than client side) with sleeps in between the send phases. The test set-up is chosen in such way that every client process initiates 500 connections handled in threads and on the server side each process representing an IP/Port pair also handles 500 connections in threads. > > The number of connections is then increased and the overall network througput is observed using nload. On FreeBSD (on server side) roughly at 50,000 connections errors begin to occur and the overall throughput won't increase further with more connections. With Linux on the server side it is possible to establish more than 120,000 connections and at 50,000 connections the overall throughput ist double that of FreeBSD with the same sending pattern. Furthermore system load on FreeBSD is much higher with 50 % system usage on each core and 80 % interrupt usage on the 8 cores handling the interrupt queues for the NIC. In comparison Linux has <10 % system usage, <10 % user usage and about 15 % interrupt usage on the 16 cores handling the network interrupts for 50,000 connections. > > Varying the numbers for the NIC interrupt queues won't change the performance (rather worsens the situation). Disabling Hyperthreading (utilising 40 cores) degrades the performance. Increasing MAXCPU to utilise all 80 cores won't improve compared to 64 cores, atkbd and uart had to be disabled to avoid kernel panics with increased MAXCPU (thanks to Andre Oppermann for investigating this). Initiallly the tests were made on 10.2 Release, later I switched to 10 Stable (later with ixgbe driver version 3.1.0) but that didn't change the numbers. > > Some sysctl configurables were modified along the network performance guidelines found on the net (e.g. https://calomel.org/freebsd_network_tuning.html, https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html, https://pleiades.ucsc.edu/hyades/FreeBSD_Network_Tuning) but most of them didn't have any measuarable impact. Final sysctl.conf and loader.conf settings see below. Actually the only tunables that provided any improvement were identified to be hw.ix.txd, and hw.ix.rxd that were reduced (!) to the minimum value of 64 and hw.ix.tx_process_limit and hw.ix.rx_process_limit that were set to -1. > > Any ideas what tunables might be changed to get a higher number of TCP connections (it's not a question of the overall throughput as changing the sending pattern allows me to fully utilise the 10Gb bandwidth)? How can I determine where the kernel is spending its time that causes the high CPU load? Any pointers are highly appreciated, I can't believe that there is such a blatant difference in network performance compared to Linux. > > Regards, > Wolfgang > [SNIP] Hi Wolfgang, hwpmc is your friend here if you need to investigate where are your processors wasting their time. Either you will find them contending for network stack (probably the pcb hash table), either they are fighting each other in the scheduler's lock(s) trying to steal jobs from working ones. Also check QPI links activity that may reveal interesting facts about PCI root-complexes geography vs processes locations and migration. You have two options here: Either you persist in using a 4x10 core machine and you will have a long time rearranging stickyness of processes and interrupt to specific cores/packages (Driver, then isr rings, then userland) and police the whole thing (read peacekeeping the riot), either you go to the much simpler solution that is 1 (yes, one) socket machine, fastest available proc with low core (E5-1630v2/3 or 1650) that can handle 10G links hands down out-of-the-box. Also note that there are specific and interesting optimization in the L2 generation on -head that you may want to try if the problem is stack-centered. You may also have a threading problem (userland ones). In the domain of counting instructions per packets (you can practice that with netmap as a wonderfull mean of really 'sensing' what 40Gbps is), threading is bad (and Hyperthreading is evil). Thanks. From owner-freebsd-net@freebsd.org Thu Feb 4 10:23:58 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86428A9BC5A for ; Thu, 4 Feb 2016 10:23:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7757813C0 for ; Thu, 4 Feb 2016 10:23:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u14ANvgl028555 for ; Thu, 4 Feb 2016 10:23:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 203630] [Hyper-V] [nat] [tcp] 10.2 NAT bug in TCP stack or hyperv netsvc driver Date: Thu, 04 Feb 2016 10:23:57 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: franco@opnsense.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: weh@microsoft.com X-Bugzilla-Flags: maintainer-feedback? mfc-stable9? mfc-stable10+ X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 10:23:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203630 --- Comment #28 from Franco Fichtner --- Hello, We've run into this too over at OPNsense. This is a harsh regression from 1= 0.1 to 10.2. It needs an errata for 10.2. Thank you, Franco --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Feb 4 11:39:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD6C0A9B710 for ; Thu, 4 Feb 2016 11:39:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AECE13ED for ; Thu, 4 Feb 2016 11:39:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u14BdtP7009875 for ; Thu, 4 Feb 2016 11:39:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 203630] [Hyper-V] [nat] [tcp] 10.2 NAT bug in TCP stack or hyperv netsvc driver Date: Thu, 04 Feb 2016 11:39:55 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: onyx@netfusion.fr X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: weh@microsoft.com X-Bugzilla-Flags: maintainer-feedback? mfc-stable9? mfc-stable10+ X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 11:39:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203630 --- Comment #29 from Eddy --- Hello everybody, The issue was fixed with patch r291156. I tested it on a clean FreeBSD inst= all by recompiling the kernel in a test environment and it worked. It was merged to the STABLE 10 branch (Fri Dec 18 14:56:49 UTC 2015). I ass= ume that the latest build include the fix, however I'm running 10.2-RELEASE-p12= on my production server but the problem still occurs. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Feb 4 13:50:17 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59D7FA9C6A2; Thu, 4 Feb 2016 13:50:17 +0000 (UTC) (envelope-from honzhan@microsoft.com) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0138.outbound.protection.outlook.com [157.56.110.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A53E51D6; Thu, 4 Feb 2016 13:50:15 +0000 (UTC) (envelope-from honzhan@microsoft.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=KUyW6GgtJ2QiBWtpqKpCa2vK5VTlhJsDOdgaXOcbTM4=; b=lInxb4OiHPqEe41gHOhhHGTeiNQgrvzY+0oCHcix1wDreEXmHber3gL/8mfb/arp4qQNQbLGB7SD8YhmjiqfMq8cp1umHYUHsZUmv8CyMb2My2ZG3QAvgWSaFZL1h+c8sQa176o0wyvT3y4zsBf/ntdrvUr9iQRF5X68CA6fWKQ= Received: from BY2PR03CA007.namprd03.prod.outlook.com (10.255.93.24) by BLUPR0301MB1540.namprd03.prod.outlook.com (10.162.213.158) with Microsoft SMTP Server (TLS) id 15.1.396.15; Thu, 4 Feb 2016 13:35:08 +0000 Received: from BL2FFO11FD029.protection.gbl (10.255.93.4) by BY2PR03CA007.outlook.office365.com (10.255.93.24) with Microsoft SMTP Server (TLS) id 15.1.403.16 via Frontend Transport; Thu, 4 Feb 2016 13:35:07 +0000 Authentication-Results: spf=pass (sender IP is 206.191.228.180) smtp.mailfrom=microsoft.com; hob.de; dkim=none (message not signed) header.d=none;hob.de; dmarc=pass action=none header.from=microsoft.com; Received-SPF: Pass (protection.outlook.com: domain of microsoft.com designates 206.191.228.180 as permitted sender) receiver=protection.outlook.com; client-ip=206.191.228.180; helo=064-smtp-out.microsoft.com; Received: from 064-smtp-out.microsoft.com (206.191.228.180) by BL2FFO11FD029.mail.protection.outlook.com (10.173.160.69) with Microsoft SMTP Server (TLS) id 15.1.409.7 via Frontend Transport; Thu, 4 Feb 2016 13:35:05 +0000 Received: from SG2PR3002MB0106.064d.mgd.msft.net (141.251.56.18) by SG2PR3002MB0107.064d.mgd.msft.net (141.251.56.19) with Microsoft SMTP Server (TLS) id 15.1.403.10; Thu, 4 Feb 2016 13:35:02 +0000 Received: from SG2PR3002MB0106.064d.mgd.msft.net ([141.251.56.18]) by SG2PR3002MB0106.064d.mgd.msft.net ([141.251.56.18]) with mapi id 15.01.0403.016; Thu, 4 Feb 2016 13:35:02 +0000 From: Hongjiang Zhang To: "Meyer, Wolfgang" , "'freebsd-net@FreeBSD.org'" CC: "'freebsd-performance@FreeBSD.org'" Subject: RE: ixgbe: Network performance tuning (#TCP connections) Thread-Topic: ixgbe: Network performance tuning (#TCP connections) Thread-Index: AdFefq3DO+J8ScSLR4uotZFUcfP67wAG7Oyw Date: Thu, 4 Feb 2016 13:35:02 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [180.170.80.203] X-MS-Office365-Filtering-Correlation-Id: f5ac5343-39fd-4d76-0e69-08d32d67fe55 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD029; 1:vUwuqiTdPhCBH/XtzJYIA9Lcwlqcx4J+2VfiOOZbCWa16t//G/SVZY1ge4wvI+b/7quVVogkTm8KTaS747XU/pBPb0umas9S4ItG0wBr5CEfmBYQsZA43ZKdYJLGe8jIVknyO1uS4GIs0vqaKFhLp8tH/+Vx6Jkrk0pSGV3Yj8NBjC5uAMHoX24864ovKA4lsKATLMf4cmjLEc5Ghu1z1n01X+C1RAhoWlEflcGMAkWmj9Y+4pLkxvjwuNUs2gpu12j2mi9At2W0OXJnfIBBVuN1TjA5Htt8MT3SFPZJesugRqC6Ni/hGMpYSytAK+6yCFywU4d+GbnxWNl/rbxWqA== X-Forefront-Antispam-Report: CIP:206.191.228.180; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(2980300002)(438002)(13464003)(199003)(3905003)(189002)(377454003)(16796002)(50466002)(102836003)(23726003)(1220700001)(1096002)(586003)(3846002)(6116002)(24736003)(10290500002)(10400500002)(189998001)(5001960100002)(5005710100001)(5001770100001)(86612001)(106466001)(97756001)(6806005)(5008740100001)(92566002)(19580395003)(46406003)(108616004)(19580405001)(86146001)(11100500001)(47776003)(87936001)(54356999)(575784001)(86362001)(76176999)(2950100001)(33646002)(66066001)(2900100001)(2906002)(4326007)(5003600100002)(15975445007)(50986999)(491001); DIR:OUT; SFP:1102; SCL:1; SRVR:BLUPR0301MB1540; H:064-smtp-out.microsoft.com; FPR:; SPF:Pass; MLV:sfv; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0301MB1540; 2:fTAqvAk/I2b1QK63qoglNfRDa+RFRPq1+cOHwzc74/AqmuNyAypB3oLDIanl/sxQX/0sTg1iiFXsmCLtsr2WcyTcZ8B4gRg9ShZOZURo5feR4QLCt4YDGwj0XW4XP9RmxOtq6qc2IdT9BddoyaErR6QMBUUgkAGnJ0W/+Jj0OaZODhNFeuAd3Db65+DGoCG3; 3:ttkFRxccojWw+0YPG/S6P4geHLpqOpTxjczPntTt5qw1SfZQLU35dSkIcXiSuPAl+gdM/emDwZDXS5dBRTkqZgz1tUyj97HBBogqOxb4CksHa9yIWjGw5tVcXyWbKJNVQIjyuRyPYoQp/g5IqY3TPlI9ybs7Wmyx5ByYBC75ZbK/RrccxI7NuRsFDqDwH6f5IVeuiHpXuVoQ7Yxk/VFCNN0ggCCKA4etuq1pmjSTFoM8Ig4twccfckVqXFHWbvvaezEN3tBQwvTz3SBtCkUP1A==; 25:TZadMu1vzuSyQyHqrEZ4kuPsNeI6Sgaa0Fm19J7S9vGFm8+IkTX1Av+EMGvEeFtRtFE0dcokcoEdFvGsEzhVIexwOPoC4G3oJO0QRBwwMgL2lqgO7x78Ez5uKrWTa9v0Y56vROGex4f76vrdDSpiMWJ1CBP48JGZhJSncP/e+TfCsQI5Q4SAQV61TU/sk46k2FEY4OStE4U8TtS0IFOm9Rkzs0hWnDiWlwsi/2IpJD6MoGX4/QYPA/exgVMsbhHUW+1cSW2UbAyGmFQNFqbveDGcpA8jDGWTT8pUt7i/Nz4YduOsmHCVHwUN0I0N6LW4q0jQef/2NI3DmTHxD2uwTw== X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501002); SRVR:BLUPR0301MB1540; X-O365EOP-Header: O365_EOP: AllowList from IP - set SCL to -1 X-Microsoft-Exchange-Diagnostics: 1; BLUPR0301MB1540; 20:TztFL3ECnvk8h05f4nG4adDZnerx5rWM5Gk4kHCE8lDJsArnq2zAWGNWLwLmQmQYyIrWjHkUQ//HvRZUdTHO0hZO6yiQNKgP05Il41wZk0RiaK06hghnAbIyC270qW0o8WL5A0lEM5QDMK7htD/MqRIQ6xrqA7UjOsmSDrvP9U9QvIZRcLgZA8+LkNLRMKlceuLNNzNFtodguD0R8d1r55DHRPPiVWHEJDavChiiZ8ZxfFb8u13J+ucCuHsG0Nkqr5+Dl9Ssdumi3ql5tYxXhSEamZzH+NTy1JzaqIv/JAPgdkOEspzf5Y3e+aOj09a4k1urDAfwyVz1+7ZvxCdLIlWY2zG7hjXmdq2PxmMrU4+beMylHNNerdSJE6Gw3is0J0Hgzu5NGqwpox+otqzhEeOkmpKWafec8624eHsYndORIOcbGCzI/ESQ8qGp1b2tM82528NG+igOUFKW1Jakn4dVKxM5ZAXUedXarwQlZ7nKZmrvjSWJMY8aL6eSrb+6ro1ExHvQeJsMWAfRS0FuX/K6pgKcws96BsRimTEYgqpNbegF25UITHBJa30RVgjTx6x7NZuKa2Q4KwzV2XBWnhQOq1EtGTtD1af77G71WzU= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(61425038)(601004)(2401047)(5005006)(13016025)(8121501046)(13018025)(10201501046)(3002001)(61426038)(61427038); SRVR:BLUPR0301MB1540; BCL:0; PCL:0; RULEID:; SRVR:BLUPR0301MB1540; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0301MB1540; 4:f36l2FIMlY0gmFvIU/4/RImzjeCOgrW58HHFWCS0ClQfS/PKBZF7lnYLRT8GGBEP/P/zKP6XNhixcwKEYVmt+Squ+9sr1hxwcQfcFNkdWOTrEmaIjgcxuyH8RQErj/Se77KnVcvWdU66TXk3EqrVkQRTj0fcAlAIZgpQm/zYSD/iShmMc0fHb93SNP5Mw0YrIQ3Rk4XgebsxHhET9wTziRPpg6GmP1GJGdIShju+MzFRAFw/WTrCGIoyq56mFOmqd6Mhg5uoaMW1WKa4LC0qgLxqGqujJpOQEG0OJujparwNcx1oaECnB1hEMmsHC7C64vqHbB9fM7Gy16Nmp5Yd0eieQusincmrHHn0Hq15ckOjb9lSx4aYdd+PZC5VBhvuzGw/vl1gqg3Te3B7jNUbRy24/3IQIe6VIxBYLz80T/D9+d/NBc4WQls8FMTJH8SJTcaWzHYpOclXGi91glPY5Q== X-Forefront-PRVS: 084285FC5C X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR0301MB1540; 23:7DVRo5D31+pBpdQgTaqjuJd5ZScbYpVuyromR38?= =?us-ascii?Q?MuQ2X4rCNgulBPoTUwV27n1OABmzNKb2C35KJfDhg3tEkJE0AKTse4UxecN0?= =?us-ascii?Q?UUEH4TBf8aD/+4gTJpsSz6G6Y1SaEpF0bIeFOnVec9TGCr7hYvZy+elf5op/?= =?us-ascii?Q?n+MpizpUG2QW8B9NmKEoO0tHcxx79p/4ZiprSqZQqScqRvFciGTfRHIZaHUG?= =?us-ascii?Q?UNCZvktZvGWjIww0LG2BzgWT2zFsdUUHR+7JyoWHTUTz8yFrtbIW8ZNgD6Ep?= =?us-ascii?Q?D3ET9YOOihhu1moE3luD+H6jlj6+cG4OodMH+OcAyxDqPOLXu0iuflfOxY3Y?= =?us-ascii?Q?mYMcPSJlQaOpkLln+OhPOHugUfrDkOBpnf9Bay6+O6in59YxVEh3rfJWO5vX?= =?us-ascii?Q?VptNAT9POb0wXlwaa1A2f2OAIb7fw4Vp/uuF6gYvRX1FHR4ybR0AfkYlAoNK?= =?us-ascii?Q?PtA7VwFdW/gq6mJN/jrvN2vyoXwh9OVGixInAFwGwrUnZLyV/OYcvIbZrfFD?= =?us-ascii?Q?p6uLQmfiyC/uNwrPkenQPjKXT7f/4HgAwJ7yS5dPazSMwLOJracwtaDdDaRL?= =?us-ascii?Q?38eAAslWepgNnjBbJu/kb5lnuoCletzHJCFJhKKzboGMa3Lz1PT+ssNpmo8l?= =?us-ascii?Q?rjJkPBTKY7M80qbkVO/94TuzMOK6yyPflZfR1zfsJEMcNa8yL8755hHkVDLV?= =?us-ascii?Q?gYvYVkOx/nBaRp9Et4DS0EwHIKePPFJUXoWCJbj4ghh8iS+UL1avv7dgaCL0?= =?us-ascii?Q?IzOK2H2JNsVssAnZi+/gP5HzMPGPh55Ot4IXbjSSJ4CETWIBDKSuIKHuBgBa?= =?us-ascii?Q?61i47BVWV/rXmQL9wAPZj2yGbGCyysuIpYfS60Buw1hgNTOzQxJeybTGDjcZ?= =?us-ascii?Q?sFsy7Xdas0B1ScObeUSkzyYDcZgi5+L6j0xG1DfmwBCnGpWgEQutQhaSng/4?= =?us-ascii?Q?pOPaZfX82qsRPaXF9AFRpGIikwMHqoku73ISLNmo7hFXH7sU/AimbmjhNVjT?= =?us-ascii?Q?3iNac3Xyw1U/4QGI2lmJrCS/lheQgKIMMFY9ImEO9U6sBzBMdLXqirOJuTmh?= =?us-ascii?Q?9WudF1Pc7Z/Pgkdt2spw4D8Y/APp7qmEzUPtkymTQn+k6JG4bcUbNMuHjrHE?= =?us-ascii?Q?2erD+lBkMnKzFXUnciSxV7vNwiQ8OTdFyiSKfrlcbyDkquOan7St3zgjpkej?= =?us-ascii?Q?o0qoMQY+ivLSflYTErJGA9qWX1UO3VZeVXIQCujjotTa94HZy7L5oIF1oY4J?= =?us-ascii?Q?yC2Uk1XOv2wPNSLH/tr/d88upGMn7uLjAHHPwoze8mfn8gPo4kbc7Or4h84G?= =?us-ascii?Q?010ljcaNa5Nl3hVZXKWpZd8Iq7QpR2QvUtJLo524Xvc88?= X-Microsoft-Exchange-Diagnostics: 1; BLUPR0301MB1540; 5:eDsOs3g22CnTop9Z/8cL1SXS8cRoBVrP6Pz7L/c3NCn1+b7r0FEpKJNP8QfRmcRWfzcj8WR64AX8SbDprAkUbGQAE0KTT/3uQEORhw3PcMhGgq9lu2PCT56URc1cx9vMHN2DtJ8dwyQNHyGFYbmA/g==; 24:jj6/YCkK79ROyDQLGNQ8t4ibspl5IgAlY0JLig/OkXBvGN5vSHdAIKV5QrrfB+YAcOM1GygPIJBeThwEoeU59G/KgAKi9k3u+zbytfrGEFA= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Feb 2016 13:35:05.6716 (UTC) X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=72f988bf-86f1-41af-91ab-2d7cd011db47; Ip=[206.191.228.180]; Helo=[064-smtp-out.microsoft.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0301MB1540 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 13:50:17 -0000 Did you enable LRO on FreeBSD side (check 'ifconfig' output)? Linux default= enables GRO (see the output of 'ethtool -k eth0'). Thanks Hongjiang Zhang -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Meyer, Wolfgang Sent: Wednesday, February 3, 2016 9:37 PM To: 'freebsd-net@FreeBSD.org' Cc: 'freebsd-performance@FreeBSD.org' Subject: ixgbe: Network performance tuning (#TCP connections) Hello, we are evaluating network performance on a DELL-Server (PowerEdge R930 with= 4 Sockets, hw.model: Intel(R) Xeon(R) CPU E7-8891 v3 @ 2.80GHz) with 10 Gb= E-Cards. We use programs that on server side accepts connections on a IP-ad= dress+port from the client side and after establishing the connection data = is sent in turns between server and client in a predefined pattern (server = side sends more data than client side) with sleeps in between the send phas= es. The test set-up is chosen in such way that every client process initiat= es 500 connections handled in threads and on the server side each process r= epresenting an IP/Port pair also handles 500 connections in threads. The number of connections is then increased and the overall network througp= ut is observed using nload. On FreeBSD (on server side) roughly at 50,000 c= onnections errors begin to occur and the overall throughput won't increase = further with more connections. With Linux on the server side it is possible= to establish more than 120,000 connections and at 50,000 connections the o= verall throughput ist double that of FreeBSD with the same sending pattern.= Furthermore system load on FreeBSD is much higher with 50 % system usage o= n each core and 80 % interrupt usage on the 8 cores handling the interrupt = queues for the NIC. In comparison Linux has <10 % system usage, <10 % user = usage and about 15 % interrupt usage on the 16 cores handling the network i= nterrupts for 50,000 connections. Varying the numbers for the NIC interrupt queues won't change the performan= ce (rather worsens the situation). Disabling Hyperthreading (utilising 40 c= ores) degrades the performance. Increasing MAXCPU to utilise all 80 cores w= on't improve compared to 64 cores, atkbd and uart had to be disabled to avo= id kernel panics with increased MAXCPU (thanks to Andre Oppermann for inves= tigating this). Initiallly the tests were made on 10.2 Release, later I swi= tched to 10 Stable (later with ixgbe driver version 3.1.0) but that didn't = change the numbers. Some sysctl configurables were modified along the network performance guide= lines found on the net (e.g. https://na01.safelinks.protection.outlook.com/= ?url=3Dhttps%3a%2f%2fcalomel.org%2ffreebsd_network_tuning.html%2c&data=3D01= %7c01%7chonzhan%40064d.mgd.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12= %7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=3DxsMoC%2b1ZcnoHBnPqhLUMDIr8V= LBcLejnrXgkRyDWzYc%3d https://na01.safelinks.protection.outlook.com/?url=3D= https%3a%2f%2fwww.freebsd.org%2fdoc%2fhandbook%2fconfigtuning-kernel-limits= .html%2c&data=3D01%7c01%7chonzhan%40064d.mgd.microsoft.com%7cf827a05328ca4c= a9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=3DXNqvrYfTN= zfe2btrip%2f5FoX3iTTpTSbNrDjbhtVBevo%3d https://na01.safelinks.protection.o= utlook.com/?url=3Dhttps%3a%2f%2fpleiades.ucsc.edu%2fhyades%2fFreeBSD_Networ= k_Tuning&data=3D01%7c01%7chonzhan%40064d.mgd.microsoft.com%7cf827a05328ca4c= a9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=3D%2bQ66X%2= frnqNakX%2fSGcK08QTTrsDjUUWBHOXu6%2fOBIBNQ%3d) but most of them didn't have= any measuarable impact. Final sysctl.conf and loader.conf settings see bel= ow. Actually the only tunables that provided any improvement were identifie= d to be hw.ix.txd, and hw.ix.rxd that were reduced (!) to the minimum value= of 64 and hw.ix.tx_process_limit and hw.ix.rx_process_limit that were set = to -1. Any ideas what tunables might be changed to get a higher number of TCP conn= ections (it's not a question of the overall throughput as changing the send= ing pattern allows me to fully utilise the 10Gb bandwidth)? How can I deter= mine where the kernel is spending its time that causes the high CPU load? A= ny pointers are highly appreciated, I can't believe that there is such a bl= atant difference in network performance compared to Linux. Regards, Wolfgang : cc_htcp_load=3D"YES" hw.ix.txd=3D"64" hw.ix.rxd=3D"64" hw.ix.tx_process_limit=3D"-1" hw.ix.rx_process_limit=3D"-1" hw.ix.num_queues=3D"8" #hw.ix.enable_aim=3D"0" #hw.ix.max_interrupt_rate=3D"31250" #net.isr.maxthreads=3D"16" : kern.ipc.soacceptqueue=3D1024 kern.ipc.maxsockbuf=3D16777216 net.inet.tcp.sendbuf_max=3D16777216 net.inet.tcp.recvbuf_max=3D16777216 net.inet.tcp.tso=3D0 net.inet.tcp.mssdflt=3D1460 net.inet.tcp.minmss=3D1300 net.inet.tcp.nolocaltimewait=3D1 net.inet.tcp.syncache.rexmtlimit=3D0 #net.inet.tcp.syncookies=3D0 net.inet.tcp.drop_synfin=3D1 net.inet.tcp.fast_finwait2_recycle=3D1 net.inet.tcp.icmp_may_rst=3D0 net.inet.tcp.msl=3D5000 net.inet.tcp.path_mtu_discovery=3D0 net.inet.tcp.blackhole=3D1 net.inet.udp.blackhole=3D1 net.inet.tcp.cc.algorithm=3Dhtcp net.inet.tcp.cc.htcp.adaptive_backoff=3D1 net.inet.tcp.cc.htcp.rtt_scaling=3D1 net.inet.ip.forwarding=3D1 net.inet.ip.fastforwarding=3D1 net.inet.ip.rtexpire=3D1 net.inet.ip.rtminexpire=3D1 ________________________________ Follow HOB: - HOB: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%2fww= w.hob.de%2fredirect%2fhob.html&data=3D01%7c01%7chonzhan%40064d.mgd.microsof= t.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011db47= %7c1&sdata=3D%2fenM%2fs72BvfP9H6CnrFeXqhrZoetovqoIMB%2bk0RFfQM%3d - Xing: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%2fw= ww.hob.de%2fredirect%2fxing.html&data=3D01%7c01%7chonzhan%40064d.mgd.micros= oft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011db= 47%7c1&sdata=3DSi0aQmWV%2ba%2fWx5%2f6tZtOWQDj5cmq9t57DhA2h0qsa7Y%3d - LinkedIn: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f= %2fwww.hob.de%2fredirect%2flinkedin.html&data=3D01%7c01%7chonzhan%40064d.mg= d.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d= 7cd011db47%7c1&sdata=3DBc0iX1u2twaazr%2fzi2wuTRqTZOk2rwtu61lNqJrui14%3d - HOBLink Mobile: https://na01.safelinks.protection.outlook.com/?url=3Dhttp= %3a%2f%2fwww.hob.de%2fredirect%2fhoblinkmobile.html&data=3D01%7c01%7chonzha= n%40064d.mgd.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f= 141af91ab2d7cd011db47%7c1&sdata=3DI1ViplWsQ2HMWM%2fLnhWRKseziJBoNlyVBj2wiWF= x1wM%3d - Facebook: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f= %2fwww.hob.de%2fredirect%2ffacebook.html&data=3D01%7c01%7chonzhan%40064d.mg= d.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d= 7cd011db47%7c1&sdata=3D1WliqTPX5YN3AdvK6DzAB7yDkQmjyC3jh%2f47PZ2uU7Y%3d - Twitter: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%= 2fwww.hob.de%2fredirect%2ftwitter.html&data=3D01%7c01%7chonzhan%40064d.mgd.= microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7c= d011db47%7c1&sdata=3DgaLO9OTaBis4F3IGoFY55nwMIOGPaZ0ri%2fK7N7nx7kI%3d - YouTube: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%= 2fwww.hob.de%2fredirect%2fyoutube.html&data=3D01%7c01%7chonzhan%40064d.mgd.= microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7c= d011db47%7c1&sdata=3DT%2fhNogZgLHoTCOj%2fKjsSJhPDlBqyxCAD8tJj5fueiqw%3d - E-Mail: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%2= fwww.hob.de%2fredirect%2fmail.html&data=3D01%7c01%7chonzhan%40064d.mgd.micr= osoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011= db47%7c1&sdata=3DQnWAw2culSVFforLRWYgXfyHaj4PyB4Hn1rj4jVQvjU%3d HOB GmbH & Co. KG Schwadermuehlstr. 3 D-90556 Cadolzburg Geschaeftsfuehrung: Klaus Brandstaetter, Zoran Adamovic AG Fuerth, HRA 5180 Steuer-Nr. 218/163/00107 USt-ID-Nr. DE 132747002 Komplementaerin HOB electronic Beteiligungs GmbH AG Fuerth, HRB 3416 ______= _________________________________________ freebsd-net@freebsd.org mailing list https://na01.safelinks.protection.outlook.com/?url=3Dhttps%3a%2f%2flists.fr= eebsd.org%2fmailman%2flistinfo%2ffreebsd-net&data=3D01%7c01%7chonzhan%40064= d.mgd.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91= ab2d7cd011db47%7c1&sdata=3DGU28hzKU5SSyA3hbP1PNtUNh7G5ut%2fMD6mdRmuJNkEs%3d To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Thu Feb 4 13:50:48 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C128A9C705; Thu, 4 Feb 2016 13:50:48 +0000 (UTC) (envelope-from honzhan@microsoft.com) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bn0107.outbound.protection.outlook.com [157.56.110.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 93C142C3; Thu, 4 Feb 2016 13:50:47 +0000 (UTC) (envelope-from honzhan@microsoft.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=7XURdZm9tz1byzix476KT1rU4mcxIKyGIjlUxC2DDXM=; b=HkoVMy5EBY791q+3lu0JoTmu9DZ7jGwjmWZmSPTTlUCmW1ziKIQGJEI2b1Pb4l9DoI8EO4DUrwR+dvStOqn1dp2vGDGelCyQ2ETx4RCdaNTq8442dfoHumhGIHV4mLhVg+w5td0LcZlZqfm4V8qHBsCYVt4b75YVdofMfEDpN7o= Received: from CH1PR03CA003.namprd03.prod.outlook.com (10.255.156.148) by CY1PR0301MB2089.namprd03.prod.outlook.com (10.164.2.147) with Microsoft SMTP Server (TLS) id 15.1.396.15; Thu, 4 Feb 2016 13:35:15 +0000 Received: from BN1BFFO11FD014.protection.gbl (10.255.156.132) by CH1PR03CA003.outlook.office365.com (10.255.156.148) with Microsoft SMTP Server (TLS) id 15.1.396.15 via Frontend Transport; Thu, 4 Feb 2016 13:35:15 +0000 Authentication-Results: spf=pass (sender IP is 206.191.228.180) smtp.mailfrom=microsoft.com; hob.de; dkim=none (message not signed) header.d=none;hob.de; dmarc=pass action=none header.from=microsoft.com; Received-SPF: Pass (protection.outlook.com: domain of microsoft.com designates 206.191.228.180 as permitted sender) receiver=protection.outlook.com; client-ip=206.191.228.180; helo=064-smtp-out.microsoft.com; Received: from 064-smtp-out.microsoft.com (206.191.228.180) by BN1BFFO11FD014.mail.protection.outlook.com (10.58.144.77) with Microsoft SMTP Server (TLS) id 15.1.409.7 via Frontend Transport; Thu, 4 Feb 2016 13:35:14 +0000 Received: from SG2PR3002MB0106.064d.mgd.msft.net (141.251.56.18) by SG2PR3002MB0105.064d.mgd.msft.net (141.251.56.17) with Microsoft SMTP Server (TLS) id 15.1.403.10; Thu, 4 Feb 2016 13:35:10 +0000 Received: from SG2PR3002MB0106.064d.mgd.msft.net ([141.251.56.18]) by SG2PR3002MB0106.064d.mgd.msft.net ([141.251.56.18]) with mapi id 15.01.0403.016; Thu, 4 Feb 2016 13:35:10 +0000 From: Hongjiang Zhang To: "Meyer, Wolfgang" , "'freebsd-net@FreeBSD.org'" CC: "'freebsd-performance@FreeBSD.org'" Subject: RE: ixgbe: Network performance tuning (#TCP connections) Thread-Topic: ixgbe: Network performance tuning (#TCP connections) Thread-Index: AdFefq3DO+J8ScSLR4uotZFUcfP67wAHKtgg Date: Thu, 4 Feb 2016 13:35:10 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [180.170.80.203] X-MS-Office365-Filtering-Correlation-Id: 178e7cc1-5c93-4225-26e5-08d32d680392 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BN1BFFO11FD014; 1:ntAtAWEK8GlRhOfrLvWWo0iNr05s703QjpHSg8sEVBLHjr4e57tdXWwKPZnyBOu4IAWJafgRZKXA+Kc0lgPysd0SAlKWHJqtX1IEP/XuoaMpZUkyHWCEIB+GssjYU802kAPLGsBjpF9ngaT+Z7EftlMGMb5LG0TXI0hp9kxkWr4m4skRjqH3C7fTmiAS009xo26h/jQ5Xl83swrN3pTUK4GgKEjIHUPJ7SMVAc3CGW8uyKoznTSnhM36ltQwfAnqLyzBZIFIaT6qElRfXZM+RuMqrabDN/J1icxjL62q26I5uqD5n0VodV/w/8sOdD4ehPTq3ZwHxgYuxSqgRRdtGg== X-Forefront-Antispam-Report: CIP:206.191.228.180; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(2980300002)(438002)(377454003)(3905003)(13464003)(199003)(189002)(24736003)(15975445007)(2950100001)(6116002)(97756001)(33646002)(3846002)(102836003)(47776003)(23726003)(2900100001)(92566002)(87936001)(586003)(5001960100002)(5008740100001)(76176999)(66066001)(86146001)(6806005)(1096002)(54356999)(189998001)(11100500001)(1220700001)(16796002)(50986999)(108616004)(4326007)(19580395003)(2906002)(86612001)(5003600100002)(575784001)(86362001)(10400500002)(19580405001)(5005710100001)(50466002)(46406003)(10290500002)(5001770100001)(106466001)(491001); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR0301MB2089; H:064-smtp-out.microsoft.com; FPR:; SPF:Pass; MLV:sfv; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB2089; 2:yzQobEZ8/kdGTNoEndrvh42hCAgw9OjqojMd9No6JDoETWyEzXN5cN6viYGJMNvZWS52dNmqQVwoD9QYjDec/owBLUyhhoIlwYJjWZEi3Y+sT7LTcOxNc/bQ2J9nEhvKs1mIcBNYgqoPZp+USnlWDjFBySR1DyHPRzYb8umCI26voFP1J3Uq2V4YaPM1p2Cv; 3:8UsR44mbp2hsE1Oe9yacrhd3jnqM4Qm5tXT6RMyP2pJZhHiKFx6Wz5dza5snSh1vN8BmK0ECPa7s2ujbi2WAYsv0zwX0Z8VXcBXnV4Pysgn7fiMw7M9GtFMs+QWaEcrmvqP9oKhzmYzcWXRuuZTmhwtJtIVU3ILvtXjvrlND5eI6l1La6yRLziaaCyfeF3PeJp4UqfxhHQzjCdRkiON3OVThaRpsRPvlPZ5l+7yXAYQfeJmKMECe1zFUW0tdI5dW9arDkGv4+7sDG6czouUc8w==; 25:Y2SPeu6ph74MT6oSz9FllACCD5AA3OtY3CXwnNlfJA9lTKo+Xzh7oeXqhJLBh1kIiJWJs6XOipwMEUqypeQPAUNBxTIIwu5npgeFSbMpDNBGlmwZsq0l8kXx8cbky69im27sI5CkevRyjKyCS/7iKK4o+ZcAJ3d/p5LT5WLSDjQfE+T1liLchylh+bCkjXO8JC4Dk0kTEDDL60ZCIDpBAvGqbg4OJv3S+VX57uBS9HpMX/peIgGWEq/SAGw8a9htcCeEYr9PNl9VtJ2qyKIevglClKuFWQu1hHzv8nf7Cpb3Y18Q4Rwc+mASqwQLALCN0Z4fPrCfuo2DiHwHNXAzdA== X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501002); SRVR:CY1PR0301MB2089; X-O365EOP-Header: O365_EOP: AllowList from IP - set SCL to -1 X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB2089; 20:RDCqWzyvNhSg+JxrvoKeHN/kuNwg+r4IzkBstIZFYdxaELRXVHTa5cDcSy0z+f6/P35cOeEpH6wcObwQETOhkK9XRhTotWl+ZS+zlcKtfP0kRt6loATW6QTllcdR6OQnqeTjdyjSIeiKC8DD0yb0DqyUo1qT7Xs3VT0mq4qQZCRFLlbtdE3cbQl81cP/ByAkF7HIM6ufzmBMXcos/w91I8LmyjFJiOOz2PjC0zISL6hrNbORDKKp80ZsyC9zvJ4OLsoj6PTdgwftrrrdNULkZ+k3GC2el+J3gVj1jiH7i+chrou8oZBoPOdyLyk+bzLkbRqcEozC1pp+J40fDm/xlwbYCG8S0X9k5erHEwPXrTyMhDftT3MOzbQvse6WLEw72BxOA1CWQ2Iq35KjHj481cAmkkhWMqXGtPdh4aNtWwuv/2jBZ9G0MovZY7szHXo6aIWtjnmlm175BbI+lL09bSWqzg0slvYIAuodjJSjb7wn4wYZH/a9RFGgiK4lerIUmsIfcZooK+MMJSFqXSJWW2hMql1PSFq2WSLQtZB2rWrmyN9h34PcHgzYldLD+idgzi3QvbPz5LK0UGQuZQxknhKtB1yNEZGJNOUd3aW+0e0= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(61425038)(601004)(2401047)(5005006)(13016025)(8121501046)(13018025)(3002001)(10201501046)(61426038)(61427038); SRVR:CY1PR0301MB2089; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0301MB2089; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB2089; 4:UeVcVSthzafHtLZ93teCM2JvnOVOdqPw/VhKRdM0ePuzJ8tmA3KSNoAdiHdNYqWBOGFWFPRfaT0XqnGMmUVq3QgYVzMjLIXVUVol9mYnzK0B3H+Jsed7Y5by5aaHDUMn1SNcRihsxvvJnFUxMxFG2A3wNupBdBdF16uOW7KLGfWf3tNUcyGffm+1qdfNvnianiSIdE8HWc/y5A548BxLxqEmR8GJmj7SF3qMbdlHGXwXDN57QU1tRx93/G8fffJg2AAjT3JbqAcqhyTWTut5+kdf6ZWVODhz09AbOc+DF1R3xoATuy6ddz0w8ThLESEkWXabFd4rgvrBqT1vxWM7fPjhAu9fhv/+CAbwBeyF9z8f+35h30j9Lfzkb1bf6tE5ORX7xBCYkWaiSJk0msFDhwc/FoUm37dXIbzLlyaHvbJkuru+EcRjGoBhKKdOBXLnH4OamZ/XPzG5U0x9hAPF4g== X-Forefront-PRVS: 084285FC5C X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0301MB2089; 23:wvyZaz/+/NibWBM7DEl1u2pCBdzXgEJ9mIGWjVu?= =?us-ascii?Q?wIPmDK5dpxOJ4xUKtNEgPLpZTAn7N7naBpUsyxDzFddfc18qhvYb20FdKRb2?= =?us-ascii?Q?MMcKjvXElHYlasrRnH8l1a2tba+IJrkrTjCKPNCV+FsWxCSAArfKxRZrkP8V?= =?us-ascii?Q?6xYQTP2/niSkQsnOtPDAVGaBzcHSCohWHc0oBGTEpOHrREpuChFVAClv7Rbi?= =?us-ascii?Q?3fl8W5swdLLSZm79213eZpMRJayD/bhLePM6Oqd0zy0r4baMUME20+616PIC?= =?us-ascii?Q?8ZEKGmQX9AaoxAXvLxHiCm2VqV00IZjL6+ShqIapzae4Ka/3DXakIU9OiDdN?= =?us-ascii?Q?QkJn9f5ZzwlAZCOaz/6msyJAh9HWupyND02OCSjqaRgKJ2IhEYO+3W9uwq30?= =?us-ascii?Q?LQXYY1Dfp6OOnIMy8+awXNodLHcFNQN4KTm2GwAeGb/MqIM7QOWuACon9HFS?= =?us-ascii?Q?S33pLoNec/tKqUiZyelR5DhdKzp2DRfZem6CnQJgUgRnpVnE64IkSFNyTOCG?= =?us-ascii?Q?a/HAQ0d6yBm++eRJHwHoSu1UMcBLBhE5qjfQufv0/KrstaYRlgyqcwMWtqHG?= =?us-ascii?Q?sRScFkESOLGcTbHsVlJZfLQ0ODx2bUgaDSy9oZbzQA5P+ekfUBXdq8CCI9m3?= =?us-ascii?Q?YzeuMFyvcSBzh0Z1A6bF8TZ4QmmGX/HYchRWlJwp8GCUjSJuPW92VnhV/F+y?= =?us-ascii?Q?6SV37hE3eEw89cO1cmaf663ZUTbtjmed2Gmtl8gy17Zr1TCs5OKCNq7K7bYZ?= =?us-ascii?Q?sxE+93I2t+gOWJxx05RDSRo7JfaxvMn3EyCDQHZPxyGj5kD21R2o/m6ixBvT?= =?us-ascii?Q?QL2mSYd8RzcCPxF14p0T02uO3sWE30KgfbkXO1WgxsqQMQ8Jt7jOJtrOHAlE?= =?us-ascii?Q?vzkn3GSipYy6C5lFn4FEn68ZgYkqc6H2LtZnHAMs10iPzkC13YENQzu/Qcel?= =?us-ascii?Q?5I/leoCr6QXPc27V6HiZPUY0MzwbeHK8eekZARK45AbF+SGLEeqWftxs1F5h?= =?us-ascii?Q?JkIqkpctTcEuUrnLJIcwYrNzc/QxMt1Np2UvaHNVZ3CKapE6WzqHW4OUZmZq?= =?us-ascii?Q?TqRMVg6Jqmfs7AAWkWQco1boz1GiH639h0UYiHR0VtCaTvEGzHOSDozyXBlz?= =?us-ascii?Q?oBjA/OSUsYm1kaVJoSk1t88u6JTfQfNyiZqeoosqYxH2vcfYvBfwYyU7v01G?= =?us-ascii?Q?n9U0SFIofLzkdSNgTr/w6dziwV0lGFKzkGGTqv4lHHvQwX+ZlMWICYg3Ws+P?= =?us-ascii?Q?E6HFdM5rRvQButcVBwnvfIbdb8Dp5OethJclbaVdi5ehsPiWyBrZmpZJOkvC?= =?us-ascii?Q?0rrZyDvxp1wmod04DGctCVc2SHAkQzZ6la/f8iBCNrN+E?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0301MB2089; 5:MEFAOzt/1NyF4xEh1bB9ouM0AptaVEQijN2ngECnXT6oyBZ8nRXs5Xux4OF75JrF8s/SpgWCGx7CFrjr4L/AA5Xj2gMwKryqJkPrvuB2iYI8/WO23rFsTAiQbI0nPW9iONFV4sH4mIyPXeJ/zOIwVA==; 24:BhMQ+p0Jz9GLDZy9zSIv2ko53fI2S30viC6Vpx5DrIFu6U6R8SWZHVx1aOMJbWo1Hoe29K8d0wIPxhydVA9J7aaxi4lTzY+nB8cQRxue8+c= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Feb 2016 13:35:14.4450 (UTC) X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=72f988bf-86f1-41af-91ab-2d7cd011db47; Ip=[206.191.228.180]; Helo=[064-smtp-out.microsoft.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0301MB2089 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 13:50:48 -0000 Please check whether LRO is enabled on your FreeBSD server with "ifconfig".= Linux default enables GRO (see the output of 'ethtool -k eth0'), which cov= ers LRO optimization. Thanks Hongjiang Zhang -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Meyer, Wolfgang Sent: Wednesday, February 3, 2016 9:37 PM To: 'freebsd-net@FreeBSD.org' Cc: 'freebsd-performance@FreeBSD.org' Subject: ixgbe: Network performance tuning (#TCP connections) Hello, we are evaluating network performance on a DELL-Server (PowerEdge R930 with= 4 Sockets, hw.model: Intel(R) Xeon(R) CPU E7-8891 v3 @ 2.80GHz) with 10 Gb= E-Cards. We use programs that on server side accepts connections on a IP-ad= dress+port from the client side and after establishing the connection data = is sent in turns between server and client in a predefined pattern (server = side sends more data than client side) with sleeps in between the send phas= es. The test set-up is chosen in such way that every client process initiat= es 500 connections handled in threads and on the server side each process r= epresenting an IP/Port pair also handles 500 connections in threads. The number of connections is then increased and the overall network througp= ut is observed using nload. On FreeBSD (on server side) roughly at 50,000 c= onnections errors begin to occur and the overall throughput won't increase = further with more connections. With Linux on the server side it is possible= to establish more than 120,000 connections and at 50,000 connections the o= verall throughput ist double that of FreeBSD with the same sending pattern.= Furthermore system load on FreeBSD is much higher with 50 % system usage o= n each core and 80 % interrupt usage on the 8 cores handling the interrupt = queues for the NIC. In comparison Linux has <10 % system usage, <10 % user = usage and about 15 % interrupt usage on the 16 cores handling the network i= nterrupts for 50,000 connections. Varying the numbers for the NIC interrupt queues won't change the performan= ce (rather worsens the situation). Disabling Hyperthreading (utilising 40 c= ores) degrades the performance. Increasing MAXCPU to utilise all 80 cores w= on't improve compared to 64 cores, atkbd and uart had to be disabled to avo= id kernel panics with increased MAXCPU (thanks to Andre Oppermann for inves= tigating this). Initiallly the tests were made on 10.2 Release, later I swi= tched to 10 Stable (later with ixgbe driver version 3.1.0) but that didn't = change the numbers. Some sysctl configurables were modified along the network performance guide= lines found on the net (e.g. https://na01.safelinks.protection.outlook.com/= ?url=3Dhttps%3a%2f%2fcalomel.org%2ffreebsd_network_tuning.html%2c&data=3D01= %7c01%7chonzhan%40064d.mgd.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12= %7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=3DxsMoC%2b1ZcnoHBnPqhLUMDIr8V= LBcLejnrXgkRyDWzYc%3d https://na01.safelinks.protection.outlook.com/?url=3D= https%3a%2f%2fwww.freebsd.org%2fdoc%2fhandbook%2fconfigtuning-kernel-limits= .html%2c&data=3D01%7c01%7chonzhan%40064d.mgd.microsoft.com%7cf827a05328ca4c= a9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=3DXNqvrYfTN= zfe2btrip%2f5FoX3iTTpTSbNrDjbhtVBevo%3d https://na01.safelinks.protection.o= utlook.com/?url=3Dhttps%3a%2f%2fpleiades.ucsc.edu%2fhyades%2fFreeBSD_Networ= k_Tuning&data=3D01%7c01%7chonzhan%40064d.mgd.microsoft.com%7cf827a05328ca4c= a9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=3D%2bQ66X%2= frnqNakX%2fSGcK08QTTrsDjUUWBHOXu6%2fOBIBNQ%3d) but most of them didn't have= any measuarable impact. Final sysctl.conf and loader.conf settings see bel= ow. Actually the only tunables that provided any improvement were identifie= d to be hw.ix.txd, and hw.ix.rxd that were reduced (!) to the minimum value= of 64 and hw.ix.tx_process_limit and hw.ix.rx_process_limit that were set = to -1. Any ideas what tunables might be changed to get a higher number of TCP conn= ections (it's not a question of the overall throughput as changing the send= ing pattern allows me to fully utilise the 10Gb bandwidth)? How can I deter= mine where the kernel is spending its time that causes the high CPU load? A= ny pointers are highly appreciated, I can't believe that there is such a bl= atant difference in network performance compared to Linux. Regards, Wolfgang : cc_htcp_load=3D"YES" hw.ix.txd=3D"64" hw.ix.rxd=3D"64" hw.ix.tx_process_limit=3D"-1" hw.ix.rx_process_limit=3D"-1" hw.ix.num_queues=3D"8" #hw.ix.enable_aim=3D"0" #hw.ix.max_interrupt_rate=3D"31250" #net.isr.maxthreads=3D"16" : kern.ipc.soacceptqueue=3D1024 kern.ipc.maxsockbuf=3D16777216 net.inet.tcp.sendbuf_max=3D16777216 net.inet.tcp.recvbuf_max=3D16777216 net.inet.tcp.tso=3D0 net.inet.tcp.mssdflt=3D1460 net.inet.tcp.minmss=3D1300 net.inet.tcp.nolocaltimewait=3D1 net.inet.tcp.syncache.rexmtlimit=3D0 #net.inet.tcp.syncookies=3D0 net.inet.tcp.drop_synfin=3D1 net.inet.tcp.fast_finwait2_recycle=3D1 net.inet.tcp.icmp_may_rst=3D0 net.inet.tcp.msl=3D5000 net.inet.tcp.path_mtu_discovery=3D0 net.inet.tcp.blackhole=3D1 net.inet.udp.blackhole=3D1 net.inet.tcp.cc.algorithm=3Dhtcp net.inet.tcp.cc.htcp.adaptive_backoff=3D1 net.inet.tcp.cc.htcp.rtt_scaling=3D1 net.inet.ip.forwarding=3D1 net.inet.ip.fastforwarding=3D1 net.inet.ip.rtexpire=3D1 net.inet.ip.rtminexpire=3D1 ________________________________ Follow HOB: - HOB: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%2fww= w.hob.de%2fredirect%2fhob.html&data=3D01%7c01%7chonzhan%40064d.mgd.microsof= t.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011db47= %7c1&sdata=3D%2fenM%2fs72BvfP9H6CnrFeXqhrZoetovqoIMB%2bk0RFfQM%3d - Xing: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%2fw= ww.hob.de%2fredirect%2fxing.html&data=3D01%7c01%7chonzhan%40064d.mgd.micros= oft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011db= 47%7c1&sdata=3DSi0aQmWV%2ba%2fWx5%2f6tZtOWQDj5cmq9t57DhA2h0qsa7Y%3d - LinkedIn: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f= %2fwww.hob.de%2fredirect%2flinkedin.html&data=3D01%7c01%7chonzhan%40064d.mg= d.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d= 7cd011db47%7c1&sdata=3DBc0iX1u2twaazr%2fzi2wuTRqTZOk2rwtu61lNqJrui14%3d - HOBLink Mobile: https://na01.safelinks.protection.outlook.com/?url=3Dhttp= %3a%2f%2fwww.hob.de%2fredirect%2fhoblinkmobile.html&data=3D01%7c01%7chonzha= n%40064d.mgd.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f= 141af91ab2d7cd011db47%7c1&sdata=3DI1ViplWsQ2HMWM%2fLnhWRKseziJBoNlyVBj2wiWF= x1wM%3d - Facebook: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f= %2fwww.hob.de%2fredirect%2ffacebook.html&data=3D01%7c01%7chonzhan%40064d.mg= d.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d= 7cd011db47%7c1&sdata=3D1WliqTPX5YN3AdvK6DzAB7yDkQmjyC3jh%2f47PZ2uU7Y%3d - Twitter: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%= 2fwww.hob.de%2fredirect%2ftwitter.html&data=3D01%7c01%7chonzhan%40064d.mgd.= microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7c= d011db47%7c1&sdata=3DgaLO9OTaBis4F3IGoFY55nwMIOGPaZ0ri%2fK7N7nx7kI%3d - YouTube: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%= 2fwww.hob.de%2fredirect%2fyoutube.html&data=3D01%7c01%7chonzhan%40064d.mgd.= microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7c= d011db47%7c1&sdata=3DT%2fhNogZgLHoTCOj%2fKjsSJhPDlBqyxCAD8tJj5fueiqw%3d - E-Mail: https://na01.safelinks.protection.outlook.com/?url=3Dhttp%3a%2f%2= fwww.hob.de%2fredirect%2fmail.html&data=3D01%7c01%7chonzhan%40064d.mgd.micr= osoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91ab2d7cd011= db47%7c1&sdata=3DQnWAw2culSVFforLRWYgXfyHaj4PyB4Hn1rj4jVQvjU%3d HOB GmbH & Co. KG Schwadermuehlstr. 3 D-90556 Cadolzburg Geschaeftsfuehrung: Klaus Brandstaetter, Zoran Adamovic AG Fuerth, HRA 5180 Steuer-Nr. 218/163/00107 USt-ID-Nr. DE 132747002 Komplementaerin HOB electronic Beteiligungs GmbH AG Fuerth, HRB 3416 ______= _________________________________________ freebsd-net@freebsd.org mailing list https://na01.safelinks.protection.outlook.com/?url=3Dhttps%3a%2f%2flists.fr= eebsd.org%2fmailman%2flistinfo%2ffreebsd-net&data=3D01%7c01%7chonzhan%40064= d.mgd.microsoft.com%7cf827a05328ca4ca9781608d32c9f5b12%7c72f988bf86f141af91= ab2d7cd011db47%7c1&sdata=3DGU28hzKU5SSyA3hbP1PNtUNh7G5ut%2fMD6mdRmuJNkEs%3d To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Thu Feb 4 14:26:36 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F77DA9B9C4 for ; Thu, 4 Feb 2016 14:26:36 +0000 (UTC) (envelope-from dudu.meyer@gmail.com) Received: from mail-ob0-x22b.google.com (mail-ob0-x22b.google.com [IPv6:2607:f8b0:4003:c01::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1F0721E5 for ; Thu, 4 Feb 2016 14:26:36 +0000 (UTC) (envelope-from dudu.meyer@gmail.com) Received: by mail-ob0-x22b.google.com with SMTP id ba1so66079910obb.3 for ; Thu, 04 Feb 2016 06:26:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=u6opQ2ruhpE4UFnxX5Cp6y8893fR6zmC3V8ei2f5Cp4=; b=HuY0VqhjiEBRa0ZEw9OGW5lSM8pXDFgl1xOmKL5xVKg4rJpO0zF/UTBHvODDUA/Iae 5Im/0ww04rGiV/mUPgfkQyCJGn1zU5QVIg0HE5O/La+HP9R99W/w91TjgYIrC1gRQfNA Ei6wNiKnONR/N+r1UFYRsH5roVO0kK+zO74Owbpx2XdT4r0M9ibWyDSYTo+bFlY1fkRG PAUWti88UVkMSq7JzLV6qjXn6OqHShXK5tIYnKBuj/bSTn3GAY3fZg3Y66x1wxBsdkHj cFKXD0B9exsfnYCbAFBMFpLDzWEshptWyP5PWbP4qh9nESIn98IdDTyu4CdskV26b9Of wPwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=u6opQ2ruhpE4UFnxX5Cp6y8893fR6zmC3V8ei2f5Cp4=; b=Imyr+A3cRHF6UURts7XSWCHKo0jbq1+KazwcEJ8rAUmUt1kyGc4zMZndUFRzDdZ6E0 hR6Kz/u5gBG5K4GOM2NKS8YDd+ogOvUodnJHIehUcjeOt9Mif3WDTD3CwJAvwFBdsrxA xM8jkb4QEua5T57/Py4n8TEnuR9+6nHCiIjzm2CpT5uiYwMOFPROHVCCq5NfNPS52cVM f6Tk4fzYx8qOpgov9seRk2hFq7katN6xbtcvNZ9xljobhPT0RTnZTizkT7kbQPUNAPex ubZNyH0ckDtFmgreL3NtqgB061lKBP2xqKi+XItFvAewvvQlYsuSMe+vXxBY5FIzjSF1 otDA== X-Gm-Message-State: AG10YORDfvU7KvE7h6kAnXWeitJrr0PfKRdWEi1jQPI5fjHtzl/enJPZnPlrr1he+dBIqSbu4db8hrimtLIuIA== MIME-Version: 1.0 X-Received: by 10.60.81.67 with SMTP id y3mr7963205oex.61.1454595995222; Thu, 04 Feb 2016 06:26:35 -0800 (PST) Received: by 10.182.88.138 with HTTP; Thu, 4 Feb 2016 06:26:35 -0800 (PST) Date: Thu, 4 Feb 2016 12:26:35 -0200 Message-ID: Subject: dev.netmap.buf_size and packett size from host From: Eduardo Meyer To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 14:26:36 -0000 hello, I have a netmap application which has host mode bridge/fwd, with default settings I have the following error some often: 884.260394 [2950] netmap_transmit igb1 from_host, drop packet size 2962 > 2048 the only application which relies on host mode is bird, so those packets are probably from bird daemon, when I get those errors I get bird sessions failing and restart I raised dev.netmap.buf_size to 5000 it ajusted to 5120, things got better but I still have logs: netmap_transmit igb1 from_host, drop packet size 5858 > 5120 Now the main question is, when dev.netmap.buf_size is 2048 the application uses 1.3G of RAM but when I raise to 5120 it uses 3G of RAM. So I need to understand, is this packet size really related from what I get from the application packets coming from host to netmap? If so can I allow for bigger sizes, like 16k (lo0 mtu) without pre-alloc so much more RAM? thank you E. Meyer From owner-freebsd-net@freebsd.org Thu Feb 4 14:30:00 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 971BFA9BB15 for ; Thu, 4 Feb 2016 14:30:00 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lf0-x235.google.com (mail-lf0-x235.google.com [IPv6:2a00:1450:4010:c07::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FCDC3C6 for ; Thu, 4 Feb 2016 14:30:00 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-lf0-x235.google.com with SMTP id l143so37232683lfe.2 for ; Thu, 04 Feb 2016 06:30:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=JBMz0jfe+qO0bOqmAc2pXJC9gHjTEEKnoQXlsKoeOgo=; b=b2uyH/pzmapLgExRb9qrdQgFt4hpTmRKlF3kZhxUT1oo6/mBEKnuHF1+/J31xpMk/v /S99JPZRmfmKqFso3cEzyGJanZ6h0UQOBkM8DrAW2h9pElM+K8L8+9NmAyDEVGISkR0C LYR6dt0gr31K84pmXJga3Vnf2ScvFYoL7kfxx20UXgHLggF1s8qJrZXNiYfyhMJ5q3Cm XoDbW5OpOuLuyg1kYU+5lEWjZkcLBNQ9jO14Pt0LZrjVeChngljfdJy5qgVbz2ftZ0su Hzx1pO5Y65S1FMtYQcf3W3mO36n7YagXGL8pYnDu1lhhmxpGgmMhSdWEV/7nf1gP/1q7 3ysQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=JBMz0jfe+qO0bOqmAc2pXJC9gHjTEEKnoQXlsKoeOgo=; b=OWIwZIpvXudl21zz0h3N1p2Eo2LZE1XKtkerSqjj7G4/f8L7759aNnDK9CtWGRdVrC waD69hTBMaehb2GPZYC3WtM2VzzzjVjLD1ZZnQhpWA+dEYQI5zUuQw0bXH3Z2PdOTOq6 V0lDo2eDu/Z0ZhRvTHzfXTGJoRWB81NaaaO41/MPo5JnOnYpqZO7qAYS9vM8NeauV4NX JnfD0gwU09U1vM8/U5tnRfWJ20bPzYhaTbt4UjSwN6malefNTtXc9Omj9EFXQHz5nRZq EERmpQG9ftDkQFwG8VGnqeGjCCPa8BmzORsKHD0T2ukYfsMcOKKCENyPl4lLHf+zISzW G0kQ== X-Gm-Message-State: AG10YOT8+B4GVS7f98bCqngdpnCqk06GrfnRZSFmWCfoX4lzWR3Dq1S51/4N6C//9gw6GKLXOBYKlIy10BSnXA== MIME-Version: 1.0 X-Received: by 10.25.29.193 with SMTP id d184mr3605046lfd.28.1454596198163; Thu, 04 Feb 2016 06:29:58 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.4.232 with HTTP; Thu, 4 Feb 2016 06:29:58 -0800 (PST) In-Reply-To: References: Date: Thu, 4 Feb 2016 15:29:58 +0100 X-Google-Sender-Auth: YkaHQcClb0UV-Dx8FejPGgVv_xs Message-ID: Subject: Re: dev.netmap.buf_size and packett size from host From: Luigi Rizzo To: Eduardo Meyer Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 14:30:00 -0000 Make sure you disable TSO on the interface used in netmap mode, and then check that you use an MTU of 1500 on that interface. You should not receive frames larger than MTU coming from the host in these conditions. cheers luigi On Thu, Feb 4, 2016 at 3:26 PM, Eduardo Meyer wrote: > hello, > > I have a netmap application which has host mode bridge/fwd, with default > settings I have the following error some often: > > 884.260394 [2950] netmap_transmit igb1 from_host, drop packet > size 2962 > 2048 > > the only application which relies on host mode is bird, so those packets > are probably from bird daemon, when I get those errors I get bird sessions > failing and restart > > I raised dev.netmap.buf_size to 5000 it ajusted to 5120, things got better > but I still have logs: > > netmap_transmit igb1 from_host, drop packet size 5858 > 5120 > > Now the main question is, when dev.netmap.buf_size is 2048 the application > uses 1.3G of RAM but when I raise to 5120 it uses 3G of RAM. > > So I need to understand, is this packet size really related from what I get > from the application packets coming from host to netmap? If so can I allow > for bigger sizes, like 16k (lo0 mtu) without pre-alloc so much more RAM? > > thank you > > E. Meyer > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@freebsd.org Thu Feb 4 14:34:09 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF4DDA9BEC2 for ; Thu, 4 Feb 2016 14:34:09 +0000 (UTC) (envelope-from dudu.meyer@gmail.com) Received: from mail-ob0-x231.google.com (mail-ob0-x231.google.com [IPv6:2607:f8b0:4003:c01::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 94E59C46 for ; Thu, 4 Feb 2016 14:34:09 +0000 (UTC) (envelope-from dudu.meyer@gmail.com) Received: by mail-ob0-x231.google.com with SMTP id xk3so66032903obc.2 for ; Thu, 04 Feb 2016 06:34:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=z4HVDqOMTB92mmYOJwz1Tb8imaPh2yDwDQSbHQpOR48=; b=pAVwt7sUOv3PNJLlg36YdrwrDHI/1kqebNNXS6aJlEjiIfYSGV4YsdOz/yY3dkciGt LxAK/Hj38ojsroXiNkI0WlB+BYgqpRlXktJNlVJVdhJamvq5U8nHUYt9g1sNh2ab3kbw 78tixurSKMzWlx9ez02QoH73fE6txXBGr41PTj8i2I8hDKG7kdsRP6T206H83KyLGWL1 pjrp9mlTdBo7861SQqDlBauKDnv3RIEYmFmpX7x+fVT+XhKVARdSlAczFhr4UYZ6B74Z bGvlaxoUzuvtE7gU4vi2lPTfiIwA/f3gc4biBCJKSwH9ZS3wrVDpEde7kB4IzBwqIKO6 BM6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=z4HVDqOMTB92mmYOJwz1Tb8imaPh2yDwDQSbHQpOR48=; b=a+C2CKCHMikpptuo16MVXjT7B5K4OIsKsDDVVoIv8FqIN/rzLP06sUUOmx3UWYVssT sZ7rrBIu1WjZm30tcEYdc44YX1DnXUKsmF8Q8sMH0rXmIA4lTKNIDDiGoCtrRbWeEIu2 8+M+JAJ175NeC+q6mvOZ1YaAKgCCE3WM5+YZ628kxasjEvHa49MtUO/tusst+/CPs5fU ybVWSRNs2l4njm24vzid52LAwnQtwxKqUipFGj4pGxA96NzxPJvpV70E2qIdZ+o3vlQv ByikURQj9sRFf5r90zSMtb6ZIqxyW0FU1FKOhqtXcZ5j/dDfVyfloQQgxdxn2gaaxs/E CNyQ== X-Gm-Message-State: AG10YOQO19D1biKTVXc68UxYLslGeVU+ByDXuphScQsMUQpxUaKMachL+go0X9JdNINqPq5gH5oOMPG+syBT1g== MIME-Version: 1.0 X-Received: by 10.60.117.102 with SMTP id kd6mr7544520oeb.73.1454596448769; Thu, 04 Feb 2016 06:34:08 -0800 (PST) Received: by 10.182.88.138 with HTTP; Thu, 4 Feb 2016 06:34:08 -0800 (PST) In-Reply-To: References: Date: Thu, 4 Feb 2016 12:34:08 -0200 Message-ID: Subject: Re: dev.netmap.buf_size and packett size from host From: Eduardo Meyer To: Luigi Rizzo Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 14:34:09 -0000 mtu is good, TSO was on, thank you will retest right now. which other port features should I disable? I only disabled txcsum and rxcsum before, now tso on the list, anything else in netmap mode? On Thu, Feb 4, 2016 at 12:29 PM, Luigi Rizzo wrote: > Make sure you disable TSO on the interface used in netmap > mode, and then check that you use an MTU of 1500 on that > interface. > You should not receive frames larger than MTU coming from > the host in these conditions. > > cheers > luigi > > > On Thu, Feb 4, 2016 at 3:26 PM, Eduardo Meyer > wrote: > > hello, > > > > I have a netmap application which has host mode bridge/fwd, with default > > settings I have the following error some often: > > > > 884.260394 [2950] netmap_transmit igb1 from_host, drop packet > > size 2962 > 2048 > > > > the only application which relies on host mode is bird, so those packets > > are probably from bird daemon, when I get those errors I get bird > sessions > > failing and restart > > > > I raised dev.netmap.buf_size to 5000 it ajusted to 5120, things got > better > > but I still have logs: > > > > netmap_transmit igb1 from_host, drop packet size 5858 > 5120 > > > > Now the main question is, when dev.netmap.buf_size is 2048 the > application > > uses 1.3G of RAM but when I raise to 5120 it uses 3G of RAM. > > > > So I need to understand, is this packet size really related from what I > get > > from the application packets coming from host to netmap? If so can I > allow > > for bigger sizes, like 16k (lo0 mtu) without pre-alloc so much more RAM? > > > > thank you > > > > E. Meyer > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2217533 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > -- =========== Eduardo Meyer pessoal: dudu.meyer@gmail.com profissional: ddm.farmaciap@saude.gov.br From owner-freebsd-net@freebsd.org Thu Feb 4 14:35:11 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5450A9BFBC for ; Thu, 4 Feb 2016 14:35:11 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-lb0-x229.google.com (mail-lb0-x229.google.com [IPv6:2a00:1450:4010:c04::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C5A4D33 for ; Thu, 4 Feb 2016 14:35:11 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: by mail-lb0-x229.google.com with SMTP id dx2so32237943lbd.3 for ; Thu, 04 Feb 2016 06:35:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+qDE0L8qqMx0cAY8ZV8Y/KJVivDiJvvoXnG7N4eqIpI=; b=nmJZ1EnA+ZpIO228OK97aKBZhYvZPo1n8pVBNwJwqDAICOMHaJe83H+7PoNIf2D2jn 8PQ/Evad+KBANDaq0Dha7wyQW6rdJ5zpAvNCEvwOA+N6fe3tjB9+X6izr6NaG7pUarw5 pUTydaFlsvEQ5PWhdNgVTTUR6+rRgex7PNypt47PYxANBH0O2r2Ksrih8X4xJ53wzw0N KMJ109OBcjJ2JHImmmHdY6FekKfUkNSwvryG+2P4+2HmhaVg6GmOxQmIYmo74h2zIrK9 qUQS1Wf6biByOBnLC6Gjnq/QVvqUNSv8HHOyoWKt659hzVVO0O0J0yZQW3t/JjM0asbj Z5kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=+qDE0L8qqMx0cAY8ZV8Y/KJVivDiJvvoXnG7N4eqIpI=; b=FhxxlXZPKUK7nHyyTeNdtclTauVtpGfF4DY80qmXppbKeSaF/PFmMuJENb3221m5f1 eOSoi8U3gC3IqtBf0dFD5rSAW+i5YVbMUz7Amb7JlSiClaTeRRYVojayt7ZLnTXRtppS M/sq3Qg9/GH25qIgdjYpyX+l8Yt1HbMbD3mzumGEcuzDw8nIiZQYxYGoqCZy7FEEmicH EQf47Ur6n+QRUF4zo8AfLN6ObGAPW9V3d/xxqKnZPWGOJEhO6Vh85i+GQ0zPUHz37IB9 NAfMChnzJlbxRhULrTXNuULwaKeOP9nkIYHXhsQAJ7Zr64D7u9jHG/hgCy5sZCOKLqaB 1LeQ== X-Gm-Message-State: AG10YOSCW0cwa9pDX78drLzQjh75sXf23V2AJ9Ugdc8ClvfpfIs3HmnxYAvZzeSk9PIhgd1HqdXZFl4ANeuPDw== MIME-Version: 1.0 X-Received: by 10.112.13.165 with SMTP id i5mr3097639lbc.79.1454596509415; Thu, 04 Feb 2016 06:35:09 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.114.4.232 with HTTP; Thu, 4 Feb 2016 06:35:09 -0800 (PST) In-Reply-To: References: Date: Thu, 4 Feb 2016 15:35:09 +0100 X-Google-Sender-Auth: N33EkyqTJlKazDdjdgNOohY47N4 Message-ID: Subject: Re: dev.netmap.buf_size and packett size from host From: Luigi Rizzo To: Eduardo Meyer Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 14:35:12 -0000 disable all hardware accelerations when using netmap. cheers luigi On Thu, Feb 4, 2016 at 3:34 PM, Eduardo Meyer wrote: > mtu is good, TSO was on, thank you will retest right now. > > which other port features should I disable? I only disabled txcsum and > rxcsum before, now tso on the list, anything else in netmap mode? > > On Thu, Feb 4, 2016 at 12:29 PM, Luigi Rizzo wrote: >> >> Make sure you disable TSO on the interface used in netmap >> mode, and then check that you use an MTU of 1500 on that >> interface. >> You should not receive frames larger than MTU coming from >> the host in these conditions. >> >> cheers >> luigi >> >> >> On Thu, Feb 4, 2016 at 3:26 PM, Eduardo Meyer >> wrote: >> > hello, >> > >> > I have a netmap application which has host mode bridge/fwd, with default >> > settings I have the following error some often: >> > >> > 884.260394 [2950] netmap_transmit igb1 from_host, drop packet >> > size 2962 > 2048 >> > >> > the only application which relies on host mode is bird, so those packets >> > are probably from bird daemon, when I get those errors I get bird >> > sessions >> > failing and restart >> > >> > I raised dev.netmap.buf_size to 5000 it ajusted to 5120, things got >> > better >> > but I still have logs: >> > >> > netmap_transmit igb1 from_host, drop packet size 5858 > 5120 >> > >> > Now the main question is, when dev.netmap.buf_size is 2048 the >> > application >> > uses 1.3G of RAM but when I raise to 5120 it uses 3G of RAM. >> > >> > So I need to understand, is this packet size really related from what I >> > get >> > from the application packets coming from host to netmap? If so can I >> > allow >> > for bigger sizes, like 16k (lo0 mtu) without pre-alloc so much more RAM? >> > >> > thank you >> > >> > E. Meyer >> > _______________________________________________ >> > freebsd-net@freebsd.org mailing list >> > https://lists.freebsd.org/mailman/listinfo/freebsd-net >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> >> >> -- >> -----------------------------------------+------------------------------- >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> TEL +39-050-2217533 . via Diotisalvi 2 >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> -----------------------------------------+------------------------------- > > > > > -- > =========== > Eduardo Meyer > pessoal: dudu.meyer@gmail.com > profissional: ddm.farmaciap@saude.gov.br -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2217533 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-net@freebsd.org Thu Feb 4 15:57:30 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C87A5A9BE14 for ; Thu, 4 Feb 2016 15:57:30 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id B3AA31F16 for ; Thu, 4 Feb 2016 15:57:30 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id B19BF1070E6; Thu, 4 Feb 2016 15:57:30 +0000 (UTC) Date: Thu, 4 Feb 2016 15:57:30 +0000 To: freebsd-net@freebsd.org From: "gallatin (Andrew Gallatin)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Updated] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <92af95e80a60b52489dfd5ff53883ea1@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFazdOo= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 15:57:30 -0000 gallatin added a comment. It might be nice to make these general tunables that could be done centrally and apply to all drivers, but that's probably outside the scope of the review. INLINE COMMENTS sys/netinet/tcp_lro.c:655 Can you just initialize ack_append_limit to the max value for whatever type it is and eliminate the check for a 0 ack_append_limit? That would eliminate one clause from this conditional. sys/netinet/tcp_lro.c:684 Rather than adding more clauses to this condition, how would to feel about setting an append limit in bytes, and replacing the hard-coded 65535 with this new limit? The default lro init would initialize the new limit to 65535. And hn(4) would initialize it in terms of multiples of its MTU. REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, transport, gallatin Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 16:02:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46BA8A73298 for ; Thu, 4 Feb 2016 16:02:04 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 31D19900 for ; Thu, 4 Feb 2016 16:02:04 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 2F5891073FF; Thu, 4 Feb 2016 16:02:04 +0000 (UTC) Date: Thu, 4 Feb 2016 16:02:04 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Commented On] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFazdfw= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 16:02:04 -0000 adrian added inline comments. INLINE COMMENTS sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c:455 this should be a separate commit REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, transport, gallatin Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Thu Feb 4 16:58:10 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 679AEA9BAD3; Thu, 4 Feb 2016 16:58:10 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mx1.shrew.net (mx1.shrew.net [38.97.5.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 219B5113E; Thu, 4 Feb 2016 16:58:09 +0000 (UTC) (envelope-from mgrooms@shrew.net) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx1.shrew.net (8.14.7/8.14.7) with ESMTP id u14GtUax077373; Thu, 4 Feb 2016 10:55:31 -0600 (CST) (envelope-from mgrooms@shrew.net) Received: from [10.16.32.30] (72-48-144-84.static.grandenetworks.net [72.48.144.84]) by mail.shrew.net (Postfix) with ESMTPSA id 8C88218C859; Thu, 4 Feb 2016 10:55:25 -0600 (CST) Subject: Re: 10.2-RELEASE-p12 pf+GRE crashing To: freebsd-stable@freebsd.org, freebsd-net@freebsd.org References: <56B285B0.8010306@shrew.net> <56B29FA0.4080000@shrew.net> From: Matthew Grooms Message-ID: <56B38313.8070004@shrew.net> Date: Thu, 4 Feb 2016 10:57:55 -0600 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <56B29FA0.4080000@shrew.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mx1.shrew.net [10.24.10.10]); Thu, 04 Feb 2016 10:55:31 -0600 (CST) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 16:58:10 -0000 On 2/3/2016 6:47 PM, Matthew Grooms wrote: > This turned out to be another issue that was patched in head but not > back ported to stable. I can't explain why it didn't get tripped when > GRE tunnels were disabled. With the patch applied, I can reload my > rule sets again without crashing ... > > https://svnweb.freebsd.org/base?view=revision&revision=264689 > I wanted to clarify in case another user runs into this issue and searches the mailing list history for a solution: The patch I applied to fix this particular kernel crash wasn't 264689, it was ... https://svnweb.freebsd.org/base?view=revision&revision=264915 Sorry for the misinformation. I cut and pasted the wrong link. -Matthew > (kgdb) bt > #0 doadump (textdump=) at pcpu.h:219 > #1 0xffffffff807c81f2 in kern_reboot (howto=260) at > ../../../kern/kern_shutdown.c:451 > #2 0xffffffff807c85d5 in vpanic (fmt=, ap= optimized out>) > at ../../../kern/kern_shutdown.c:758 > #3 0xffffffff807c8463 in panic (fmt=0x0) at > ../../../kern/kern_shutdown.c:687 > #4 0xffffffff80bdc10b in trap_fatal (frame=, > eva=) at ../../../amd64/amd64/trap.c:851 > #5 0xffffffff80bdc40d in trap_pfault (frame=0xfffffe0000233a80, > usermode=) at ../../../amd64/amd64/trap.c:674 > #6 0xffffffff80bdbaaa in trap (frame=0xfffffe0000233a80) > at ../../../amd64/amd64/trap.c:440 > #7 0xffffffff80bc1fa2 in calltrap () at > ../../../amd64/amd64/exception.S:236 > #8 0xffffffff809c07f4 in pfr_detach_table (kt=0x0) at > ../../../netpfil/pf/pf_table.c:2047 > #9 0xffffffff809a91f4 in pf_empty_pool (poola=0xffffffff813c3d68) > at ../../../netpfil/pf/pf_ioctl.c:354 > #10 0xffffffff809ab3e5 in pfioctl (dev=, > cmd=, > addr=0xfffff8005eaf6800 "", flags=, td= optimized out>) > at ../../../netpfil/pf/pf_ioctl.c:2189 > #11 0xffffffff806b5659 in devfs_ioctl_f (fp=0xfffff8000a2927d0, > com=3295691827, > data=0xfffff8005eaf6800, cred=, > td=0xfffff8000a25f000) > at ../../../fs/devfs/devfs_vnops.c:785 > #12 0xffffffff8081b805 in kern_ioctl (td=0xfffff8000a25f000, fd= optimized out>, > com=2) at file.h:320 > #13 0xffffffff8081b500 in sys_ioctl (td=0xfffff8000a25f000, > uap=0xfffffe0000234b40) > at ../../../kern/sys_generic.c:718 > #14 0xffffffff80bdca27 in amd64_syscall (td=0xfffff8000a25f000, traced=0) > at subr_syscall.c:134 > #15 0xffffffff80bc228b in Xfast_syscall () at > ../../../amd64/amd64/exception.S:396 > #16 0x0000000800dd9fda in ?? () > Previous frame inner to this frame (corrupt stack?) > Current language: auto; currently minimal > > -Matthew > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Thu Feb 4 18:53:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8100BA9BEB1 for ; Thu, 4 Feb 2016 18:53:04 +0000 (UTC) (envelope-from dudu.meyer@gmail.com) Received: from mail-ob0-x233.google.com (mail-ob0-x233.google.com [IPv6:2607:f8b0:4003:c01::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4597510EB for ; Thu, 4 Feb 2016 18:53:04 +0000 (UTC) (envelope-from dudu.meyer@gmail.com) Received: by mail-ob0-x233.google.com with SMTP id ba1so73470908obb.3 for ; Thu, 04 Feb 2016 10:53:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FrVsAcLucwPlD6fcfyAsIFSw+di85odkNOUxDpRqaQI=; b=Wqic/9lhMZ+JvVkMzbRYdH5DJgV9VZ9hb9hbW+kUg7xmVICkMcO0APi4ZZL0sNq5mj e6ahxatSbILTZsYZHaIfiZvLr77oerkUlnFs58VrV3KWdWWusbQqQ7CP+kmCspYnXcfB snbOduofNwgkq89yqtQ67Cm9eUQrug6tML9sIKuCLGE7Y29Me9AiBxw1n3ZuMU+PcdtX KUX3IU4wLLGkwMr6swwy/MA6Y7Gv02+XYe4cIDjn7T3aJMoD4KcDkz1Krb9Jc1GTXb07 VtbDtoQ+mzMLx+PpN/QfPoJZX2UEiUC7tx0Fi9gfQF6zthB//x4UQsvSlkg1DcVaGjLE tycA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=FrVsAcLucwPlD6fcfyAsIFSw+di85odkNOUxDpRqaQI=; b=Wb/tuSSrM8FVujxTBmTQWp2JLU6ruLgPvjZBIdpVyJnIlBtbvaxINKA4EnJcZTvltL lNZmEijn204KglIIv3kAJO0JYOT/NkSygWFqqZjmhIHr/Jw4Q6M1USckIn3xm3TEIOPQ DoHKYH6HuklwJ8/r1cjHOUlQ+YxuxCt1Qj4MChy3CDc278RaQ5el3jwqozcuS1nqNxJ0 suq6B3OdS2mdjqzWNziWjxig5E8IJct63YdxnAVt1IVQaGKEX0s4XiwHSZu6lFRpaSAH LU6oK7P0CgisgbQoLjqwFuxgq5JQZbsFzikn+l1/t4OVCGYUweyv/rAw6+VpSydSVF5U MDjw== X-Gm-Message-State: AG10YOQIJQ4o80JeM8Z71TKqWOtQRkI8go8dzdX7ZjM0/uTa4o08FzeZNV91mD8TG2EbsMPq4KeCzbBTWY5z5Q== MIME-Version: 1.0 X-Received: by 10.182.120.40 with SMTP id kz8mr8774645obb.24.1454611983402; Thu, 04 Feb 2016 10:53:03 -0800 (PST) Received: by 10.182.88.138 with HTTP; Thu, 4 Feb 2016 10:53:03 -0800 (PST) In-Reply-To: References: Date: Thu, 4 Feb 2016 16:53:03 -0200 Message-ID: Subject: Re: dev.netmap.buf_size and packett size from host From: Eduardo Meyer To: Luigi Rizzo Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 18:53:04 -0000 when i disabled LRO it ruined communication on the port to network (altough from host was ok), everything else looks good and so far I had no problem with big packets coming from host, so -tso did it, thank you! On Thu, Feb 4, 2016 at 12:35 PM, Luigi Rizzo wrote: > disable all hardware accelerations when using netmap. > > cheers > luigi > > On Thu, Feb 4, 2016 at 3:34 PM, Eduardo Meyer > wrote: > > mtu is good, TSO was on, thank you will retest right now. > > > > which other port features should I disable? I only disabled txcsum and > > rxcsum before, now tso on the list, anything else in netmap mode? > > > > On Thu, Feb 4, 2016 at 12:29 PM, Luigi Rizzo wrote: > >> > >> Make sure you disable TSO on the interface used in netmap > >> mode, and then check that you use an MTU of 1500 on that > >> interface. > >> You should not receive frames larger than MTU coming from > >> the host in these conditions. > >> > >> cheers > >> luigi > >> > >> > >> On Thu, Feb 4, 2016 at 3:26 PM, Eduardo Meyer > >> wrote: > >> > hello, > >> > > >> > I have a netmap application which has host mode bridge/fwd, with > default > >> > settings I have the following error some often: > >> > > >> > 884.260394 [2950] netmap_transmit igb1 from_host, drop > packet > >> > size 2962 > 2048 > >> > > >> > the only application which relies on host mode is bird, so those > packets > >> > are probably from bird daemon, when I get those errors I get bird > >> > sessions > >> > failing and restart > >> > > >> > I raised dev.netmap.buf_size to 5000 it ajusted to 5120, things got > >> > better > >> > but I still have logs: > >> > > >> > netmap_transmit igb1 from_host, drop packet size 5858 > 5120 > >> > > >> > Now the main question is, when dev.netmap.buf_size is 2048 the > >> > application > >> > uses 1.3G of RAM but when I raise to 5120 it uses 3G of RAM. > >> > > >> > So I need to understand, is this packet size really related from what > I > >> > get > >> > from the application packets coming from host to netmap? If so can I > >> > allow > >> > for bigger sizes, like 16k (lo0 mtu) without pre-alloc so much more > RAM? > >> > > >> > thank you > >> > > >> > E. Meyer > >> > _______________________________________________ > >> > freebsd-net@freebsd.org mailing list > >> > https://lists.freebsd.org/mailman/listinfo/freebsd-net > >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > " > >> > >> > >> > >> -- > >> > -----------------------------------------+------------------------------- > >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > dell'Informazione > >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > >> TEL +39-050-2217533 . via Diotisalvi 2 > >> Mobile +39-338-6809875 . 56122 PISA (Italy) > >> > -----------------------------------------+------------------------------- > > > > > > > > > > -- > > =========== > > Eduardo Meyer > > pessoal: dudu.meyer@gmail.com > > profissional: ddm.farmaciap@saude.gov.br > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2217533 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > -- =========== Eduardo Meyer pessoal: dudu.meyer@gmail.com profissional: ddm.farmaciap@saude.gov.br From owner-freebsd-net@freebsd.org Thu Feb 4 22:23:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28366A9C85D for ; Thu, 4 Feb 2016 22:23:04 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D96D31E2C for ; Thu, 4 Feb 2016 22:23:03 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: by mail-ig0-x22d.google.com with SMTP id xg9so1659389igb.1 for ; Thu, 04 Feb 2016 14:23:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=XeHHOBfBgOYRyRVSML8Dd1HhP820NWdcpbawp3WfWk8=; b=x+PHQQJ/DTH2Lv0egAIRqBhmhGbpScF2YKrwo+FMiW7X0QurOO6jA/oYGWIhHi68bx GKABXjrFA3hGUiC+VUlRzEk0obQqmadlDGZpNLF5d2n8vIFqWkHfc/JlXpFAvJkJrZhx pBjgvI9hZGSnMM6Jtamz+6aFB4rwJUbgKx6I1sxgPB5lH+8PFFgK6CmMeDbWXQyz9U+I isET4adTxUnWcNvxEYxDyNYb/mpAUkcC5qSX6X9cHw1LY6fJkH3P6roOmbNJ8GC5Wc7Y uLjdpMaii8bpmi7JqSh04wAULaRWIftZDho+tQ4rkpj3DQnN//hRgGSQSwzMbxkrn6FA P6Xw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rice-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=XeHHOBfBgOYRyRVSML8Dd1HhP820NWdcpbawp3WfWk8=; b=P8QyIgEbFj8zpM3Uo5uHtVv/AoGrRhVa8GnSwU0x0e15yAfAmsOupWdxNYj0/Erg3+ Ki9O5lbhBBax2/EyXmBagwLHBY1TtgTrslJNYW/9st32o2c4NqMiNPZlivi6jNmOaLu0 tjxYxiySfWjc/w1l+bDmTIjJeBKKYy0t50pm61Lpj7FVLBBx1VZgjWRcy4LJAHfXVuLb oULcAw6vAMDs2wELuCcSibl4X5a0qLYZ86TxO2a300RSb+CbPRG63E/l64+5wvv8RhlD FuUV6ae5IWxhEjLafKfQQaqXqEcD/2kNtZjXs9NRUcRClwW73YYr6C+lLESpyUO1WJp9 861A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=XeHHOBfBgOYRyRVSML8Dd1HhP820NWdcpbawp3WfWk8=; b=e9zsNOpRi1ezW69FWbhZig5hFRn+8W5OniZsgHJPzJiE8/HauIU99MfE7KSdr89Swr kWzkBCK1HBfzT+l3YuPjefqEOwlbTM0PhhWZMhZNu4re638nVV7VNaXxkABNOWvExpv/ g3zuAE5EbD9IC0vL1G1BYnXud1snncIQl22InKkQoYNK/5HkuUoA4p0f2rXF/gbTT4eY xWoFFqZ2z050FQmBEldlKk0vEjA5PySHPysvwyxBAHXWFI1nGTEF5Z3vlbs8CAnBz2HP OjjNAr6gc2bHeI8hZhgUa9xY6e2nKCER7rtRlFI0/bTCHInuVndXlo/o2Jrr1JDriC4z TyPQ== X-Gm-Message-State: AG10YORhZ+b8RkG35+hWK/x9O+hrYPZNew0+7BnnlqPOKq1c3veeWvFY0JE+J35ts4ArFEFb2OVlnuxC+TcYcA== MIME-Version: 1.0 X-Received: by 10.50.142.68 with SMTP id ru4mr12939607igb.54.1454624583199; Thu, 04 Feb 2016 14:23:03 -0800 (PST) Sender: sunxiaoye07@gmail.com Received: by 10.36.98.82 with HTTP; Thu, 4 Feb 2016 14:23:03 -0800 (PST) In-Reply-To: References: Date: Thu, 4 Feb 2016 16:23:03 -0600 X-Google-Sender-Auth: UQiXXWxBMEIDEFqEHyWkud0daAc Message-ID: Subject: Fwd: swaping ring slots between NIC ring and Host ring does not always success From: Xiaoye Sun To: Luigi Rizzo , Pavel Odintsov , freebsd-net@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 22:23:04 -0000 Hi Luigi, Thanks for your explanation. I used three machines to do this experiment. They are directly connected. [(machine1) eth1]---[eth2 (machine2) eth3]---[eth4 (machine3)]. First, I tried to run bridge.c on machine2 using the command *bridge -i netmap:eth2 -i netmap:eth3*. (sender receiver or XYZ were not running on machine 1or3) For my understanding, in this setup, machine2 will be transparent to machine1&3 since it forwards packet from its eth2 to eth3 and vice versa without any modification to the packets. I tried to ping machine 3 from machine 1 using the command like *ping 10.11.10.3*. However, it still does not success. This is because that before machine1 sends ping message to machine3, it will first send a ARP request message to get the mac address of machine3. machine3 gets that ARP request, and send the reply back (I use tcpdump to verify that machine3 gets the ARP request and send out the ARP reply). However, machine1 does not get the ARP reply. I checked that the bridge can only forwarding packet in one direction at the same time. it gets the ARP request but doesn't see the ARP reply (*pkt_queued* always returns 0 for one nic...). This behavior looks very weird to me. Do you think there is a compatibility issues between netmap and the os I am using? Is there a verified linux distribution (also the version) that perfectly works well with netmap? The OS I use is 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) x86_64 GNU/Linux. Linux kernel version is *3.16.0-4-amd64* Thanks! Xiaoye On Wed, Feb 3, 2016 at 2:12 AM, Luigi Rizzo wrote: > On Tue, Feb 2, 2016 at 10:48 PM, Xiaoye Sun wrote: > > > > > > On Mon, Feb 1, 2016 at 11:34 PM, Luigi Rizzo wrote: > >> > >> On Tue, Feb 2, 2016 at 6:23 AM, Xiaoye Sun wrote: > >> > Hi Luigi, > >> > > >> > I have to clarify about the *jumping issue* about the slot indexes. > >> > In the bridge.c program, the slot index never jumps and it increases > >> > sequentially. > >> > In the receiver.c program, the udp packet seq jumps and I showed the > >> > slot > >> > index that each udp packet uses. So the slot index jumps together with > >> > the > >> > udp seq (at the receiver program only). > >> > >> So let me understand, is the "slot" some information written > >> in the packet by bridge.c (referring to the rx or tx slot, > >> I am not sure) and then read and printed by receiver.c > >> (which gets the packet through recvfrom so there isn't > >> really any slot index) ? > >> > > It works in the other way: > > The bridge.c checks the seq numbers of the udp packets in netmap slots > (in > > nic rx ring) before the swap; then it records the seq number, slot > > number(both rx and tx (tx indexes were not shown in the previous email > since > > they all look correct)) and buf_idx (rx and tx). The bridge.c does not > > change anything in the buffer and it knows the slot and buf_idx that a > > packet uses. Please refer to the added code in *process_rings* function > > http://www.owlnet.rice.edu/~xs6/bridge.c > > The receiver.c checks the seq numbers only and print out the seq numbers > it > > receive sequentially. > > With these information, I manually match the seq number I got from > > receiver.c and the seq number I got from bridge.c. So we know what is the > > seq order the receiver sees and which slot a packet uses when bridge.c > swaps > > the buf_idxs. > > > >> Do you see any ordering inversion when the receiver > >> gets packets through the NETMAP API (e.g. using bridge.c > >> instead of receiver.c) ? > >> > > There is no ordering inversion seen by bridge.c (As I said in the > previous > > paragraph, the bridge.c checks the seq number and I did not see any order > > inversion in THIS simple experiment (In my multicast protocol (mentioned > in > > the first email), there is ordering inversion. But let us solve the > simple > > bridge.c's problem first. I think they are two relatively independent > > issues.)). > > Sorry there was a misunderstanding. > I wanted you to check the following setup: > > [1: send.c] ->- [2: bridge.c] ->- [3: XYZ] > > where in XYZ you replace your receiver.c with some > netmap-based receiver (it could be pkt-gen in rx mode, > or possibly even another instance of bridge.c where > you connect the output port to a vale switch so > traffic is dropped), and then in XYZ print the content > of the packets. > > From your previous report we know that node 2: sees packets > in order, and node 3: sees packets out of order. > However, if the problem were due to bridge.c sending > the old buffer and not the new one, you'd see not only > reordering but also replication of packets. > > The fact that you see only the reordering in 3: makes > me think that the problem is in that node, and it could > be the network stack in 3: that does something strange. > So if you can run something netmap based in 3: and make > sure there is only one queue to read from, we could > at least figure out what is going on. > > cheers > luigi > > > is that > > > >> > >> Are you using native netmap drivers or the emulated mode ? > >> You can check that by playing with the "admode" sysctl entry > >> (or sysfs on linux) - try setting to 1 and 2 and see if > >> the behaviour changes. > >> > >> dev.netmap.admode: 0 > >> Controls the use of native or emulated adapter mode. > >> 0 uses the best available option, > >> 1 forces native and fails if not available, > >> 2 forces emulated hence never fails. > >> > > I was using admode 0. I changed the admode to 1 and 2 using the command > like > > *echo 1 > /sys/module/netmap/parameters/admode* and restart the bridge > > program. The behavior keeps the same. > > > >> > >> cheers > >> luigi > >> > >> > > >> > There is really one ring (tx and rx) for NIC and one ring (tx and rx) > >> > for > >> > the host. > >> > I also doubt that there might be multiple tx rings for the host. It > >> > seems > >> > like that bridge program swap packet to multiple host rings and the > udp > >> > recv > >> > program drains packets from these rings. But this is not the case > here. > >> > > >> > The bridge program prints a line like this > >> > *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* > >> > this is printed by the following line the original program > >> > *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, > >> > pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, > >> > pb->first_rx_ring, > >> > pb->req.nr_rx_rings);* > >> > > >> > I think this shows that there is really one NIC ring and one HOST > ring. > >> > > >> > Is there another way to verify the number of ring that netmap has? > >> > > >> > Thanks! > >> > Xiaoye > >> > > >> > On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo > wrote: > >> >> > >> >> Hi, > >> >> there must be some wrong with your setting because > >> >> slot indexes must be sequential and in your case they > >> >> are not (see the jump from 295 to 474 and then > >> >> back from 485 to 296, and the numerous interleavings > >> >> that you are seeing later). > >> >> > >> >> I have no idea of the cause but typically this pattern > >> >> is what you see when there are multiple input rings and > >> >> not just one. > >> >> > >> >> Cheers > >> >> Luigi > >> >> > >> >> > >> >> > >> >> > >> >> On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun > >> >> wrote: > >> >> > Hi Luigi, > >> >> > > >> >> > Thanks for the detailed advice. > >> >> > > >> >> > With more detailed experiments, actually I found that the udp > >> >> > sender/receiver packet reorder issue *might* be irrelevant to the > >> >> > original > >> >> > issue I posted. However, I think we should solve the udp > >> >> > sender/receiver > >> >> > issue first. > >> >> > I run the experiment with more detailed log. Here is my findings. > >> >> > > >> >> > 1. I am running a netmap version available since about Oct 13rd > from > >> >> > github > >> >> > (https://github.com/luigirizzo/netmap). So I think this is not the > >> >> > one > >> >> > related to the buffer allocation issue. I tried to running the > newest > >> >> > version, however, that version causes problem when I exit the > bridge > >> >> > program > >> >> > (something like kernel error which make the os crash). > >> >> > > >> >> > 2 & 3. I changed the receiver.c & bridge.c so that I can get more > >> >> > information (more detailed log). > >> >> > The reorder happens multiple times (about 10 times) within a > second. > >> >> > Here is > >> >> > one example trace collected from the above two programs. > (remembering > >> >> > that > >> >> > we have udp sender running on one machine; netmap bridge and udp > >> >> > receiver > >> >> > are running on another machine). > >> >> > There is only one pair of rings each with 512 slots (511 slot > usable) > >> >> > on > >> >> > the > >> >> > receiver machine. > >> >> > > >> >> > =================== packet trace collected from receiver.c > >> >> > =================== > >> >> > ===== together with the slot and buf_idx of the corresponding > netmap > >> >> > ring > >> >> > slots ====== > >> >> > [seq] [slot] [buf_idx] > >> >> > 8208 294 1833 > >> >> > 8209 295 1834 > >> >> > 8388 474 2013 > >> >> > ... (packet received in order) > >> >> > 8398 484 2023 > >> >> > 8399 485 2024 > >> >> > 8210 296 1835 > >> >> > 8211 297 1836 > >> >> > ... (packet received in order) > >> >> > ... > >> >> > 8222 308 1847 > >> >> > 8400 486 2025 > >> >> > 8223 309 1848 > >> >> > 8401 487 2026 > >> >> > 8224 310 1849 > >> >> > 8402 488 2027 > >> >> > 8225 311 1850 > >> >> > 8403 489 2028 > >> >> > 8226 312 1851 > >> >> > 8404 450 2029 > >> >> > 8227 313 1852 > >> >> > 8228 314 1853 > >> >> > =================================================================== > >> >> > As we can see that the udp receiver got packet 8210 after it got > >> >> > 8399, > >> >> > which > >> >> > is the first reorder. Then, the receiver got 8211 to 8222 > >> >> > sequentially. > >> >> > Then > >> >> > it got packet from 8223-8227 and 8400-8404 interleaved. > >> >> > > >> >> > > >> >> > ==================== event order seen by netmap bridge > >> >> > ================== > >> >> > get 8209 > >> >> > poll called > >> >> > get 8210 > >> >> > ... > >> >> > ... > >> >> > get 8228 > >> >> > poll called > >> >> > get 8229 > >> >> > ... > >> >> > ... > >> >> > get 8383 > >> >> > poll called > >> >> > get 8384 > >> >> > ... > >> >> > get 8387 > >> >> > poll called > >> >> > get 8388 > >> >> > ... > >> >> > get 8393 > >> >> > poll called > >> >> > get 8394 > >> >> > ... > >> >> > get 8399 > >> >> > poll called > >> >> > get 8400 > >> >> > ... > >> >> > get 8404 > >> >> > poll called > >> >> > get 8405 > >> >> > =================================================================== > >> >> > As we can see, from the event ordering see by the bridge.c, all the > >> >> > packets > >> >> > are receiver in order, which means the the reorder happens when the > >> >> > bridge > >> >> > code swap the buf_idx between the nic ring(slot) and the host > >> >> > ring(slot). > >> >> > The reordered seq usually right before or after the poll function > >> >> > call. > >> >> > > >> >> > Best, > >> >> > Xiaoye > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo > >> >> > wrote: > >> >> >> > >> >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun > >> >> >> wrote: > >> >> >> > Hi Luigi, > >> >> >> > > >> >> >> > Thanks for your advice. > >> >> >> > I forgot to mention that I use the command "ethtool -L eth1 > >> >> >> > combined > >> >> >> > 1" > >> >> >> > to > >> >> >> > set the number of rings of the nic to 1. The host also only has > >> >> >> > one > >> >> >> > ring. > >> >> >> > I understand the situation where the first tx ring is full so > the > >> >> >> > bridge > >> >> >> > will swap the packets to the second tx ring and then the > host/nic > >> >> >> > might > >> >> >> > drain either rings. But this is not the case in the experiment. > >> >> >> > >> >> >> ok good to know that. > >> >> >> > >> >> >> So if we have ruled out multiqueue and iommu, let's look at > >> >> >> the internal allocator and at bridge.c > >> >> >> > >> >> >> 1. are you running the most recent version of netmap ? > >> >> >> Some older version (probably 1-2 years ago) had a bug > >> >> >> in the buffer allocator and some buffers were allocated > >> >> >> twice. > >> >> >> > >> >> >> 2. can you tweak your receiver.c to report some more info > >> >> >> on how often you get out of sequence packets, how much > >> >> >> out of sequence they are ? > >> >> >> Also it would be useful to report gaps on the increasing side > >> >> >> (i.e. new_seq != old_seq +1 ) > >> >> >> > >> >> >> 3. can you tweak bridge.c so that it writes into the packet > >> >> >> the netmap buffer indexes and slots on the rx and tx side, > >> >> >> so when you detect a sequence error we can figure out > >> >> >> where it is happening. > >> >> >> Ideally you could also add the sequence number detection > >> >> >> code in bridge.c so we can check whether the errors appear > >> >> >> on the input or output sides. > >> >> >> > >> >> >> cheers > >> >> >> luigi > >> >> >> > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> > >> >> > -----------------------------------------+------------------------------- > >> >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > >> >> dell'Informazione > >> >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > >> >> TEL +39-050-2217533 . via Diotisalvi 2 > >> >> Mobile +39-338-6809875 . 56122 PISA (Italy) > >> >> > >> >> > -----------------------------------------+------------------------------- > >> >> > >> > > >> > >> > >> > >> -- > >> > -----------------------------------------+------------------------------- > >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > dell'Informazione > >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > >> TEL +39-050-2217533 . via Diotisalvi 2 > >> Mobile +39-338-6809875 . 56122 PISA (Italy) > >> > -----------------------------------------+------------------------------- > >> > > > > > > -- > -----------------------------------------+------------------------------- > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > TEL +39-050-2217533 . via Diotisalvi 2 > Mobile +39-338-6809875 . 56122 PISA (Italy) > -----------------------------------------+------------------------------- > > From owner-freebsd-net@freebsd.org Thu Feb 4 22:25:13 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D71EA9C8CE for ; Thu, 4 Feb 2016 22:25:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F1361EE4 for ; Thu, 4 Feb 2016 22:25:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u14MPDwe055490 for ; Thu, 4 Feb 2016 22:25:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206932] Realtek 8111 card stops responding under high load in netmap mode Date: Thu, 04 Feb 2016 22:25:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 22:25:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206932 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Feb 4 22:27:36 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 595CCA9CA5B for ; Thu, 4 Feb 2016 22:27:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B7CB18C for ; Thu, 4 Feb 2016 22:27:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u14MRZl1058672 for ; Thu, 4 Feb 2016 22:27:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206904] tailq crash/nd inet6 Date: Thu, 04 Feb 2016 22:27:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 22:27:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206904 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Feb 4 23:31:37 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4665BA9DFE9 for ; Thu, 4 Feb 2016 23:31:37 +0000 (UTC) (envelope-from victordetoni@gmail.com) Received: from mail-ig0-x22b.google.com (mail-ig0-x22b.google.com [IPv6:2607:f8b0:4001:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 094E9B4F for ; Thu, 4 Feb 2016 23:31:37 +0000 (UTC) (envelope-from victordetoni@gmail.com) Received: by mail-ig0-x22b.google.com with SMTP id 5so2767444igt.0 for ; Thu, 04 Feb 2016 15:31:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=L/DKyOzasvjR7Igk+MHP0SRcL4/dO/lM/1tcw1TDH4s=; b=VdtvwkvkFVzcAu2uqCrPlTX//2rdyWSveL2zafgIYqTignUdvf2usunhbz0ZJy/qfJ omfYGF55kY5Y+AgLzgO/0SsSQkVR1kYDk9Fj3A/QJjt9dutghRIxz0xxc+bxlaM5ciys QQiyvf+PeBBEFlPqsK86Zl+gCj9yCE58jz7f61BZdbkYRJOoklmTeC1SebkMkSShKHmA sLnXUmkb535P7e89Jovc/S52mlP9BHUAoeN0/hW5SQJDNc+xi9bCqisjE8Qvn5eE2M7I 9S1twz1UyqSdn57dv3Rcmm6YK5gmKiTNqHp+UwKS2qkEPv2Ooq9vJIZ4wWUQ2+HLAjFc yL8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=L/DKyOzasvjR7Igk+MHP0SRcL4/dO/lM/1tcw1TDH4s=; b=iWGySZcClF0dH1o/upcYiu+euY0k0mIcWlheCbVF+u2/u+/cHpU9JiUjuk49hkyc5X MEWeHIYIj79cCRgzh7wB/zaQVLEt8MEdSr0gZTLZVz6Z9oPMf7pmI1Sl0yQYKVSZS4ol n+itkg9IgVB4ZKAEOY+oVKRIGDe2WO2RGHP326vBemSCPIf0hgkzE2uSReSifLm1F38i ICrJFdob64VhWZrj4YO4pYCWYmU/LoyBYJVCOdEbNkJH48b+8s1b2/pALoyOz1awNAfz c5JYKg81VEgQY8Pw0prsV4gk2MBAhnMpDXs4A3aInmBJKfqkqUDnzG0TpjTD359TVT8M hDDQ== X-Gm-Message-State: AG10YOSxUYPBlPYoxkmACTwVQnF7GBXVyqrqlGHa/V6feb/jkCNq88x2nCzL+MqQeY1XQufksXFjWi/YxBby4g== MIME-Version: 1.0 X-Received: by 10.50.102.40 with SMTP id fl8mr12463237igb.85.1454628696387; Thu, 04 Feb 2016 15:31:36 -0800 (PST) Received: by 10.107.52.205 with HTTP; Thu, 4 Feb 2016 15:31:36 -0800 (PST) In-Reply-To: References: Date: Thu, 4 Feb 2016 21:31:36 -0200 Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Victor Detoni To: Xiaoye Sun Cc: Luigi Rizzo , Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2016 23:31:37 -0000 Both interfaces are up? Like ifconfig... up I had this the same problem and I solve with commands above Em quinta-feira, 4 de fevereiro de 2016, Xiaoye Sun escreveu: > Hi Luigi, > > Thanks for your explanation. > > I used three machines to do this experiment. They are directly connected. > > [(machine1) eth1]---[eth2 (machine2) eth3]---[eth4 (machine3)]. > > First, I tried to run bridge.c on machine2 using the command *bridge -i > netmap:eth2 -i netmap:eth3*. (sender receiver or XYZ were not running on > machine 1or3) > > For my understanding, in this setup, machine2 will be transparent to > machine1&3 since it forwards packet from its eth2 to eth3 and vice versa > without any modification to the packets. > > I tried to ping machine 3 from machine 1 using the command like *ping > 10.11.10.3*. However, it still does not success. > This is because that before machine1 sends ping message to machine3, it > will first send a ARP request message to get the mac address of machine3. > machine3 gets that ARP request, and send the reply back (I use tcpdump to > verify that machine3 gets the ARP request and send out the ARP reply). > However, machine1 does not get the ARP reply. > > I checked that the bridge can only forwarding packet in one direction at > the same time. it gets the ARP request but doesn't see the ARP reply > (*pkt_queued* always returns 0 for one nic...). > > This behavior looks very weird to me. Do you think there is a compatibility > issues between netmap and the os I am using? Is there a verified linux > distribution (also the version) that perfectly works well with netmap? > > The OS I use is 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) > x86_64 GNU/Linux. > Linux kernel version is *3.16.0-4-amd64* > > > Thanks! > Xiaoye > > > > > > > On Wed, Feb 3, 2016 at 2:12 AM, Luigi Rizzo > wrote: > > > On Tue, Feb 2, 2016 at 10:48 PM, Xiaoye Sun > wrote: > > > > > > > > > On Mon, Feb 1, 2016 at 11:34 PM, Luigi Rizzo > wrote: > > >> > > >> On Tue, Feb 2, 2016 at 6:23 AM, Xiaoye Sun > wrote: > > >> > Hi Luigi, > > >> > > > >> > I have to clarify about the *jumping issue* about the slot indexes. > > >> > In the bridge.c program, the slot index never jumps and it increases > > >> > sequentially. > > >> > In the receiver.c program, the udp packet seq jumps and I showed the > > >> > slot > > >> > index that each udp packet uses. So the slot index jumps together > with > > >> > the > > >> > udp seq (at the receiver program only). > > >> > > >> So let me understand, is the "slot" some information written > > >> in the packet by bridge.c (referring to the rx or tx slot, > > >> I am not sure) and then read and printed by receiver.c > > >> (which gets the packet through recvfrom so there isn't > > >> really any slot index) ? > > >> > > > It works in the other way: > > > The bridge.c checks the seq numbers of the udp packets in netmap slots > > (in > > > nic rx ring) before the swap; then it records the seq number, slot > > > number(both rx and tx (tx indexes were not shown in the previous email > > since > > > they all look correct)) and buf_idx (rx and tx). The bridge.c does not > > > change anything in the buffer and it knows the slot and buf_idx that a > > > packet uses. Please refer to the added code in *process_rings* function > > > http://www.owlnet.rice.edu/~xs6/bridge.c > > > The receiver.c checks the seq numbers only and print out the seq > numbers > > it > > > receive sequentially. > > > With these information, I manually match the seq number I got from > > > receiver.c and the seq number I got from bridge.c. So we know what is > the > > > seq order the receiver sees and which slot a packet uses when bridge.c > > swaps > > > the buf_idxs. > > > > > >> Do you see any ordering inversion when the receiver > > >> gets packets through the NETMAP API (e.g. using bridge.c > > >> instead of receiver.c) ? > > >> > > > There is no ordering inversion seen by bridge.c (As I said in the > > previous > > > paragraph, the bridge.c checks the seq number and I did not see any > order > > > inversion in THIS simple experiment (In my multicast protocol > (mentioned > > in > > > the first email), there is ordering inversion. But let us solve the > > simple > > > bridge.c's problem first. I think they are two relatively independent > > > issues.)). > > > > Sorry there was a misunderstanding. > > I wanted you to check the following setup: > > > > [1: send.c] ->- [2: bridge.c] ->- [3: XYZ] > > > > where in XYZ you replace your receiver.c with some > > netmap-based receiver (it could be pkt-gen in rx mode, > > or possibly even another instance of bridge.c where > > you connect the output port to a vale switch so > > traffic is dropped), and then in XYZ print the content > > of the packets. > > > > From your previous report we know that node 2: sees packets > > in order, and node 3: sees packets out of order. > > However, if the problem were due to bridge.c sending > > the old buffer and not the new one, you'd see not only > > reordering but also replication of packets. > > > > The fact that you see only the reordering in 3: makes > > me think that the problem is in that node, and it could > > be the network stack in 3: that does something strange. > > So if you can run something netmap based in 3: and make > > sure there is only one queue to read from, we could > > at least figure out what is going on. > > > > cheers > > luigi > > > > > > is that > > > > > >> > > >> Are you using native netmap drivers or the emulated mode ? > > >> You can check that by playing with the "admode" sysctl entry > > >> (or sysfs on linux) - try setting to 1 and 2 and see if > > >> the behaviour changes. > > >> > > >> dev.netmap.admode: 0 > > >> Controls the use of native or emulated adapter mode. > > >> 0 uses the best available option, > > >> 1 forces native and fails if not available, > > >> 2 forces emulated hence never fails. > > >> > > > I was using admode 0. I changed the admode to 1 and 2 using the command > > like > > > *echo 1 > /sys/module/netmap/parameters/admode* and restart the bridge > > > program. The behavior keeps the same. > > > > > >> > > >> cheers > > >> luigi > > >> > > >> > > > >> > There is really one ring (tx and rx) for NIC and one ring (tx and > rx) > > >> > for > > >> > the host. > > >> > I also doubt that there might be multiple tx rings for the host. It > > >> > seems > > >> > like that bridge program swap packet to multiple host rings and the > > udp > > >> > recv > > >> > program drains packets from these rings. But this is not the case > > here. > > >> > > > >> > The bridge program prints a line like this > > >> > *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* > > >> > this is printed by the following line the original program > > >> > *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, > > >> > pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, > > >> > pb->first_rx_ring, > > >> > pb->req.nr_rx_rings);* > > >> > > > >> > I think this shows that there is really one NIC ring and one HOST > > ring. > > >> > > > >> > Is there another way to verify the number of ring that netmap has? > > >> > > > >> > Thanks! > > >> > Xiaoye > > >> > > > >> > On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo > > > wrote: > > >> >> > > >> >> Hi, > > >> >> there must be some wrong with your setting because > > >> >> slot indexes must be sequential and in your case they > > >> >> are not (see the jump from 295 to 474 and then > > >> >> back from 485 to 296, and the numerous interleavings > > >> >> that you are seeing later). > > >> >> > > >> >> I have no idea of the cause but typically this pattern > > >> >> is what you see when there are multiple input rings and > > >> >> not just one. > > >> >> > > >> >> Cheers > > >> >> Luigi > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun > > > >> >> wrote: > > >> >> > Hi Luigi, > > >> >> > > > >> >> > Thanks for the detailed advice. > > >> >> > > > >> >> > With more detailed experiments, actually I found that the udp > > >> >> > sender/receiver packet reorder issue *might* be irrelevant to the > > >> >> > original > > >> >> > issue I posted. However, I think we should solve the udp > > >> >> > sender/receiver > > >> >> > issue first. > > >> >> > I run the experiment with more detailed log. Here is my findings. > > >> >> > > > >> >> > 1. I am running a netmap version available since about Oct 13rd > > from > > >> >> > github > > >> >> > (https://github.com/luigirizzo/netmap). So I think this is not > the > > >> >> > one > > >> >> > related to the buffer allocation issue. I tried to running the > > newest > > >> >> > version, however, that version causes problem when I exit the > > bridge > > >> >> > program > > >> >> > (something like kernel error which make the os crash). > > >> >> > > > >> >> > 2 & 3. I changed the receiver.c & bridge.c so that I can get more > > >> >> > information (more detailed log). > > >> >> > The reorder happens multiple times (about 10 times) within a > > second. > > >> >> > Here is > > >> >> > one example trace collected from the above two programs. > > (remembering > > >> >> > that > > >> >> > we have udp sender running on one machine; netmap bridge and udp > > >> >> > receiver > > >> >> > are running on another machine). > > >> >> > There is only one pair of rings each with 512 slots (511 slot > > usable) > > >> >> > on > > >> >> > the > > >> >> > receiver machine. > > >> >> > > > >> >> > =================== packet trace collected from receiver.c > > >> >> > =================== > > >> >> > ===== together with the slot and buf_idx of the corresponding > > netmap > > >> >> > ring > > >> >> > slots ====== > > >> >> > [seq] [slot] [buf_idx] > > >> >> > 8208 294 1833 > > >> >> > 8209 295 1834 > > >> >> > 8388 474 2013 > > >> >> > ... (packet received in order) > > >> >> > 8398 484 2023 > > >> >> > 8399 485 2024 > > >> >> > 8210 296 1835 > > >> >> > 8211 297 1836 > > >> >> > ... (packet received in order) > > >> >> > ... > > >> >> > 8222 308 1847 > > >> >> > 8400 486 2025 > > >> >> > 8223 309 1848 > > >> >> > 8401 487 2026 > > >> >> > 8224 310 1849 > > >> >> > 8402 488 2027 > > >> >> > 8225 311 1850 > > >> >> > 8403 489 2028 > > >> >> > 8226 312 1851 > > >> >> > 8404 450 2029 > > >> >> > 8227 313 1852 > > >> >> > 8228 314 1853 > > >> >> > > =================================================================== > > >> >> > As we can see that the udp receiver got packet 8210 after it got > > >> >> > 8399, > > >> >> > which > > >> >> > is the first reorder. Then, the receiver got 8211 to 8222 > > >> >> > sequentially. > > >> >> > Then > > >> >> > it got packet from 8223-8227 and 8400-8404 interleaved. > > >> >> > > > >> >> > > > >> >> > ==================== event order seen by netmap bridge > > >> >> > ================== > > >> >> > get 8209 > > >> >> > poll called > > >> >> > get 8210 > > >> >> > ... > > >> >> > ... > > >> >> > get 8228 > > >> >> > poll called > > >> >> > get 8229 > > >> >> > ... > > >> >> > ... > > >> >> > get 8383 > > >> >> > poll called > > >> >> > get 8384 > > >> >> > ... > > >> >> > get 8387 > > >> >> > poll called > > >> >> > get 8388 > > >> >> > ... > > >> >> > get 8393 > > >> >> > poll called > > >> >> > get 8394 > > >> >> > ... > > >> >> > get 8399 > > >> >> > poll called > > >> >> > get 8400 > > >> >> > ... > > >> >> > get 8404 > > >> >> > poll called > > >> >> > get 8405 > > >> >> > > =================================================================== > > >> >> > As we can see, from the event ordering see by the bridge.c, all > the > > >> >> > packets > > >> >> > are receiver in order, which means the the reorder happens when > the > > >> >> > bridge > > >> >> > code swap the buf_idx between the nic ring(slot) and the host > > >> >> > ring(slot). > > >> >> > The reordered seq usually right before or after the poll function > > >> >> > call. > > >> >> > > > >> >> > Best, > > >> >> > Xiaoye > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo > > > >> >> > wrote: > > >> >> >> > > >> >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun < > Xiaoye.Sun@rice.edu > > > >> >> >> wrote: > > >> >> >> > Hi Luigi, > > >> >> >> > > > >> >> >> > Thanks for your advice. > > >> >> >> > I forgot to mention that I use the command "ethtool -L eth1 > > >> >> >> > combined > > >> >> >> > 1" > > >> >> >> > to > > >> >> >> > set the number of rings of the nic to 1. The host also only > has > > >> >> >> > one > > >> >> >> > ring. > > >> >> >> > I understand the situation where the first tx ring is full so > > the > > >> >> >> > bridge > > >> >> >> > will swap the packets to the second tx ring and then the > > host/nic > > >> >> >> > might > > >> >> >> > drain either rings. But this is not the case in the > experiment. > > >> >> >> > > >> >> >> ok good to know that. > > >> >> >> > > >> >> >> So if we have ruled out multiqueue and iommu, let's look at > > >> >> >> the internal allocator and at bridge.c > > >> >> >> > > >> >> >> 1. are you running the most recent version of netmap ? > > >> >> >> Some older version (probably 1-2 years ago) had a bug > > >> >> >> in the buffer allocator and some buffers were allocated > > >> >> >> twice. > > >> >> >> > > >> >> >> 2. can you tweak your receiver.c to report some more info > > >> >> >> on how often you get out of sequence packets, how much > > >> >> >> out of sequence they are ? > > >> >> >> Also it would be useful to report gaps on the increasing side > > >> >> >> (i.e. new_seq != old_seq +1 ) > > >> >> >> > > >> >> >> 3. can you tweak bridge.c so that it writes into the packet > > >> >> >> the netmap buffer indexes and slots on the rx and tx side, > > >> >> >> so when you detect a sequence error we can figure out > > >> >> >> where it is happening. > > >> >> >> Ideally you could also add the sequence number detection > > >> >> >> code in bridge.c so we can check whether the errors appear > > >> >> >> on the input or output sides. > > >> >> >> > > >> >> >> cheers > > >> >> >> luigi > > >> >> >> > > >> >> > > > >> >> > > >> >> > > >> >> > > >> >> -- > > >> >> > > >> >> > > -----------------------------------------+------------------------------- > > >> >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di > Ing. > > >> >> dell'Informazione > > >> >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > > >> >> TEL +39-050-2217533 . via Diotisalvi 2 > > >> >> Mobile +39-338-6809875 . 56122 PISA (Italy) > > >> >> > > >> >> > > -----------------------------------------+------------------------------- > > >> >> > > >> > > > >> > > >> > > >> > > >> -- > > >> > > -----------------------------------------+------------------------------- > > >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > > dell'Informazione > > >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > > >> TEL +39-050-2217533 . via Diotisalvi 2 > > >> Mobile +39-338-6809875 . 56122 PISA (Italy) > > >> > > -----------------------------------------+------------------------------- > > >> > > > > > > > > > > > -- > > -----------------------------------------+------------------------------- > > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. > dell'Informazione > > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa > > TEL +39-050-2217533 . via Diotisalvi 2 > > Mobile +39-338-6809875 . 56122 PISA (Italy) > > -----------------------------------------+------------------------------- > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > " > From owner-freebsd-net@freebsd.org Fri Feb 5 00:04:14 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B66A5A9BE26 for ; Fri, 5 Feb 2016 00:04:14 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 71FC81D20 for ; Fri, 5 Feb 2016 00:04:14 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: by mail-io0-x22c.google.com with SMTP id g73so111895738ioe.3 for ; Thu, 04 Feb 2016 16:04:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EmbMO+V8STj0nkKx9wYAsjGyFG6//7OyO7hpkWYBZEg=; b=t8ded/CcT/PAB10LtsYBShqejBmufBVVU083H1U2ZsMvs++d1nrUkQW6BaipU8jJpU MOWDA9no6jLfIrnxdnuxwB9yZktDUAq+Y0kOFZ9BSvLdWvc3gkI9fyIQDj8+OlkSDJyM vomMjD35bllTfV/9TJQrDbIrcZkfdEEcxdGHra8sCe6X25OpgDBIAvEw/MKDg/+9FpKl nNLLT0qrcUulpLdHErt/R2aUrrlKMdkH+yNegM2qRsphCXaXoWvrARTc5x37/lU3vnlT 20ueHwC4nrKtOT4WuTrdvQd7sWpA5Brr/upZdrpGXU3IBe6bRFBhqnINln6qD6UrQOx7 GfXg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rice-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EmbMO+V8STj0nkKx9wYAsjGyFG6//7OyO7hpkWYBZEg=; b=2C701jeTGRN7BW+4BpfZFaJGEpwYIf+RrZ7xUw74Tx+1dXnZYwikekC8OzSjByj582 olH86KesnbG+JZ90O85wN10jIRyFSrj2hoa2sz23CF3Yy0IHgzcNew0gubl4v4mmBvYd M+Gbx/QUhlU/0/DOz1BSAeav2Q1ERw0bSEOCOiuG82VTIJABblYboyid6qckCXLkJfuK F/bZI2KzdajAyMm53Bx/RFf8sgkPQXIf8ztcHlFRMrCNSiKmdX8rydPK7Iu6Kdri0dC0 /W3KKrETpxruHLUFl3dzV/Z3FpcZ8EYFGvKS+5Meg6ajk8/ZEKRHt51jq0/c6d8OT1cl WH8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=EmbMO+V8STj0nkKx9wYAsjGyFG6//7OyO7hpkWYBZEg=; b=Tpf2dPEX4L/Q7elUG3gVeRipRJVk4LsVxjEF+auH5BPk4mpAfcOIL3WoBOL6CUl7wx j3gM7oeyoZXPgE8IVk39WI3LA/pzB4MUy+PbhnTI+nLjrLHavUSwe9MfcnSYsVGGXOuE i2gRlFKRNSFU1ZXtDgBX9DINwK8dKAu/w6sopJu/CSlFLX6awHQLFTH8+WoSteaW+f89 WnaVFvdSqH4+IAv/apX90250ruLuk6Zg8TGY+e+wpQ0iPnhnXa1RN6dsl+9hMN+Mc2DI 8tyQy03r36Di8bLlG/jIBoxenHFMobMzVFoLAF6kwNiR6NqW0fcmog/8cmV4ZjqYuyvg nqdg== X-Gm-Message-State: AG10YOSRUj0pI/RShm5YHz2VuKI5HCLbe035WpTfclsrRnpJRlkdlIQ/q81NqfXLJWuPwwBN89mUcBMQeUPKCg== MIME-Version: 1.0 X-Received: by 10.107.137.100 with SMTP id l97mr13789927iod.110.1454630653906; Thu, 04 Feb 2016 16:04:13 -0800 (PST) Sender: sunxiaoye07@gmail.com Received: by 10.36.98.82 with HTTP; Thu, 4 Feb 2016 16:04:13 -0800 (PST) In-Reply-To: References: Date: Thu, 4 Feb 2016 18:04:13 -0600 X-Google-Sender-Auth: ET4AEWuAfQQRzIamygAw9YTDev0 Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Xiaoye Sun To: Victor Detoni Cc: Luigi Rizzo , Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 00:04:14 -0000 Yes. all the interfaces are up. Are you able to get ARP request when the interfaces are down? On Thursday, February 4, 2016, Victor Detoni wrote: > Both interfaces are up? Like ifconfig... up > > I had this the same problem and I solve with commands above > > Em quinta-feira, 4 de fevereiro de 2016, Xiaoye Sun > escreveu: > >> Hi Luigi, >> >> Thanks for your explanation. >> >> I used three machines to do this experiment. They are directly connected. >> >> [(machine1) eth1]---[eth2 (machine2) eth3]---[eth4 (machine3)]. >> >> First, I tried to run bridge.c on machine2 using the command *bridge -i >> netmap:eth2 -i netmap:eth3*. (sender receiver or XYZ were not running on >> machine 1or3) >> >> For my understanding, in this setup, machine2 will be transparent to >> machine1&3 since it forwards packet from its eth2 to eth3 and vice versa >> without any modification to the packets. >> >> I tried to ping machine 3 from machine 1 using the command like *ping >> 10.11.10.3*. However, it still does not success. >> This is because that before machine1 sends ping message to machine3, it >> will first send a ARP request message to get the mac address of machine3. >> machine3 gets that ARP request, and send the reply back (I use tcpdump to >> verify that machine3 gets the ARP request and send out the ARP reply). >> However, machine1 does not get the ARP reply. >> >> I checked that the bridge can only forwarding packet in one direction at >> the same time. it gets the ARP request but doesn't see the ARP reply >> (*pkt_queued* always returns 0 for one nic...). >> >> This behavior looks very weird to me. Do you think there is a >> compatibility >> issues between netmap and the os I am using? Is there a verified linux >> distribution (also the version) that perfectly works well with netmap? >> >> The OS I use is 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) >> x86_64 GNU/Linux. >> Linux kernel version is *3.16.0-4-amd64* >> >> >> Thanks! >> Xiaoye >> >> >> >> >> >> >> On Wed, Feb 3, 2016 at 2:12 AM, Luigi Rizzo wrote: >> >> > On Tue, Feb 2, 2016 at 10:48 PM, Xiaoye Sun >> wrote: >> > > >> > > >> > > On Mon, Feb 1, 2016 at 11:34 PM, Luigi Rizzo >> wrote: >> > >> >> > >> On Tue, Feb 2, 2016 at 6:23 AM, Xiaoye Sun >> wrote: >> > >> > Hi Luigi, >> > >> > >> > >> > I have to clarify about the *jumping issue* about the slot indexes. >> > >> > In the bridge.c program, the slot index never jumps and it >> increases >> > >> > sequentially. >> > >> > In the receiver.c program, the udp packet seq jumps and I showed >> the >> > >> > slot >> > >> > index that each udp packet uses. So the slot index jumps together >> with >> > >> > the >> > >> > udp seq (at the receiver program only). >> > >> >> > >> So let me understand, is the "slot" some information written >> > >> in the packet by bridge.c (referring to the rx or tx slot, >> > >> I am not sure) and then read and printed by receiver.c >> > >> (which gets the packet through recvfrom so there isn't >> > >> really any slot index) ? >> > >> >> > > It works in the other way: >> > > The bridge.c checks the seq numbers of the udp packets in netmap slots >> > (in >> > > nic rx ring) before the swap; then it records the seq number, slot >> > > number(both rx and tx (tx indexes were not shown in the previous email >> > since >> > > they all look correct)) and buf_idx (rx and tx). The bridge.c does not >> > > change anything in the buffer and it knows the slot and buf_idx that a >> > > packet uses. Please refer to the added code in *process_rings* >> function >> > > http://www.owlnet.rice.edu/~xs6/bridge.c >> > > The receiver.c checks the seq numbers only and print out the seq >> numbers >> > it >> > > receive sequentially. >> > > With these information, I manually match the seq number I got from >> > > receiver.c and the seq number I got from bridge.c. So we know what is >> the >> > > seq order the receiver sees and which slot a packet uses when bridge.c >> > swaps >> > > the buf_idxs. >> > > >> > >> Do you see any ordering inversion when the receiver >> > >> gets packets through the NETMAP API (e.g. using bridge.c >> > >> instead of receiver.c) ? >> > >> >> > > There is no ordering inversion seen by bridge.c (As I said in the >> > previous >> > > paragraph, the bridge.c checks the seq number and I did not see any >> order >> > > inversion in THIS simple experiment (In my multicast protocol >> (mentioned >> > in >> > > the first email), there is ordering inversion. But let us solve the >> > simple >> > > bridge.c's problem first. I think they are two relatively independent >> > > issues.)). >> > >> > Sorry there was a misunderstanding. >> > I wanted you to check the following setup: >> > >> > [1: send.c] ->- [2: bridge.c] ->- [3: XYZ] >> > >> > where in XYZ you replace your receiver.c with some >> > netmap-based receiver (it could be pkt-gen in rx mode, >> > or possibly even another instance of bridge.c where >> > you connect the output port to a vale switch so >> > traffic is dropped), and then in XYZ print the content >> > of the packets. >> > >> > From your previous report we know that node 2: sees packets >> > in order, and node 3: sees packets out of order. >> > However, if the problem were due to bridge.c sending >> > the old buffer and not the new one, you'd see not only >> > reordering but also replication of packets. >> > >> > The fact that you see only the reordering in 3: makes >> > me think that the problem is in that node, and it could >> > be the network stack in 3: that does something strange. >> > So if you can run something netmap based in 3: and make >> > sure there is only one queue to read from, we could >> > at least figure out what is going on. >> > >> > cheers >> > luigi >> > >> > >> > is that >> > > >> > >> >> > >> Are you using native netmap drivers or the emulated mode ? >> > >> You can check that by playing with the "admode" sysctl entry >> > >> (or sysfs on linux) - try setting to 1 and 2 and see if >> > >> the behaviour changes. >> > >> >> > >> dev.netmap.admode: 0 >> > >> Controls the use of native or emulated adapter mode. >> > >> 0 uses the best available option, >> > >> 1 forces native and fails if not available, >> > >> 2 forces emulated hence never fails. >> > >> >> > > I was using admode 0. I changed the admode to 1 and 2 using the >> command >> > like >> > > *echo 1 > /sys/module/netmap/parameters/admode* and restart the bridge >> > > program. The behavior keeps the same. >> > > >> > >> >> > >> cheers >> > >> luigi >> > >> >> > >> > >> > >> > There is really one ring (tx and rx) for NIC and one ring (tx and >> rx) >> > >> > for >> > >> > the host. >> > >> > I also doubt that there might be multiple tx rings for the host. It >> > >> > seems >> > >> > like that bridge program swap packet to multiple host rings and the >> > udp >> > >> > recv >> > >> > program drains packets from these rings. But this is not the case >> > here. >> > >> > >> > >> > The bridge program prints a line like this >> > >> > *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* >> > >> > this is printed by the following line the original program >> > >> > *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, >> > >> > pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, >> > >> > pb->first_rx_ring, >> > >> > pb->req.nr_rx_rings);* >> > >> > >> > >> > I think this shows that there is really one NIC ring and one HOST >> > ring. >> > >> > >> > >> > Is there another way to verify the number of ring that netmap has? >> > >> > >> > >> > Thanks! >> > >> > Xiaoye >> > >> > >> > >> > On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo >> > wrote: >> > >> >> >> > >> >> Hi, >> > >> >> there must be some wrong with your setting because >> > >> >> slot indexes must be sequential and in your case they >> > >> >> are not (see the jump from 295 to 474 and then >> > >> >> back from 485 to 296, and the numerous interleavings >> > >> >> that you are seeing later). >> > >> >> >> > >> >> I have no idea of the cause but typically this pattern >> > >> >> is what you see when there are multiple input rings and >> > >> >> not just one. >> > >> >> >> > >> >> Cheers >> > >> >> Luigi >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun >> > >> >> wrote: >> > >> >> > Hi Luigi, >> > >> >> > >> > >> >> > Thanks for the detailed advice. >> > >> >> > >> > >> >> > With more detailed experiments, actually I found that the udp >> > >> >> > sender/receiver packet reorder issue *might* be irrelevant to >> the >> > >> >> > original >> > >> >> > issue I posted. However, I think we should solve the udp >> > >> >> > sender/receiver >> > >> >> > issue first. >> > >> >> > I run the experiment with more detailed log. Here is my >> findings. >> > >> >> > >> > >> >> > 1. I am running a netmap version available since about Oct 13rd >> > from >> > >> >> > github >> > >> >> > (https://github.com/luigirizzo/netmap). So I think this is not >> the >> > >> >> > one >> > >> >> > related to the buffer allocation issue. I tried to running the >> > newest >> > >> >> > version, however, that version causes problem when I exit the >> > bridge >> > >> >> > program >> > >> >> > (something like kernel error which make the os crash). >> > >> >> > >> > >> >> > 2 & 3. I changed the receiver.c & bridge.c so that I can get >> more >> > >> >> > information (more detailed log). >> > >> >> > The reorder happens multiple times (about 10 times) within a >> > second. >> > >> >> > Here is >> > >> >> > one example trace collected from the above two programs. >> > (remembering >> > >> >> > that >> > >> >> > we have udp sender running on one machine; netmap bridge and udp >> > >> >> > receiver >> > >> >> > are running on another machine). >> > >> >> > There is only one pair of rings each with 512 slots (511 slot >> > usable) >> > >> >> > on >> > >> >> > the >> > >> >> > receiver machine. >> > >> >> > >> > >> >> > =================== packet trace collected from receiver.c >> > >> >> > =================== >> > >> >> > ===== together with the slot and buf_idx of the corresponding >> > netmap >> > >> >> > ring >> > >> >> > slots ====== >> > >> >> > [seq] [slot] [buf_idx] >> > >> >> > 8208 294 1833 >> > >> >> > 8209 295 1834 >> > >> >> > 8388 474 2013 >> > >> >> > ... (packet received in order) >> > >> >> > 8398 484 2023 >> > >> >> > 8399 485 2024 >> > >> >> > 8210 296 1835 >> > >> >> > 8211 297 1836 >> > >> >> > ... (packet received in order) >> > >> >> > ... >> > >> >> > 8222 308 1847 >> > >> >> > 8400 486 2025 >> > >> >> > 8223 309 1848 >> > >> >> > 8401 487 2026 >> > >> >> > 8224 310 1849 >> > >> >> > 8402 488 2027 >> > >> >> > 8225 311 1850 >> > >> >> > 8403 489 2028 >> > >> >> > 8226 312 1851 >> > >> >> > 8404 450 2029 >> > >> >> > 8227 313 1852 >> > >> >> > 8228 314 1853 >> > >> >> > >> =================================================================== >> > >> >> > As we can see that the udp receiver got packet 8210 after it got >> > >> >> > 8399, >> > >> >> > which >> > >> >> > is the first reorder. Then, the receiver got 8211 to 8222 >> > >> >> > sequentially. >> > >> >> > Then >> > >> >> > it got packet from 8223-8227 and 8400-8404 interleaved. >> > >> >> > >> > >> >> > >> > >> >> > ==================== event order seen by netmap bridge >> > >> >> > ================== >> > >> >> > get 8209 >> > >> >> > poll called >> > >> >> > get 8210 >> > >> >> > ... >> > >> >> > ... >> > >> >> > get 8228 >> > >> >> > poll called >> > >> >> > get 8229 >> > >> >> > ... >> > >> >> > ... >> > >> >> > get 8383 >> > >> >> > poll called >> > >> >> > get 8384 >> > >> >> > ... >> > >> >> > get 8387 >> > >> >> > poll called >> > >> >> > get 8388 >> > >> >> > ... >> > >> >> > get 8393 >> > >> >> > poll called >> > >> >> > get 8394 >> > >> >> > ... >> > >> >> > get 8399 >> > >> >> > poll called >> > >> >> > get 8400 >> > >> >> > ... >> > >> >> > get 8404 >> > >> >> > poll called >> > >> >> > get 8405 >> > >> >> > >> =================================================================== >> > >> >> > As we can see, from the event ordering see by the bridge.c, all >> the >> > >> >> > packets >> > >> >> > are receiver in order, which means the the reorder happens when >> the >> > >> >> > bridge >> > >> >> > code swap the buf_idx between the nic ring(slot) and the host >> > >> >> > ring(slot). >> > >> >> > The reordered seq usually right before or after the poll >> function >> > >> >> > call. >> > >> >> > >> > >> >> > Best, >> > >> >> > Xiaoye >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo < >> rizzo@iet.unipi.it> >> > >> >> > wrote: >> > >> >> >> >> > >> >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun < >> Xiaoye.Sun@rice.edu> >> > >> >> >> wrote: >> > >> >> >> > Hi Luigi, >> > >> >> >> > >> > >> >> >> > Thanks for your advice. >> > >> >> >> > I forgot to mention that I use the command "ethtool -L eth1 >> > >> >> >> > combined >> > >> >> >> > 1" >> > >> >> >> > to >> > >> >> >> > set the number of rings of the nic to 1. The host also only >> has >> > >> >> >> > one >> > >> >> >> > ring. >> > >> >> >> > I understand the situation where the first tx ring is full so >> > the >> > >> >> >> > bridge >> > >> >> >> > will swap the packets to the second tx ring and then the >> > host/nic >> > >> >> >> > might >> > >> >> >> > drain either rings. But this is not the case in the >> experiment. >> > >> >> >> >> > >> >> >> ok good to know that. >> > >> >> >> >> > >> >> >> So if we have ruled out multiqueue and iommu, let's look at >> > >> >> >> the internal allocator and at bridge.c >> > >> >> >> >> > >> >> >> 1. are you running the most recent version of netmap ? >> > >> >> >> Some older version (probably 1-2 years ago) had a bug >> > >> >> >> in the buffer allocator and some buffers were allocated >> > >> >> >> twice. >> > >> >> >> >> > >> >> >> 2. can you tweak your receiver.c to report some more info >> > >> >> >> on how often you get out of sequence packets, how much >> > >> >> >> out of sequence they are ? >> > >> >> >> Also it would be useful to report gaps on the increasing >> side >> > >> >> >> (i.e. new_seq != old_seq +1 ) >> > >> >> >> >> > >> >> >> 3. can you tweak bridge.c so that it writes into the packet >> > >> >> >> the netmap buffer indexes and slots on the rx and tx side, >> > >> >> >> so when you detect a sequence error we can figure out >> > >> >> >> where it is happening. >> > >> >> >> Ideally you could also add the sequence number detection >> > >> >> >> code in bridge.c so we can check whether the errors appear >> > >> >> >> on the input or output sides. >> > >> >> >> >> > >> >> >> cheers >> > >> >> >> luigi >> > >> >> >> >> > >> >> > >> > >> >> >> > >> >> >> > >> >> >> > >> >> -- >> > >> >> >> > >> >> >> > >> -----------------------------------------+------------------------------- >> > >> >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >> > >> >> dell'Informazione >> > >> >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> > >> >> TEL +39-050-2217533 . via Diotisalvi 2 >> > >> >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> > >> >> >> > >> >> >> > >> -----------------------------------------+------------------------------- >> > >> >> >> > >> > >> > >> >> > >> >> > >> >> > >> -- >> > >> >> > >> -----------------------------------------+------------------------------- >> > >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >> > dell'Informazione >> > >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> > >> TEL +39-050-2217533 . via Diotisalvi 2 >> > >> Mobile +39-338-6809875 . 56122 PISA (Italy) >> > >> >> > >> -----------------------------------------+------------------------------- >> > >> >> > > >> > >> > >> > >> > -- >> > >> -----------------------------------------+------------------------------- >> > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >> dell'Informazione >> > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >> > TEL +39-050-2217533 . via Diotisalvi 2 >> > Mobile +39-338-6809875 . 56122 PISA (Italy) >> > >> -----------------------------------------+------------------------------- >> > >> > >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> > From owner-freebsd-net@freebsd.org Fri Feb 5 00:26:03 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 151DAA9C5B6 for ; Fri, 5 Feb 2016 00:26:03 +0000 (UTC) (envelope-from victordetoni@gmail.com) Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com [IPv6:2607:f8b0:4001:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CAD51BBC for ; Fri, 5 Feb 2016 00:26:02 +0000 (UTC) (envelope-from victordetoni@gmail.com) Received: by mail-ig0-x235.google.com with SMTP id xg9so3397718igb.1 for ; Thu, 04 Feb 2016 16:26:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=couDqgVekbWQZkfWc+xI2XWoBeE33MKrxstyCNOYeWg=; b=ahQ/cqR5L3JY2wM4R+GcZkAsVQG2FESJPwPGfjTkXzu1Iv2+wr5zd6h4DBKyr0PKcF /Cfj5K+2mGWN85j8dFY/oOPRqXQQV3V/RMmgGC3eYHcGJIrlqG6pXhahXpO2a/NZIi9q 25tE/Em3Ll65E8hTXU9xsVork3SgJgbAz1IgejidhdR/eibiS/X/3RKNXvpca+GM3xkN sV+8mo4vdGiCwQ9hiej3P14Rdgz1yPrnrlcZyMCY7QR4cZPclqBvtHP4s1wFmQY24QDM GuV57XX/ZaCVqFe553YWP/xz7Bqx8immzGGzNNgloOcNMaYw1YRAn6MIQBbvmQw1FL2W 7JTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=couDqgVekbWQZkfWc+xI2XWoBeE33MKrxstyCNOYeWg=; b=SnsQagUaFQECfoDoIdFBCJjsW5e2WHaujWAJbCcrnwuPGh+JSNPfJrJwDCBu4Vuswb BAyEsH9a15EK1Izj2bL2OZP+Nfdp1HpRIel8RTGzwXsCFQdo2ZqWw1Z3+ZMyGwajhv10 uV4alauwMtO9VAO6BrYMpBLLMyf6eyP7c8S1y8yEQk8ihYJ/pjCrr1hFy9tF1v+5XzNz iLeXy65oXyhoataNdnHqkxUndldwyHWxjddSB9DLi1dyh5HB9rwbAflnahZNXj2i8tXY BhHTp49/aDwbMjBt6nVOCkua4De0eLeFNcHEHv/AGL4xL9z3acjrx0BqHnXUTWXvoXEL G6gQ== X-Gm-Message-State: AG10YOQmjZVdIOLOolMu1SdCqYybtZf/BeV84CSySveU9uoW0gwetVhRR6154+wp3H2Ew13cBs6XpdjcHVEYlg== MIME-Version: 1.0 X-Received: by 10.50.102.40 with SMTP id fl8mr12685562igb.85.1454631962026; Thu, 04 Feb 2016 16:26:02 -0800 (PST) Received: by 10.107.52.205 with HTTP; Thu, 4 Feb 2016 16:26:01 -0800 (PST) In-Reply-To: References: Date: Thu, 4 Feb 2016 22:26:01 -0200 Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Victor Detoni To: Xiaoye Sun Cc: Luigi Rizzo , Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 00:26:03 -0000 I'm sorry, I made mistake. To workaround this try `ip link set $IFACE promisc on` On Thu, Feb 4, 2016 at 10:04 PM, Xiaoye Sun wrote: > Yes. all the interfaces are up. Are you able to get ARP request when the > interfaces are down? > > > On Thursday, February 4, 2016, Victor Detoni > wrote: > >> Both interfaces are up? Like ifconfig... up >> >> I had this the same problem and I solve with commands above >> >> Em quinta-feira, 4 de fevereiro de 2016, Xiaoye Sun >> escreveu: >> >>> Hi Luigi, >>> >>> Thanks for your explanation. >>> >>> I used three machines to do this experiment. They are directly connected. >>> >>> [(machine1) eth1]---[eth2 (machine2) eth3]---[eth4 (machine3)]. >>> >>> First, I tried to run bridge.c on machine2 using the command *bridge -i >>> netmap:eth2 -i netmap:eth3*. (sender receiver or XYZ were not running on >>> machine 1or3) >>> >>> For my understanding, in this setup, machine2 will be transparent to >>> machine1&3 since it forwards packet from its eth2 to eth3 and vice versa >>> without any modification to the packets. >>> >>> I tried to ping machine 3 from machine 1 using the command like *ping >>> 10.11.10.3*. However, it still does not success. >>> This is because that before machine1 sends ping message to machine3, it >>> will first send a ARP request message to get the mac address of machine3. >>> machine3 gets that ARP request, and send the reply back (I use tcpdump to >>> verify that machine3 gets the ARP request and send out the ARP reply). >>> However, machine1 does not get the ARP reply. >>> >>> I checked that the bridge can only forwarding packet in one direction at >>> the same time. it gets the ARP request but doesn't see the ARP reply >>> (*pkt_queued* always returns 0 for one nic...). >>> >>> This behavior looks very weird to me. Do you think there is a >>> compatibility >>> issues between netmap and the os I am using? Is there a verified linux >>> distribution (also the version) that perfectly works well with netmap? >>> >>> The OS I use is 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) >>> x86_64 GNU/Linux. >>> Linux kernel version is *3.16.0-4-amd64* >>> >>> >>> Thanks! >>> Xiaoye >>> >>> >>> >>> >>> >>> >>> On Wed, Feb 3, 2016 at 2:12 AM, Luigi Rizzo wrote: >>> >>> > On Tue, Feb 2, 2016 at 10:48 PM, Xiaoye Sun >>> wrote: >>> > > >>> > > >>> > > On Mon, Feb 1, 2016 at 11:34 PM, Luigi Rizzo >>> wrote: >>> > >> >>> > >> On Tue, Feb 2, 2016 at 6:23 AM, Xiaoye Sun >>> wrote: >>> > >> > Hi Luigi, >>> > >> > >>> > >> > I have to clarify about the *jumping issue* about the slot >>> indexes. >>> > >> > In the bridge.c program, the slot index never jumps and it >>> increases >>> > >> > sequentially. >>> > >> > In the receiver.c program, the udp packet seq jumps and I showed >>> the >>> > >> > slot >>> > >> > index that each udp packet uses. So the slot index jumps together >>> with >>> > >> > the >>> > >> > udp seq (at the receiver program only). >>> > >> >>> > >> So let me understand, is the "slot" some information written >>> > >> in the packet by bridge.c (referring to the rx or tx slot, >>> > >> I am not sure) and then read and printed by receiver.c >>> > >> (which gets the packet through recvfrom so there isn't >>> > >> really any slot index) ? >>> > >> >>> > > It works in the other way: >>> > > The bridge.c checks the seq numbers of the udp packets in netmap >>> slots >>> > (in >>> > > nic rx ring) before the swap; then it records the seq number, slot >>> > > number(both rx and tx (tx indexes were not shown in the previous >>> email >>> > since >>> > > they all look correct)) and buf_idx (rx and tx). The bridge.c does >>> not >>> > > change anything in the buffer and it knows the slot and buf_idx that >>> a >>> > > packet uses. Please refer to the added code in *process_rings* >>> function >>> > > http://www.owlnet.rice.edu/~xs6/bridge.c >>> > > The receiver.c checks the seq numbers only and print out the seq >>> numbers >>> > it >>> > > receive sequentially. >>> > > With these information, I manually match the seq number I got from >>> > > receiver.c and the seq number I got from bridge.c. So we know what >>> is the >>> > > seq order the receiver sees and which slot a packet uses when >>> bridge.c >>> > swaps >>> > > the buf_idxs. >>> > > >>> > >> Do you see any ordering inversion when the receiver >>> > >> gets packets through the NETMAP API (e.g. using bridge.c >>> > >> instead of receiver.c) ? >>> > >> >>> > > There is no ordering inversion seen by bridge.c (As I said in the >>> > previous >>> > > paragraph, the bridge.c checks the seq number and I did not see any >>> order >>> > > inversion in THIS simple experiment (In my multicast protocol >>> (mentioned >>> > in >>> > > the first email), there is ordering inversion. But let us solve the >>> > simple >>> > > bridge.c's problem first. I think they are two relatively independent >>> > > issues.)). >>> > >>> > Sorry there was a misunderstanding. >>> > I wanted you to check the following setup: >>> > >>> > [1: send.c] ->- [2: bridge.c] ->- [3: XYZ] >>> > >>> > where in XYZ you replace your receiver.c with some >>> > netmap-based receiver (it could be pkt-gen in rx mode, >>> > or possibly even another instance of bridge.c where >>> > you connect the output port to a vale switch so >>> > traffic is dropped), and then in XYZ print the content >>> > of the packets. >>> > >>> > From your previous report we know that node 2: sees packets >>> > in order, and node 3: sees packets out of order. >>> > However, if the problem were due to bridge.c sending >>> > the old buffer and not the new one, you'd see not only >>> > reordering but also replication of packets. >>> > >>> > The fact that you see only the reordering in 3: makes >>> > me think that the problem is in that node, and it could >>> > be the network stack in 3: that does something strange. >>> > So if you can run something netmap based in 3: and make >>> > sure there is only one queue to read from, we could >>> > at least figure out what is going on. >>> > >>> > cheers >>> > luigi >>> > >>> > >>> > is that >>> > > >>> > >> >>> > >> Are you using native netmap drivers or the emulated mode ? >>> > >> You can check that by playing with the "admode" sysctl entry >>> > >> (or sysfs on linux) - try setting to 1 and 2 and see if >>> > >> the behaviour changes. >>> > >> >>> > >> dev.netmap.admode: 0 >>> > >> Controls the use of native or emulated adapter mode. >>> > >> 0 uses the best available option, >>> > >> 1 forces native and fails if not available, >>> > >> 2 forces emulated hence never fails. >>> > >> >>> > > I was using admode 0. I changed the admode to 1 and 2 using the >>> command >>> > like >>> > > *echo 1 > /sys/module/netmap/parameters/admode* and restart the >>> bridge >>> > > program. The behavior keeps the same. >>> > > >>> > >> >>> > >> cheers >>> > >> luigi >>> > >> >>> > >> > >>> > >> > There is really one ring (tx and rx) for NIC and one ring (tx and >>> rx) >>> > >> > for >>> > >> > the host. >>> > >> > I also doubt that there might be multiple tx rings for the host. >>> It >>> > >> > seems >>> > >> > like that bridge program swap packet to multiple host rings and >>> the >>> > udp >>> > >> > recv >>> > >> > program drains packets from these rings. But this is not the case >>> > here. >>> > >> > >>> > >> > The bridge program prints a line like this >>> > >> > *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* >>> > >> > this is printed by the following line the original program >>> > >> > *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, >>> > >> > pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, >>> > >> > pb->first_rx_ring, >>> > >> > pb->req.nr_rx_rings);* >>> > >> > >>> > >> > I think this shows that there is really one NIC ring and one HOST >>> > ring. >>> > >> > >>> > >> > Is there another way to verify the number of ring that netmap has? >>> > >> > >>> > >> > Thanks! >>> > >> > Xiaoye >>> > >> > >>> > >> > On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo >>> > wrote: >>> > >> >> >>> > >> >> Hi, >>> > >> >> there must be some wrong with your setting because >>> > >> >> slot indexes must be sequential and in your case they >>> > >> >> are not (see the jump from 295 to 474 and then >>> > >> >> back from 485 to 296, and the numerous interleavings >>> > >> >> that you are seeing later). >>> > >> >> >>> > >> >> I have no idea of the cause but typically this pattern >>> > >> >> is what you see when there are multiple input rings and >>> > >> >> not just one. >>> > >> >> >>> > >> >> Cheers >>> > >> >> Luigi >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun >> > >>> > >> >> wrote: >>> > >> >> > Hi Luigi, >>> > >> >> > >>> > >> >> > Thanks for the detailed advice. >>> > >> >> > >>> > >> >> > With more detailed experiments, actually I found that the udp >>> > >> >> > sender/receiver packet reorder issue *might* be irrelevant to >>> the >>> > >> >> > original >>> > >> >> > issue I posted. However, I think we should solve the udp >>> > >> >> > sender/receiver >>> > >> >> > issue first. >>> > >> >> > I run the experiment with more detailed log. Here is my >>> findings. >>> > >> >> > >>> > >> >> > 1. I am running a netmap version available since about Oct 13rd >>> > from >>> > >> >> > github >>> > >> >> > (https://github.com/luigirizzo/netmap). So I think this is >>> not the >>> > >> >> > one >>> > >> >> > related to the buffer allocation issue. I tried to running the >>> > newest >>> > >> >> > version, however, that version causes problem when I exit the >>> > bridge >>> > >> >> > program >>> > >> >> > (something like kernel error which make the os crash). >>> > >> >> > >>> > >> >> > 2 & 3. I changed the receiver.c & bridge.c so that I can get >>> more >>> > >> >> > information (more detailed log). >>> > >> >> > The reorder happens multiple times (about 10 times) within a >>> > second. >>> > >> >> > Here is >>> > >> >> > one example trace collected from the above two programs. >>> > (remembering >>> > >> >> > that >>> > >> >> > we have udp sender running on one machine; netmap bridge and >>> udp >>> > >> >> > receiver >>> > >> >> > are running on another machine). >>> > >> >> > There is only one pair of rings each with 512 slots (511 slot >>> > usable) >>> > >> >> > on >>> > >> >> > the >>> > >> >> > receiver machine. >>> > >> >> > >>> > >> >> > =================== packet trace collected from receiver.c >>> > >> >> > =================== >>> > >> >> > ===== together with the slot and buf_idx of the corresponding >>> > netmap >>> > >> >> > ring >>> > >> >> > slots ====== >>> > >> >> > [seq] [slot] [buf_idx] >>> > >> >> > 8208 294 1833 >>> > >> >> > 8209 295 1834 >>> > >> >> > 8388 474 2013 >>> > >> >> > ... (packet received in order) >>> > >> >> > 8398 484 2023 >>> > >> >> > 8399 485 2024 >>> > >> >> > 8210 296 1835 >>> > >> >> > 8211 297 1836 >>> > >> >> > ... (packet received in order) >>> > >> >> > ... >>> > >> >> > 8222 308 1847 >>> > >> >> > 8400 486 2025 >>> > >> >> > 8223 309 1848 >>> > >> >> > 8401 487 2026 >>> > >> >> > 8224 310 1849 >>> > >> >> > 8402 488 2027 >>> > >> >> > 8225 311 1850 >>> > >> >> > 8403 489 2028 >>> > >> >> > 8226 312 1851 >>> > >> >> > 8404 450 2029 >>> > >> >> > 8227 313 1852 >>> > >> >> > 8228 314 1853 >>> > >> >> > >>> =================================================================== >>> > >> >> > As we can see that the udp receiver got packet 8210 after it >>> got >>> > >> >> > 8399, >>> > >> >> > which >>> > >> >> > is the first reorder. Then, the receiver got 8211 to 8222 >>> > >> >> > sequentially. >>> > >> >> > Then >>> > >> >> > it got packet from 8223-8227 and 8400-8404 interleaved. >>> > >> >> > >>> > >> >> > >>> > >> >> > ==================== event order seen by netmap bridge >>> > >> >> > ================== >>> > >> >> > get 8209 >>> > >> >> > poll called >>> > >> >> > get 8210 >>> > >> >> > ... >>> > >> >> > ... >>> > >> >> > get 8228 >>> > >> >> > poll called >>> > >> >> > get 8229 >>> > >> >> > ... >>> > >> >> > ... >>> > >> >> > get 8383 >>> > >> >> > poll called >>> > >> >> > get 8384 >>> > >> >> > ... >>> > >> >> > get 8387 >>> > >> >> > poll called >>> > >> >> > get 8388 >>> > >> >> > ... >>> > >> >> > get 8393 >>> > >> >> > poll called >>> > >> >> > get 8394 >>> > >> >> > ... >>> > >> >> > get 8399 >>> > >> >> > poll called >>> > >> >> > get 8400 >>> > >> >> > ... >>> > >> >> > get 8404 >>> > >> >> > poll called >>> > >> >> > get 8405 >>> > >> >> > >>> =================================================================== >>> > >> >> > As we can see, from the event ordering see by the bridge.c, >>> all the >>> > >> >> > packets >>> > >> >> > are receiver in order, which means the the reorder happens >>> when the >>> > >> >> > bridge >>> > >> >> > code swap the buf_idx between the nic ring(slot) and the host >>> > >> >> > ring(slot). >>> > >> >> > The reordered seq usually right before or after the poll >>> function >>> > >> >> > call. >>> > >> >> > >>> > >> >> > Best, >>> > >> >> > Xiaoye >>> > >> >> > >>> > >> >> > >>> > >> >> > >>> > >> >> > >>> > >> >> > >>> > >> >> > >>> > >> >> > >>> > >> >> > >>> > >> >> > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo < >>> rizzo@iet.unipi.it> >>> > >> >> > wrote: >>> > >> >> >> >>> > >> >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun < >>> Xiaoye.Sun@rice.edu> >>> > >> >> >> wrote: >>> > >> >> >> > Hi Luigi, >>> > >> >> >> > >>> > >> >> >> > Thanks for your advice. >>> > >> >> >> > I forgot to mention that I use the command "ethtool -L eth1 >>> > >> >> >> > combined >>> > >> >> >> > 1" >>> > >> >> >> > to >>> > >> >> >> > set the number of rings of the nic to 1. The host also >>> only has >>> > >> >> >> > one >>> > >> >> >> > ring. >>> > >> >> >> > I understand the situation where the first tx ring is full >>> so >>> > the >>> > >> >> >> > bridge >>> > >> >> >> > will swap the packets to the second tx ring and then the >>> > host/nic >>> > >> >> >> > might >>> > >> >> >> > drain either rings. But this is not the case in the >>> experiment. >>> > >> >> >> >>> > >> >> >> ok good to know that. >>> > >> >> >> >>> > >> >> >> So if we have ruled out multiqueue and iommu, let's look at >>> > >> >> >> the internal allocator and at bridge.c >>> > >> >> >> >>> > >> >> >> 1. are you running the most recent version of netmap ? >>> > >> >> >> Some older version (probably 1-2 years ago) had a bug >>> > >> >> >> in the buffer allocator and some buffers were allocated >>> > >> >> >> twice. >>> > >> >> >> >>> > >> >> >> 2. can you tweak your receiver.c to report some more info >>> > >> >> >> on how often you get out of sequence packets, how much >>> > >> >> >> out of sequence they are ? >>> > >> >> >> Also it would be useful to report gaps on the increasing >>> side >>> > >> >> >> (i.e. new_seq != old_seq +1 ) >>> > >> >> >> >>> > >> >> >> 3. can you tweak bridge.c so that it writes into the packet >>> > >> >> >> the netmap buffer indexes and slots on the rx and tx side, >>> > >> >> >> so when you detect a sequence error we can figure out >>> > >> >> >> where it is happening. >>> > >> >> >> Ideally you could also add the sequence number detection >>> > >> >> >> code in bridge.c so we can check whether the errors appear >>> > >> >> >> on the input or output sides. >>> > >> >> >> >>> > >> >> >> cheers >>> > >> >> >> luigi >>> > >> >> >> >>> > >> >> > >>> > >> >> >>> > >> >> >>> > >> >> >>> > >> >> -- >>> > >> >> >>> > >> >> >>> > >>> -----------------------------------------+------------------------------- >>> > >> >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >>> > >> >> dell'Informazione >>> > >> >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>> > >> >> TEL +39-050-2217533 . via Diotisalvi 2 >>> > >> >> Mobile +39-338-6809875 . 56122 PISA (Italy) >>> > >> >> >>> > >> >> >>> > >>> -----------------------------------------+------------------------------- >>> > >> >> >>> > >> > >>> > >> >>> > >> >>> > >> >>> > >> -- >>> > >> >>> > >>> -----------------------------------------+------------------------------- >>> > >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >>> > dell'Informazione >>> > >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>> > >> TEL +39-050-2217533 . via Diotisalvi 2 >>> > >> Mobile +39-338-6809875 . 56122 PISA (Italy) >>> > >> >>> > >>> -----------------------------------------+------------------------------- >>> > >> >>> > > >>> > >>> > >>> > >>> > -- >>> > >>> -----------------------------------------+------------------------------- >>> > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >>> dell'Informazione >>> > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>> > TEL +39-050-2217533 . via Diotisalvi 2 >>> > Mobile +39-338-6809875 . 56122 PISA (Italy) >>> > >>> -----------------------------------------+------------------------------- >>> > >>> > >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> >> From owner-freebsd-net@freebsd.org Fri Feb 5 01:02:31 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AC37A9D2AF for ; Fri, 5 Feb 2016 01:02:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0D56D1EB8 for ; Fri, 5 Feb 2016 01:02:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u1512UAM019932 for ; Fri, 5 Feb 2016 01:02:30 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206904] tailq crash/nd inet6 Date: Fri, 05 Feb 2016 01:02:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 01:02:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206904 --- Comment #2 from Larry Rosenman --- Created attachment 166582 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D166582&action= =3Dedit another one --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 01:02:50 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1FB27A9D2DF for ; Fri, 5 Feb 2016 01:02:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1256D1F68 for ; Fri, 5 Feb 2016 01:02:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u1512n0B020418 for ; Fri, 5 Feb 2016 01:02:49 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206904] tailq crash/nd inet6 Date: Fri, 05 Feb 2016 01:02:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 01:02:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206904 --- Comment #3 from Larry Rosenman --- Created attachment 166583 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D166583&action= =3Dedit and a 3rd --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 01:03:26 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97D2FA9D31F for ; Fri, 5 Feb 2016 01:03:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8A91783 for ; Fri, 5 Feb 2016 01:03:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u1513Q3g021291 for ; Fri, 5 Feb 2016 01:03:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206904] tailq crash/nd inet6 Date: Fri, 05 Feb 2016 01:03:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 01:03:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206904 --- Comment #4 from Larry Rosenman --- vmcore's are ALL available, and I can give a @FreeBSD.org dev access. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 01:08:30 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F058DA9D4CA for ; Fri, 5 Feb 2016 01:08:30 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id DD33B758 for ; Fri, 5 Feb 2016 01:08:30 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id D928F107605; Fri, 5 Feb 2016 01:08:30 +0000 (UTC) Date: Fri, 5 Feb 2016 01:08:30 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Commented On] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <243dd8530e3e230f0767b21add2009fe@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFaz9g4= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 01:08:31 -0000 sepherosa_gmail.com added inline comments. INLINE COMMENTS sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c:455 OK, I will split it out. REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, transport, gallatin Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Fri Feb 5 01:14:03 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7012FA9D771 for ; Fri, 5 Feb 2016 01:14:03 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 5CBDEB15 for ; Fri, 5 Feb 2016 01:14:03 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 58B3610799E; Fri, 5 Feb 2016 01:14:03 +0000 (UTC) Date: Fri, 5 Feb 2016 01:14:03 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Commented On] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <47c267ddddb7d0405fff0ce47250d91a@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFaz91s= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 01:14:03 -0000 sepherosa_gmail.com added a comment. I will adjust the patch accordingly. INLINE COMMENTS sys/netinet/tcp_lro.c:655 Sure :) sys/netinet/tcp_lro.c:684 Sounds fine to me. I did the byte limit before (https://reviews.freebsd.org/D4825). But it turns out the ACKs need seperate limit (append count based). To make them consistent, I changed the original patch to use append count too. REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, transport, gallatin Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Fri Feb 5 02:09:19 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6131A9C8A2 for ; Fri, 5 Feb 2016 02:09:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 96DA06CE for ; Fri, 5 Feb 2016 02:09:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u1529JaX052151 for ; Fri, 5 Feb 2016 02:09:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206932] Realtek 8111 card stops responding under high load in netmap mode Date: Fri, 05 Feb 2016 02:09:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: software-freebsd@interfasys.ch X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: version Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 02:09:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206932 Olivier - interfaSys s=C3=A0rl changed: What |Removed |Added ---------------------------------------------------------------------------- Version|10.2-STABLE |11.0-CURRENT --- Comment #1 from Olivier - interfaSys s=C3=A0rl --- I've just tested on 11-CURRENT and got the same results. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 03:00:28 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCC31A9D6E9 for ; Fri, 5 Feb 2016 03:00:28 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id AA19C1A27 for ; Fri, 5 Feb 2016 03:00:28 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id A800D10742D; Fri, 5 Feb 2016 03:00:28 +0000 (UTC) Date: Fri, 5 Feb 2016 03:00:28 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Updated, 114 lines] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <3c07f1fd368196912e774c3de154a663@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFa0EEw= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_3c07f1fd368196912e774c3de154a663" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 03:00:29 -0000 --b1_3c07f1fd368196912e774c3de154a663 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit sepherosa_gmail.com updated the summary for this revision. sepherosa_gmail.com updated this revision to Diff 13028. sepherosa_gmail.com added a comment. Address gallatin and adrian's concern. CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5185?vs=12995&id=13028 REVISION DETAIL https://reviews.freebsd.org/D5185 AFFECTED FILES sys/dev/hyperv/netvsc/hv_net_vsc.h sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c sys/netinet/tcp_lro.c sys/netinet/tcp_lro.h EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, gallatin, transport Cc: freebsd-virtualization-list, freebsd-net-list --b1_3c07f1fd368196912e774c3de154a663 Content-Type: text/x-patch; charset=utf-8; name="D5185.13028.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5185.13028.patch" ZGlmZiAtLWdpdCBhL3N5cy9uZXRpbmV0L3RjcF9scm8uaCBiL3N5cy9uZXRpbmV0L3RjcF9scm8u aAotLS0gYS9zeXMvbmV0aW5ldC90Y3BfbHJvLmgKKysrIGIvc3lzL25ldGluZXQvdGNwX2xyby5o CkBAIC05MSwxMSArOTEsMTYgQEAKIAl1bnNpZ25lZAlscm9fY250OwogCXVuc2lnbmVkCWxyb19t YnVmX2NvdW50OwogCXVuc2lnbmVkCWxyb19tYnVmX21heDsKKwl1bnNpZ25lZCBzaG9ydAlscm9f YWNrY250X2xpbTsJCS8qIG1heCAjIG9mIGFnZ3JlZ2F0ZWQgQUNLcyAqLworCXVuc2lnbmVkIHNo b3J0CWxyb19sZW5ndGhfbGltOwkJLyogbWF4IGxlbiBvZiBhZ2dyZWdhdGVkIGRhdGEgKi8KIAog CXN0cnVjdCBscm9faGVhZAlscm9fYWN0aXZlOwogCXN0cnVjdCBscm9faGVhZAlscm9fZnJlZTsK IH07CiAKKyNkZWZpbmUJVENQX0xST19MRU5HVEhfTUFYCTY1NTM1CisjZGVmaW5lCVRDUF9MUk9f QUNLQ05UX01BWAk2NTUzNQkJLyogdW5saW1pdGVkICovCisKIGludCB0Y3BfbHJvX2luaXQoc3Ry dWN0IGxyb19jdHJsICopOwogaW50IHRjcF9scm9faW5pdF9hcmdzKHN0cnVjdCBscm9fY3RybCAq LCBzdHJ1Y3QgaWZuZXQgKiwgdW5zaWduZWQsIHVuc2lnbmVkKTsKIHZvaWQgdGNwX2xyb19mcmVl KHN0cnVjdCBscm9fY3RybCAqKTsKZGlmZiAtLWdpdCBhL3N5cy9uZXRpbmV0L3RjcF9scm8uYyBi L3N5cy9uZXRpbmV0L3RjcF9scm8uYwotLS0gYS9zeXMvbmV0aW5ldC90Y3BfbHJvLmMKKysrIGIv c3lzL25ldGluZXQvdGNwX2xyby5jCkBAIC04Nyw2ICs4Nyw4IEBACiAJbGMtPmxyb19tYnVmX2Nv dW50ID0gMDsKIAlsYy0+bHJvX21idWZfbWF4ID0gbHJvX21idWZzOwogCWxjLT5scm9fY250ID0g bHJvX2VudHJpZXM7CisJbGMtPmxyb19hY2tjbnRfbGltID0gVENQX0xST19BQ0tDTlRfTUFYOwor CWxjLT5scm9fbGVuZ3RoX2xpbSA9IFRDUF9MUk9fTEVOR1RIX01BWDsKIAlsYy0+aWZwID0gaWZw OwogCVNMSVNUX0lOSVQoJmxjLT5scm9fZnJlZSk7CiAJU0xJU1RfSU5JVCgmbGMtPmxyb19hY3Rp dmUpOwpAQCAtNjA4LDcgKzYxMCw3IEBACiAJCX0KIAogCQkvKiBGbHVzaCBub3cgaWYgYXBwZW5k aW5nIHdpbGwgcmVzdWx0IGluIG92ZXJmbG93LiAqLwotCQlpZiAobGUtPnBfbGVuID4gKDY1NTM1 IC0gdGNwX2RhdGFfbGVuKSkgeworCQlpZiAobGUtPnBfbGVuID4gKGxjLT5scm9fbGVuZ3RoX2xp bSAtIHRjcF9kYXRhX2xlbikpIHsKIAkJCVNMSVNUX1JFTU9WRSgmbGMtPmxyb19hY3RpdmUsIGxl LCBscm9fZW50cnksIG5leHQpOwogCQkJdGNwX2xyb19mbHVzaChsYywgbGUpOwogCQkJYnJlYWs7 CkBAIC02NDYsNiArNjQ4LDE1IEBACiAKIAkJaWYgKHRjcF9kYXRhX2xlbiA9PSAwKSB7CiAJCQlt X2ZyZWVtKG0pOworCQkJLyoKKwkJCSAqIEZsdXNoIHRoaXMgTFJPIGVudHJ5LCBpZiB0aGlzIEFD SyBzaG91bGQgbm90CisJCQkgKiBiZSBmdXJ0aGVyIGRlbGF5ZWQuCisJCQkgKi8KKwkJCWlmIChs ZS0+YXBwZW5kX2NudCA+PSBsYy0+bHJvX2Fja2NudF9saW0pIHsKKwkJCQlTTElTVF9SRU1PVkUo JmxjLT5scm9fYWN0aXZlLCBsZSwgbHJvX2VudHJ5LAorCQkJCSAgICBuZXh0KTsKKwkJCQl0Y3Bf bHJvX2ZsdXNoKGxjLCBsZSk7CisJCQl9CiAJCQlyZXR1cm4gKDApOwogCQl9CiAKQEAgLTY2Niw3 ICs2NzcsNyBAQAogCQkgKiBJZiBhIHBvc3NpYmxlIG5leHQgZnVsbCBsZW5ndGggcGFja2V0IHdv dWxkIGNhdXNlIGFuCiAJCSAqIG92ZXJmbG93LCBwcm8tYWN0aXZlbHkgZmx1c2ggbm93LgogCQkg Ki8KLQkJaWYgKGxlLT5wX2xlbiA+ICg2NTUzNSAtIGxjLT5pZnAtPmlmX210dSkpIHsKKwkJaWYg KGxlLT5wX2xlbiA+IChsYy0+bHJvX2xlbmd0aF9saW0gLSBsYy0+aWZwLT5pZl9tdHUpKSB7CiAJ CQlTTElTVF9SRU1PVkUoJmxjLT5scm9fYWN0aXZlLCBsZSwgbHJvX2VudHJ5LCBuZXh0KTsKIAkJ CXRjcF9scm9fZmx1c2gobGMsIGxlKTsKIAkJfSBlbHNlCmRpZmYgLS1naXQgYS9zeXMvZGV2L2h5 cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMgYi9zeXMvZGV2L2h5cGVydi9uZXR2 c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNkLmMKLS0tIGEvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2 X25ldHZzY19kcnZfZnJlZWJzZC5jCisrKyBiL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXR2 c2NfZHJ2X2ZyZWVic2QuYwpAQCAtMTc2LDE0ICsxNzYsMTEgQEAKICNkZWZpbmUgSE5fQ1NVTV9B U1NJU1RfV0lOOAkoQ1NVTV9UQ1ApCiAjZGVmaW5lIEhOX0NTVU1fQVNTSVNUCQkoQ1NVTV9JUCB8 IENTVU1fVURQIHwgQ1NVTV9UQ1ApCiAKLS8qIFhYWCBtb3ZlIHRvIG5ldGluZXQvdGNwX2xyby5o ICovCi0jZGVmaW5lIEhOX0xST19ISVdBVF9NQVgJCQkJNjU1MzUKLSNkZWZpbmUgSE5fTFJPX0hJ V0FUX0RFRgkJCQlITl9MUk9fSElXQVRfTUFYCisjZGVmaW5lIEhOX0xST19MRU5MSU1fREVGCQko MjUgKiBFVEhFUk1UVSkKIC8qIFlZWSAyKk1UVSBpcyBhIGJpdCByb3VnaCwgYnV0IHNob3VsZCBi ZSBnb29kIGVub3VnaC4gKi8KLSNkZWZpbmUgSE5fTFJPX0hJV0FUX01UVUxJTShpZnApCQkJKDIg KiAoaWZwKS0+aWZfbXR1KQotI2RlZmluZSBITl9MUk9fSElXQVRfSVNWQUxJRChzYywgaGl3YXQp CQkJXAotICAgICgoaGl3YXQpID49IEhOX0xST19ISVdBVF9NVFVMSU0oKHNjKS0+aG5faWZwKSB8 fAlcCi0gICAgIChoaXdhdCkgPD0gSE5fTFJPX0hJV0FUX01BWCkKKyNkZWZpbmUgSE5fTFJPX0xF TkxJTV9NSU4oaWZwKQkJKDIgKiAoaWZwKS0+aWZfbXR1KQorCisjZGVmaW5lIEhOX0xST19BQ0tD TlRfREVGCQkxCiAKIC8qCiAgKiBCZSBhd2FyZSB0aGF0IHRoaXMgc2xlZXBhYmxlIG11dGV4IHdp bGwgZXhoaWJpdCBXSVRORVNTIGVycm9ycyB3aGVuCkBAIC0yNTMsOSArMjUwLDggQEAKIHN0YXRp YyB2b2lkIGhuX3N0YXJ0X3R4ZW9mKHN0cnVjdCBpZm5ldCAqaWZwKTsKIHN0YXRpYyBpbnQgaG5f aWZtZWRpYV91cGQoc3RydWN0IGlmbmV0ICppZnApOwogc3RhdGljIHZvaWQgaG5faWZtZWRpYV9z dHMoc3RydWN0IGlmbmV0ICppZnAsIHN0cnVjdCBpZm1lZGlhcmVxICppZm1yKTsKLSNpZmRlZiBI Tl9MUk9fSElXQVQKLXN0YXRpYyBpbnQgaG5fbHJvX2hpd2F0X3N5c2N0bChTWVNDVExfSEFORExF Ul9BUkdTKTsKLSNlbmRpZgorc3RhdGljIGludCBobl9scm9fbGVubGltX3N5c2N0bChTWVNDVExf SEFORExFUl9BUkdTKTsKK3N0YXRpYyBpbnQgaG5fbHJvX2Fja2NudF9zeXNjdGwoU1lTQ1RMX0hB TkRMRVJfQVJHUyk7CiBzdGF0aWMgaW50IGhuX3RydXN0X2hjc3VtX3N5c2N0bChTWVNDVExfSEFO RExFUl9BUkdTKTsKIHN0YXRpYyBpbnQgaG5fdHhfY2hpbW5leV9zaXplX3N5c2N0bChTWVNDVExf SEFORExFUl9BUkdTKTsKIHN0YXRpYyBpbnQgaG5fY2hlY2tfaXBsZW4oY29uc3Qgc3RydWN0IG1i dWYgKiwgaW50KTsKQEAgLTI2NSwxNSArMjYxLDYgQEAKIHN0YXRpYyB2b2lkIGhuX3R4ZW9mX3Rh c2tmdW5jKHZvaWQgKnhzYywgaW50IHBlbmRpbmcpOwogc3RhdGljIGludCBobl9lbmNhcChzdHJ1 Y3QgaG5fc29mdGMgKiwgc3RydWN0IGhuX3R4ZGVzYyAqLCBzdHJ1Y3QgbWJ1ZiAqKik7CiAKLXN0 YXRpYyBfX2lubGluZSB2b2lkCi1obl9zZXRfbHJvX2hpd2F0KHN0cnVjdCBobl9zb2Z0YyAqc2Ms IGludCBoaXdhdCkKLXsKLQlzYy0+aG5fbHJvX2hpd2F0ID0gaGl3YXQ7Ci0jaWZkZWYgSE5fTFJP X0hJV0FUCi0Jc2MtPmhuX2xyby5scm9faGl3YXQgPSBzYy0+aG5fbHJvX2hpd2F0OwotI2VuZGlm Ci19Ci0KIHN0YXRpYyBpbnQKIGhuX2lmbWVkaWFfdXBkKHN0cnVjdCBpZm5ldCAqaWZwIF9fdW51 c2VkKQogewpAQCAtMzU4LDcgKzM0NSw2IEBACiAJYnplcm8oc2MsIHNpemVvZihobl9zb2Z0Y190 KSk7CiAJc2MtPmhuX3VuaXQgPSB1bml0OwogCXNjLT5obl9kZXYgPSBkZXY7Ci0Jc2MtPmhuX2xy b19oaXdhdCA9IEhOX0xST19ISVdBVF9ERUY7CiAJc2MtPmhuX2RpcmVjdF90eF9zaXplID0gaG5f ZGlyZWN0X3R4X3NpemU7CiAJaWYgKGhuX3RydXN0X2hvc3R0Y3ApCiAJCXNjLT5obl90cnVzdF9o Y3N1bSB8PSBITl9UUlVTVF9IQ1NVTV9UQ1A7CkBAIC00NDIsOSArNDI4LDggQEAKIAkvKiBEcml2 ZXIgcHJpdmF0ZSBMUk8gc2V0dGluZ3MgKi8KIAlzYy0+aG5fbHJvLmlmcCA9IGlmcDsKICNlbmRp ZgotI2lmZGVmIEhOX0xST19ISVdBVAotCXNjLT5obl9scm8ubHJvX2hpd2F0ID0gc2MtPmhuX2xy b19oaXdhdDsKLSNlbmRpZgorCXNjLT5obl9scm8ubHJvX2xlbmd0aF9saW0gPSBITl9MUk9fTEVO TElNX0RFRjsKKwlzYy0+aG5fbHJvLmxyb19hY2tjbnRfbGltID0gSE5fTFJPX0FDS0NOVF9ERUY7 CiAjZW5kaWYJLyogSU5FVCB8fCBJTkVUNiAqLwogCiAjaWYgX19GcmVlQlNEX3ZlcnNpb24gPj0g MTEwMDA0NQpAQCAtNDgwLDExICs0NjUsMTIgQEAKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9s cm8ubHJvX2ZsdXNoZWQsIDAsICJMUk8gZmx1c2hlZCIpOwogCVNZU0NUTF9BRERfVUxPTkcoY3R4 LCBjaGlsZCwgT0lEX0FVVE8sICJscm9fdHJpZWQiLAogCSAgICBDVExGTEFHX1JXLCAmc2MtPmhu X2xyb190cmllZCwgIiMgb2YgTFJPIHRyaWVzIik7Ci0jaWZkZWYgSE5fTFJPX0hJV0FUCi0JU1lT Q1RMX0FERF9QUk9DKGN0eCwgY2hpbGQsIE9JRF9BVVRPLCAibHJvX2hpd2F0IiwKLQkgICAgQ1RM VFlQRV9JTlQgfCBDVExGTEFHX1JXLCBzYywgMCwgaG5fbHJvX2hpd2F0X3N5c2N0bCwKLQkgICAg IkkiLCAiTFJPIGhpZ2ggd2F0ZXJtYXJrIik7Ci0jZW5kaWYKKwlTWVNDVExfQUREX1BST0MoY3R4 LCBjaGlsZCwgT0lEX0FVVE8sICJscm9fbGVuZ3RoX2xpbSIsCisJICAgIENUTFRZUEVfSU5UIHwg Q1RMRkxBR19SVywgc2MsIDAsIGhuX2xyb19sZW5saW1fc3lzY3RsLCAiSSIsCisJICAgICJNYXgg IyBvZiBkYXRhIGJ5dGVzIHRvIGJlIGFnZ3JlZ2F0ZWQgYnkgTFJPIik7CisJU1lTQ1RMX0FERF9Q Uk9DKGN0eCwgY2hpbGQsIE9JRF9BVVRPLCAibHJvX2Fja2NudF9saW0iLAorCSAgICBDVExUWVBF X0lOVCB8IENUTEZMQUdfUlcsIHNjLCAwLCBobl9scm9fYWNrY250X3N5c2N0bCwgIkkiLAorCSAg ICAiTWF4ICMgb2YgQUNLcyB0byBiZSBhZ2dyZWdhdGVkIGJ5IExSTyIpOwogCVNZU0NUTF9BRERf UFJPQyhjdHgsIGNoaWxkLCBPSURfQVVUTywgInRydXN0X2hvc3R0Y3AiLAogCSAgICBDVExUWVBF X0lOVCB8IENUTEZMQUdfUlcsIHNjLCBITl9UUlVTVF9IQ1NVTV9UQ1AsCiAJICAgIGhuX3RydXN0 X2hjc3VtX3N5c2N0bCwgIkkiLApAQCAtMTQxMCwxMiArMTM5NiwxMyBAQAogCiAJCS8qIE9idGFp biBhbmQgcmVjb3JkIHJlcXVlc3RlZCBNVFUgKi8KIAkJaWZwLT5pZl9tdHUgPSBpZnItPmlmcl9t dHU7CisKIAkJLyoKLQkJICogTWFrZSBzdXJlIHRoYXQgTFJPIGhpZ2ggd2F0ZXJtYXJrIGlzIHN0 aWxsIHZhbGlkLAotCQkgKiBhZnRlciBNVFUgY2hhbmdlICh0aGUgMipNVFUgbGltaXQpLgorCQkg KiBNYWtlIHN1cmUgdGhhdCBMUk8gYWdncmVnYXRpb24gbGVuZ3RoIGxpbWl0IGlzIHN0aWxsCisJ CSAqIHZhbGlkLCBhZnRlciB0aGUgTVRVIGNoYW5nZS4KIAkJICovCi0JCWlmICghSE5fTFJPX0hJ V0FUX0lTVkFMSUQoc2MsIHNjLT5obl9scm9faGl3YXQpKQotCQkJaG5fc2V0X2xyb19oaXdhdChz YywgSE5fTFJPX0hJV0FUX01UVUxJTShpZnApKTsKKwkJaWYgKHNjLT5obl9scm8ubHJvX2xlbmd0 aF9saW0gPCBITl9MUk9fTEVOTElNX01JTihpZnApKQorCQkJc2MtPmhuX2xyby5scm9fbGVuZ3Ro X2xpbSA9IEhOX0xST19MRU5MSU1fTUlOKGlmcCk7CiAKIAkJZG8gewogCQkJTlZfTE9DSyhzYyk7 CkBAIC0xNzIyLDI2ICsxNzA5LDUwIEBACiB9CiAjZW5kaWYKIAotI2lmZGVmIEhOX0xST19ISVdB VAogc3RhdGljIGludAotaG5fbHJvX2hpd2F0X3N5c2N0bChTWVNDVExfSEFORExFUl9BUkdTKQor aG5fbHJvX2xlbmxpbV9zeXNjdGwoU1lTQ1RMX0hBTkRMRVJfQVJHUykKK3sKKwlzdHJ1Y3QgaG5f c29mdGMgKnNjID0gYXJnMTsKKwlpbnQgbGVubGltLCBlcnJvcjsKKworCWxlbmxpbSA9IHNjLT5o bl9scm8ubHJvX2xlbmd0aF9saW07CisJZXJyb3IgPSBzeXNjdGxfaGFuZGxlX2ludChvaWRwLCAm bGVubGltLCAwLCByZXEpOworCWlmIChlcnJvciB8fCByZXEtPm5ld3B0ciA9PSBOVUxMKQorCQly ZXR1cm4gZXJyb3I7CisKKwlpZiAobGVubGltIDwgSE5fTFJPX0xFTkxJTV9NSU4oc2MtPmhuX2lm cCkgfHwKKwkgICAgbGVubGltID4gVENQX0xST19MRU5HVEhfTUFYKQorCQlyZXR1cm4gRUlOVkFM OworCisJc2MtPmhuX2xyby5scm9fbGVuZ3RoX2xpbSA9IGxlbmxpbTsKKwlyZXR1cm4gMDsKK30K Kworc3RhdGljIGludAoraG5fbHJvX2Fja2NudF9zeXNjdGwoU1lTQ1RMX0hBTkRMRVJfQVJHUykK IHsKIAlzdHJ1Y3QgaG5fc29mdGMgKnNjID0gYXJnMTsKLQlpbnQgaGl3YXQsIGVycm9yOworCWlu dCBhY2tjbnQsIGVycm9yOwogCi0JaGl3YXQgPSBzYy0+aG5fbHJvX2hpd2F0OwotCWVycm9yID0g c3lzY3RsX2hhbmRsZV9pbnQob2lkcCwgJmhpd2F0LCAwLCByZXEpOworCS8qCisJICogbHJvX2Fj a2NudF9saW0gaXMgYXBwZW5kIGNvdW50IGxpbWl0LAorCSAqICsxIHRvIHR1cm4gaXQgaW50byBh Z2dyZWdhdGlvbiBsaW1pdC4KKwkgKi8KKwlhY2tjbnQgPSBzYy0+aG5fbHJvLmxyb19hY2tjbnRf bGltICsgMTsKKwllcnJvciA9IHN5c2N0bF9oYW5kbGVfaW50KG9pZHAsICZhY2tjbnQsIDAsIHJl cSk7CiAJaWYgKGVycm9yIHx8IHJlcS0+bmV3cHRyID09IE5VTEwpCiAJCXJldHVybiBlcnJvcjsK IAotCWlmICghSE5fTFJPX0hJV0FUX0lTVkFMSUQoc2MsIGhpd2F0KSkKKwlpZiAoYWNrY250IDwg MiB8fCBhY2tjbnQgPiAoVENQX0xST19BQ0tDTlRfTUFYICsgMSkpCiAJCXJldHVybiBFSU5WQUw7 CiAKLQlpZiAoc2MtPmhuX2xyb19oaXdhdCAhPSBoaXdhdCkKLQkJaG5fc2V0X2xyb19oaXdhdChz YywgaGl3YXQpOworCS8qCisJICogQ29udmVydCBhZ2dyZWdhdGlvbiBsaW1pdCBiYWNrIHRvIGFw cGVuZAorCSAqIGNvdW50IGxpbWl0LgorCSAqLworCXNjLT5obl9scm8ubHJvX2Fja2NudF9saW0g PSBhY2tjbnQgLSAxOwogCXJldHVybiAwOwogfQotI2VuZGlmCS8qIEhOX0xST19ISVdBVCAqLwog CiBzdGF0aWMgaW50CiBobl90cnVzdF9oY3N1bV9zeXNjdGwoU1lTQ1RMX0hBTkRMRVJfQVJHUykK ZGlmZiAtLWdpdCBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmggYi9zeXMvZGV2 L2h5cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5oCi0tLSBhL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9o dl9uZXRfdnNjLmgKKysrIGIvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuaApAQCAt MTAzMCw3ICsxMDMwLDYgQEAKIAlzdHJ1Y3QgdGFzawlobl90eGVvZl90YXNrOwogCiAJc3RydWN0 IGxyb19jdHJsCWhuX2xybzsKLQlpbnQJCWhuX2xyb19oaXdhdDsKIAogCS8qIFRydXN0IGNzdW0g dmVyaWZpY2F0aW9uIG9uIGhvc3Qgc2lkZSAqLwogCWludAkJaG5fdHJ1c3RfaGNzdW07CS8qIEhO X1RSVVNUX0hDU1VNXyAqLwoK --b1_3c07f1fd368196912e774c3de154a663-- From owner-freebsd-net@freebsd.org Fri Feb 5 04:04:12 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68F50A9ABFF for ; Fri, 5 Feb 2016 04:04:12 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 53E551E89 for ; Fri, 5 Feb 2016 04:04:12 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 51C974FFE; Fri, 5 Feb 2016 04:04:12 +0000 (UTC) Date: Fri, 5 Feb 2016 04:04:12 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5085+325+cccb179d0eef693e@reviews.freebsd.org Subject: [Differential] [Closed] D5085: hyperv/hn: Avoid duplicate csum features settings Message-ID: <9e31de93171b333264053c0b45ba41ca@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5085: hyperv/hn: Avoid duplicate csum features settings X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NDFiODMwNDM2YTQwYmUxYTY0MTNmMzBhZjFiIFa0Hzw= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_9e31de93171b333264053c0b45ba41ca" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 04:04:12 -0000 --b1_9e31de93171b333264053c0b45ba41ca Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295296: hyperv/hn: Avoid duplicate csum features settings (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5085?vs=12744&id=13030#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5085?vs=12744&id=13030 REVISION DETAIL https://reviews.freebsd.org/D5085 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_net_vsc.h head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, howard0su_gmail.com, honzhan_microsoft.com, adrian, network Cc: freebsd-net-list --b1_9e31de93171b333264053c0b45ba41ca Content-Type: text/x-patch; charset=utf-8; name="D5085.13030.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5085.13030.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTE3Niw2ICsxNzYsMTQgQEAKICAgICBDU1VNX0lQX0lTQ1NJfENTVU1fSVA2X1VEUHxD U1VNX0lQNl9UQ1B8Q1NVTV9JUDZfU0NUUHwJCVwKICAgICBDU1VNX0lQNl9UU098Q1NVTV9JUDZf SVNDU0kpCiAKKy8qCisgKiBPbmx5IGVuYWJsZSBVRFAgY2hlY2tzdW0gb2ZmbG9hZGluZyB3aGVu IGl0IGlzIG9uIDIwMTJSMiBvcgorICogbGF0ZXIuICBVRFAgY2hlY2tzdW0gb2ZmbG9hZGluZyBk b2Vzbid0IHdvcmsgb24gZWFybGllcgorICogV2luZG93cyByZWxlYXNlcy4KKyAqLworI2RlZmlu ZSBITl9DU1VNX0FTU0lTVF9XSU44CShDU1VNX1RDUCkKKyNkZWZpbmUgSE5fQ1NVTV9BU1NJU1QJ CShDU1VNX1VEUCB8IENTVU1fVENQKQorCiAvKiBYWFggbW92ZSB0byBuZXRpbmV0L3RjcF9scm8u aCAqLwogI2RlZmluZSBITl9MUk9fSElXQVRfTUFYCQkJCTY1NTM1CiAjZGVmaW5lIEhOX0xST19I SVdBVF9ERUYJCQkJSE5fTFJPX0hJV0FUX01BWApAQCAtNDQ0LDE1ICs0NTIsMTIgQEAKIAlpZnAt PmlmX2NhcGVuYWJsZSB8PQogCSAgICBJRkNBUF9WTEFOX0hXVEFHR0lORyB8IElGQ0FQX1ZMQU5f TVRVIHwgSUZDQVBfSFdDU1VNIHwgSUZDQVBfVFNPIHwKIAkgICAgSUZDQVBfTFJPOwotCS8qCi0J ICogT25seSBlbmFibGUgVURQIGNoZWNrc3VtIG9mZmxvYWRpbmcgd2hlbiBpdCBpcyBvbiAyMDEy UjIgb3IKLQkgKiBsYXRlci4gVURQIGNoZWNrc3VtIG9mZmxvYWRpbmcgZG9lc24ndCB3b3JrIG9u IGVhcmxpZXIKLQkgKiBXaW5kb3dzIHJlbGVhc2VzLgotCSAqLworCiAJaWYgKGh2X3ZtYnVzX3By b3RvY2FsX3ZlcnNpb24gPj0gSFZfVk1CVVNfVkVSU0lPTl9XSU44XzEpCi0JCWlmcC0+aWZfaHdh c3Npc3QgPSBDU1VNX1RDUCB8IENTVU1fVURQIHwgQ1NVTV9UU087CisJCXNjLT5obl9jc3VtX2Fz c2lzdCA9IEhOX0NTVU1fQVNTSVNUOwogCWVsc2UKLQkJaWZwLT5pZl9od2Fzc2lzdCA9IENTVU1f VENQIHwgQ1NVTV9UU087CisJCXNjLT5obl9jc3VtX2Fzc2lzdCA9IEhOX0NTVU1fQVNTSVNUX1dJ Tjg7CisJaWZwLT5pZl9od2Fzc2lzdCA9IHNjLT5obl9jc3VtX2Fzc2lzdCB8IENTVU1fVFNPOwog CiAJZXJyb3IgPSBodl9yZl9vbl9kZXZpY2VfYWRkKGRldmljZV9jdHgsICZkZXZpY2VfaW5mbyk7 CiAJaWYgKGVycm9yKQpAQCAtMTUwNiw0NyArMTUxMSw0MCBAQAogCQllcnJvciA9IDA7CiAJCWJy ZWFrOwogCWNhc2UgU0lPQ1NJRkNBUDoKKwkJTlZfTE9DSyhzYyk7CisKIAkJbWFzayA9IGlmci0+ aWZyX3JlcWNhcCBeIGlmcC0+aWZfY2FwZW5hYmxlOwogCQlpZiAobWFzayAmIElGQ0FQX1RYQ1NV TSkgewotCQkJaWYgKElGQ0FQX1RYQ1NVTSAmIGlmcC0+aWZfY2FwZW5hYmxlKSB7Ci0JCQkJaWZw LT5pZl9jYXBlbmFibGUgJj0gfklGQ0FQX1RYQ1NVTTsKLQkJCQlpZnAtPmlmX2h3YXNzaXN0ICY9 IH4oQ1NVTV9UQ1AgfCBDU1VNX1VEUCk7Ci0JCQl9IGVsc2UgewotCQkJCWlmcC0+aWZfY2FwZW5h YmxlIHw9IElGQ0FQX1RYQ1NVTTsKLQkJCQkvKgotCQkJCSAqIE9ubHkgZW5hYmxlIFVEUCBjaGVj a3N1bSBvZmZsb2FkaW5nIG9uCi0JCQkJICogV2luZG93cyBTZXJ2ZXIgMjAxMlIyIG9yIGxhdGVy IHJlbGVhc2VzLgotCQkJCSAqLwotCQkJCWlmIChodl92bWJ1c19wcm90b2NhbF92ZXJzaW9uID49 Ci0JCQkJICAgIEhWX1ZNQlVTX1ZFUlNJT05fV0lOOF8xKSB7Ci0JCQkJCWlmcC0+aWZfaHdhc3Np c3QgfD0KLQkJCQkJICAgIChDU1VNX1RDUCB8IENTVU1fVURQKTsKLQkJCQl9IGVsc2UgewotCQkJ CQlpZnAtPmlmX2h3YXNzaXN0IHw9IENTVU1fVENQOwotCQkJCX0KLQkJCX0KKwkJCWlmcC0+aWZf Y2FwZW5hYmxlIF49IElGQ0FQX1RYQ1NVTTsKKwkJCWlmIChpZnAtPmlmX2NhcGVuYWJsZSAmIElG Q0FQX1RYQ1NVTSkKKwkJCQlpZnAtPmlmX2h3YXNzaXN0IHw9IHNjLT5obl9jc3VtX2Fzc2lzdDsK KwkJCWVsc2UKKwkJCQlpZnAtPmlmX2h3YXNzaXN0ICY9IH5zYy0+aG5fY3N1bV9hc3Npc3Q7CiAJ CX0KIAotCQlpZiAobWFzayAmIElGQ0FQX1JYQ1NVTSkgewotCQkJaWYgKElGQ0FQX1JYQ1NVTSAm IGlmcC0+aWZfY2FwZW5hYmxlKSB7Ci0JCQkJaWZwLT5pZl9jYXBlbmFibGUgJj0gfklGQ0FQX1JY Q1NVTTsKLQkJCX0gZWxzZSB7Ci0JCQkJaWZwLT5pZl9jYXBlbmFibGUgfD0gSUZDQVBfUlhDU1VN OwotCQkJfQotCQl9CisJCWlmIChtYXNrICYgSUZDQVBfUlhDU1VNKQorCQkJaWZwLT5pZl9jYXBl bmFibGUgXj0gSUZDQVBfUlhDU1VNOworCiAJCWlmIChtYXNrICYgSUZDQVBfTFJPKQogCQkJaWZw LT5pZl9jYXBlbmFibGUgXj0gSUZDQVBfTFJPOwogCiAJCWlmIChtYXNrICYgSUZDQVBfVFNPNCkg ewogCQkJaWZwLT5pZl9jYXBlbmFibGUgXj0gSUZDQVBfVFNPNDsKLQkJCWlmcC0+aWZfaHdhc3Np c3QgXj0gQ1NVTV9JUF9UU087CisJCQlpZiAoaWZwLT5pZl9jYXBlbmFibGUgJiBJRkNBUF9UU080 KQorCQkJCWlmcC0+aWZfaHdhc3Npc3QgfD0gQ1NVTV9JUF9UU087CisJCQllbHNlCisJCQkJaWZw LT5pZl9od2Fzc2lzdCAmPSB+Q1NVTV9JUF9UU087CiAJCX0KIAogCQlpZiAobWFzayAmIElGQ0FQ X1RTTzYpIHsKIAkJCWlmcC0+aWZfY2FwZW5hYmxlIF49IElGQ0FQX1RTTzY7Ci0JCQlpZnAtPmlm X2h3YXNzaXN0IF49IENTVU1fSVA2X1RTTzsKKwkJCWlmIChpZnAtPmlmX2NhcGVuYWJsZSAmIElG Q0FQX1RTTzYpCisJCQkJaWZwLT5pZl9od2Fzc2lzdCB8PSBDU1VNX0lQNl9UU087CisJCQllbHNl CisJCQkJaWZwLT5pZl9od2Fzc2lzdCAmPSB+Q1NVTV9JUDZfVFNPOwogCQl9CiAKKwkJTlZfVU5M T0NLKHNjKTsKIAkJZXJyb3IgPSAwOwogCQlicmVhazsKIAljYXNlIFNJT0NBRERNVUxUSToKZGlm ZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuaCBiL2hlYWQv c3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuaAotLS0gYS9oZWFkL3N5cy9kZXYvaHlw ZXJ2L25ldHZzYy9odl9uZXRfdnNjLmgKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2Mv aHZfbmV0X3ZzYy5oCkBAIC0xMDQzLDYgKzEwNDMsOCBAQAogCXVfbG9uZwkJaG5fdHhkbWFfZmFp bGVkOwogCXVfbG9uZwkJaG5fdHhfY29sbGFwc2VkOwogCXVfbG9uZwkJaG5fdHhfY2hpbW5leTsK KworCXVpbnQ2NF90CWhuX2NzdW1fYXNzaXN0OwogfSBobl9zb2Z0Y190OwogCiAKCg== --b1_9e31de93171b333264053c0b45ba41ca-- From owner-freebsd-net@freebsd.org Fri Feb 5 04:10:24 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83799A9AE25 for ; Fri, 5 Feb 2016 04:10:24 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 6EABF20C for ; Fri, 5 Feb 2016 04:10:24 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 6E77C10726C; Fri, 5 Feb 2016 04:10:24 +0000 (UTC) Date: Fri, 5 Feb 2016 04:10:24 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5098+325+4e694d74d281089c@reviews.freebsd.org Subject: [Differential] [Closed] D5098: hyperv/hn: Reorganize TX csum offloading Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5098: hyperv/hn: Reorganize TX csum offloading X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: MWY5NTZkMGRiYzcxOTg0MWVmNWQ1NWNiZWRhIFa0ILA= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_cdd90e305a9c697abe3d4ccee544f60b" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 04:10:24 -0000 --b1_cdd90e305a9c697abe3d4ccee544f60b Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295297: hyperv/hn: Reorganize TX csum offloading (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5098?vs=12774&id=13031#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5098?vs=12774&id=13031 REVISION DETAIL https://reviews.freebsd.org/D5098 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_net_vsc.h head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, howard0su_gmail.com, honzhan_microsoft.com, adrian, network Cc: freebsd-net-list --b1_cdd90e305a9c697abe3d4ccee544f60b Content-Type: text/x-patch; charset=utf-8; name="D5098.13031.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5098.13031.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTE2NywxNiArMTY3LDYgQEAKICNkZWZpbmUgSE5fVFhEX0ZMQUdfRE1BTUFQCTB4Mgog CiAvKgotICogQSB1bmlmaWVkIGZsYWcgZm9yIGFsbCBvdXRib3VuZCBjaGVjayBzdW0gZmxhZ3Mg aXMgdXNlZnVsLAotICogYW5kIGl0IGhlbHBzIGF2b2lkaW5nIHVubmVjZXNzYXJ5IGNoZWNrIHN1 bSBjYWxjdWxhdGlvbiBpbgotICogbmV0d29yayBmb3J3YXJkaW5nIHNjZW5hcmlvLgotICovCi0j ZGVmaW5lIEhWX0NTVU1fRk9SX09VVEJPVU5ECQkJCQkJXAotICAgIChDU1VNX0lQfENTVU1fSVBf VURQfENTVU1fSVBfVENQfENTVU1fSVBfU0NUUHxDU1VNX0lQX1RTT3wJCVwKLSAgICBDU1VNX0lQ X0lTQ1NJfENTVU1fSVA2X1VEUHxDU1VNX0lQNl9UQ1B8Q1NVTV9JUDZfU0NUUHwJCVwKLSAgICBD U1VNX0lQNl9UU098Q1NVTV9JUDZfSVNDU0kpCi0KLS8qCiAgKiBPbmx5IGVuYWJsZSBVRFAgY2hl Y2tzdW0gb2ZmbG9hZGluZyB3aGVuIGl0IGlzIG9uIDIwMTJSMiBvcgogICogbGF0ZXIuICBVRFAg Y2hlY2tzdW0gb2ZmbG9hZGluZyBkb2Vzbid0IHdvcmsgb24gZWFybGllcgogICogV2luZG93cyBy ZWxlYXNlcy4KQEAgLTI2NSw2MiArMjU1LDYgQEAKICNlbmRpZgogfQogCi0vKgotICogTmV0VnNj IGdldCBtZXNzYWdlIHRyYW5zcG9ydCBwcm90b2NvbCB0eXBlIAotICovCi1zdGF0aWMgdWludDMy X3QgZ2V0X3RyYW5zcG9ydF9wcm90b190eXBlKHN0cnVjdCBtYnVmICptX2hlYWQpCi17Ci0JdWlu dDMyX3QgcmV0X3ZhbCA9IFRSQU5TUE9SVF9UWVBFX05PVF9JUDsKLQl1aW50MTZfdCBldGhlcl90 eXBlID0gMDsKLQlpbnQgZXRoZXJfbGVuID0gMDsKLQlzdHJ1Y3QgZXRoZXJfdmxhbl9oZWFkZXIg KmVoOwotI2lmZGVmIElORVQKLQlzdHJ1Y3QgaXAgKmlwaDsKLSNlbmRpZgotI2lmZGVmIElORVQ2 Ci0Jc3RydWN0IGlwNl9oZHIgKmlwNjsKLSNlbmRpZgotCi0JZWggPSBtdG9kKG1faGVhZCwgc3Ry dWN0IGV0aGVyX3ZsYW5faGVhZGVyKik7Ci0JaWYgKGVoLT5ldmxfZW5jYXBfcHJvdG8gPT0gaHRv bnMoRVRIRVJUWVBFX1ZMQU4pKSB7Ci0JCWV0aGVyX2xlbiA9IEVUSEVSX0hEUl9MRU4gKyBFVEhF Ul9WTEFOX0VOQ0FQX0xFTjsKLQkJZXRoZXJfdHlwZSA9IGVoLT5ldmxfcHJvdG87Ci0JfSBlbHNl IHsKLQkJZXRoZXJfbGVuID0gRVRIRVJfSERSX0xFTjsKLQkJZXRoZXJfdHlwZSA9IGVoLT5ldmxf ZW5jYXBfcHJvdG87Ci0JfQotCi0Jc3dpdGNoIChudG9ocyhldGhlcl90eXBlKSkgewotI2lmZGVm IElORVQ2Ci0JY2FzZSBFVEhFUlRZUEVfSVBWNjoKLQkJaXA2ID0gKHN0cnVjdCBpcDZfaGRyICop KG1faGVhZC0+bV9kYXRhICsgZXRoZXJfbGVuKTsKLQotCQlpZiAoSVBQUk9UT19UQ1AgPT0gaXA2 LT5pcDZfbnh0KSB7Ci0JCQlyZXRfdmFsID0gVFJBTlNQT1JUX1RZUEVfSVBWNl9UQ1A7Ci0JCX0g ZWxzZSBpZiAoSVBQUk9UT19VRFAgPT0gaXA2LT5pcDZfbnh0KSB7Ci0JCQlyZXRfdmFsID0gVFJB TlNQT1JUX1RZUEVfSVBWNl9VRFA7Ci0JCX0KLQkJYnJlYWs7Ci0jZW5kaWYKLSNpZmRlZiBJTkVU Ci0JY2FzZSBFVEhFUlRZUEVfSVA6Ci0JCWlwaCA9IChzdHJ1Y3QgaXAgKikobV9oZWFkLT5tX2Rh dGEgKyBldGhlcl9sZW4pOwotCi0JCWlmIChJUFBST1RPX1RDUCA9PSBpcGgtPmlwX3ApIHsKLQkJ CXJldF92YWwgPSBUUkFOU1BPUlRfVFlQRV9JUFY0X1RDUDsKLQkJfSBlbHNlIGlmIChJUFBST1RP X1VEUCA9PSBpcGgtPmlwX3ApIHsKLQkJCXJldF92YWwgPSBUUkFOU1BPUlRfVFlQRV9JUFY0X1VE UDsKLQkJfQotCQlicmVhazsKLSNlbmRpZgotCWRlZmF1bHQ6Ci0JCXJldF92YWwgPSBUUkFOU1BP UlRfVFlQRV9OT1RfSVA7Ci0JCWJyZWFrOwotCX0KLQotCXJldHVybiAocmV0X3ZhbCk7Ci19Ci0K IHN0YXRpYyBpbnQKIGhuX2lmbWVkaWFfdXBkKHN0cnVjdCBpZm5ldCAqaWZwIF9fdW51c2VkKQog ewpAQCAtNzgzLDE2ICs3MTcsMTMgQEAKIAlobl9zb2Z0Y190ICpzYyA9IGlmcC0+aWZfc29mdGM7 CiAJc3RydWN0IGh2X2RldmljZSAqZGV2aWNlX2N0eCA9IHZtYnVzX2dldF9kZXZjdHgoc2MtPmhu X2Rldik7CiAJbmV0dnNjX2RldiAqbmV0X2RldiA9IHNjLT5uZXRfZGV2OwotCXN0cnVjdCBldGhl cl92bGFuX2hlYWRlciAqZWg7CiAJcm5kaXNfbXNnICpybmRpc19tZXNnOwogCXJuZGlzX3BhY2tl dCAqcm5kaXNfcGt0OwogCXJuZGlzX3Blcl9wYWNrZXRfaW5mbyAqcnBwaTsKIAluZGlzXzgwMjFx X2luZm8gKnJwcGlfdmxhbl9pbmZvOwogCXJuZGlzX3RjcF9pcF9jc3VtX2luZm8gKmNzdW1faW5m bzsKIAlybmRpc190Y3BfdHNvX2luZm8gKnRzb19pbmZvOwkKLQlpbnQgZXRoZXJfbGVuOwogCXVp bnQzMl90IHJuZGlzX21zZ19zaXplID0gMDsKLQl1aW50MzJfdCB0cmFuc19wcm90b190eXBlOwog CiAJaWYgKChpZnAtPmlmX2Rydl9mbGFncyAmIChJRkZfRFJWX1JVTk5JTkcgfCBJRkZfRFJWX09B Q1RJVkUpKSAhPQogCSAgICBJRkZfRFJWX1JVTk5JTkcpCkBAIC04NzIsMTAxICs4MDMsNzggQEAK IAkJCSAgICBtX2hlYWQtPm1fcGt0aGRyLmV0aGVyX3Z0YWcgJiAweGZmZjsKIAkJfQogCi0JCS8q IE9ubHkgY2hlY2sgdGhlIGZsYWdzIGZvciBvdXRib3VuZCBhbmQgaWdub3JlIHRoZSBvbmVzIGZv ciBpbmJvdW5kICovCi0JCWlmICgwID09IChtX2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBI Vl9DU1VNX0ZPUl9PVVRCT1VORCkpIHsKLQkJCWdvdG8gcHJlX3NlbmQ7Ci0JCX0KLQotCQllaCA9 IG10b2QobV9oZWFkLCBzdHJ1Y3QgZXRoZXJfdmxhbl9oZWFkZXIqKTsKLQkJaWYgKGVoLT5ldmxf ZW5jYXBfcHJvdG8gPT0gaHRvbnMoRVRIRVJUWVBFX1ZMQU4pKSB7Ci0JCQlldGhlcl9sZW4gPSBF VEhFUl9IRFJfTEVOICsgRVRIRVJfVkxBTl9FTkNBUF9MRU47Ci0JCX0gZWxzZSB7Ci0JCQlldGhl cl9sZW4gPSBFVEhFUl9IRFJfTEVOOwotCQl9Ci0KLQkJdHJhbnNfcHJvdG9fdHlwZSA9IGdldF90 cmFuc3BvcnRfcHJvdG9fdHlwZShtX2hlYWQpOwotCQlpZiAoVFJBTlNQT1JUX1RZUEVfTk9UX0lQ ID09IHRyYW5zX3Byb3RvX3R5cGUpIHsKLQkJCWdvdG8gcHJlX3NlbmQ7Ci0JCX0KLQotCQkvKgot CQkgKiBUU08gcGFja2V0IG5lZWRsZXNzIHRvIHNldHVwIHRoZSBzZW5kIHNpZGUgY2hlY2tzdW0K LQkJICogb2ZmbG9hZC4KLQkJICovCiAJCWlmIChtX2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3Mg JiBDU1VNX1RTTykgewotCQkJZ290byBkb190c287Ci0JCX0KKwkJCXN0cnVjdCBldGhlcl92bGFu X2hlYWRlciAqZWg7CisJCQlpbnQgZXRoZXJfbGVuOwogCi0JCS8qIHNldHVwIGNoZWNrc3VtIG9m ZmxvYWQgKi8KLQkJcm5kaXNfbXNnX3NpemUgKz0gUk5ESVNfQ1NVTV9QUElfU0laRTsKLQkJcnBw aSA9IGh2X3NldF9ycHBpX2RhdGEocm5kaXNfbWVzZywgUk5ESVNfQ1NVTV9QUElfU0laRSwKLQkJ ICAgIHRjcGlwX2Noa3N1bV9pbmZvKTsKLQkJY3N1bV9pbmZvID0gKHJuZGlzX3RjcF9pcF9jc3Vt X2luZm8gKikoKGNoYXIqKXJwcGkgKwotCQkgICAgcnBwaS0+cGVyX3BhY2tldF9pbmZvX29mZnNl dCk7Ci0KLQkJaWYgKHRyYW5zX3Byb3RvX3R5cGUgJiAoVFlQRV9JUFY0IDw8IDE2KSkgewotCQkJ Y3N1bV9pbmZvLT54bWl0LmlzX2lwdjQgPSAxOwotCQl9IGVsc2UgewotCQkJY3N1bV9pbmZvLT54 bWl0LmlzX2lwdjYgPSAxOwotCQl9CisJCQllaCA9IG10b2QobV9oZWFkLCBzdHJ1Y3QgZXRoZXJf dmxhbl9oZWFkZXIqKTsKKwkJCWlmIChlaC0+ZXZsX2VuY2FwX3Byb3RvID09IGh0b25zKEVUSEVS VFlQRV9WTEFOKSkgeworCQkJCWV0aGVyX2xlbiA9IEVUSEVSX0hEUl9MRU4gKworCQkJCSAgICBF VEhFUl9WTEFOX0VOQ0FQX0xFTjsKKwkJCX0gZWxzZSB7CisJCQkJZXRoZXJfbGVuID0gRVRIRVJf SERSX0xFTjsKKwkJCX0KIAotCQlpZiAodHJhbnNfcHJvdG9fdHlwZSAmIFRZUEVfVENQKSB7Ci0J CQljc3VtX2luZm8tPnhtaXQudGNwX2NzdW0gPSAxOwotCQkJY3N1bV9pbmZvLT54bWl0LnRjcF9o ZWFkZXJfb2Zmc2V0ID0gMDsKLQkJfSBlbHNlIGlmICh0cmFuc19wcm90b190eXBlICYgVFlQRV9V RFApIHsKLQkJCWNzdW1faW5mby0+eG1pdC51ZHBfY3N1bSA9IDE7Ci0JCX0KKwkJCXJuZGlzX21z Z19zaXplICs9IFJORElTX1RTT19QUElfU0laRTsKKwkJCXJwcGkgPSBodl9zZXRfcnBwaV9kYXRh KHJuZGlzX21lc2csIFJORElTX1RTT19QUElfU0laRSwKKwkJCSAgICB0Y3BfbGFyZ2Vfc2VuZF9p bmZvKTsKIAotCQlnb3RvIHByZV9zZW5kOworCQkJdHNvX2luZm8gPSAocm5kaXNfdGNwX3Rzb19p bmZvICopKChjaGFyICopcnBwaSArCisJCQkgICAgcnBwaS0+cGVyX3BhY2tldF9pbmZvX29mZnNl dCk7CisJCQl0c29faW5mby0+bHNvX3YyX3htaXQudHlwZSA9CisJCQkgICAgUk5ESVNfVENQX0xB UkdFX1NFTkRfT0ZGTE9BRF9WMl9UWVBFOwogCi1kb190c286Ci0JCS8qIHNldHVwIFRDUCBzZWdt ZW50YXRpb24gb2ZmbG9hZCAqLwotCQlybmRpc19tc2dfc2l6ZSArPSBSTkRJU19UU09fUFBJX1NJ WkU7Ci0JCXJwcGkgPSBodl9zZXRfcnBwaV9kYXRhKHJuZGlzX21lc2csIFJORElTX1RTT19QUElf U0laRSwKLQkJICAgIHRjcF9sYXJnZV9zZW5kX2luZm8pOwotCQkKLQkJdHNvX2luZm8gPSAocm5k aXNfdGNwX3Rzb19pbmZvICopKChjaGFyICopcnBwaSArCi0JCSAgICBycHBpLT5wZXJfcGFja2V0 X2luZm9fb2Zmc2V0KTsKLQkJdHNvX2luZm8tPmxzb192Ml94bWl0LnR5cGUgPQotCQkgICAgUk5E SVNfVENQX0xBUkdFX1NFTkRfT0ZGTE9BRF9WMl9UWVBFOwotCQkKICNpZmRlZiBJTkVUCi0JCWlm ICh0cmFuc19wcm90b190eXBlICYgKFRZUEVfSVBWNCA8PCAxNikpIHsKLQkJCXN0cnVjdCBpcCAq aXAgPQotCQkJICAgIChzdHJ1Y3QgaXAgKikobV9oZWFkLT5tX2RhdGEgKyBldGhlcl9sZW4pOwot CQkJdW5zaWduZWQgbG9uZyBpcGhfbGVuID0gaXAtPmlwX2hsIDw8IDI7Ci0JCQlzdHJ1Y3QgdGNw aGRyICp0aCA9Ci0JCQkgICAgKHN0cnVjdCB0Y3BoZHIgKikoKGNhZGRyX3QpaXAgKyBpcGhfbGVu KTsKLQkJCi0JCQl0c29faW5mby0+bHNvX3YyX3htaXQuaXBfdmVyc2lvbiA9Ci0JCQkgICAgUk5E SVNfVENQX0xBUkdFX1NFTkRfT0ZGTE9BRF9JUFY0OwotCQkJaXAtPmlwX2xlbiA9IDA7Ci0JCQlp cC0+aXBfc3VtID0gMDsKLQkJCi0JCQl0aC0+dGhfc3VtID0gaW5fcHNldWRvKGlwLT5pcF9zcmMu c19hZGRyLAotCQkJICAgIGlwLT5pcF9kc3Quc19hZGRyLAotCQkJICAgIGh0b25zKElQUFJPVE9f VENQKSk7Ci0JCX0KKwkJCWlmIChtX2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBDU1VNX0lQ X1RTTykgeworCQkJCXN0cnVjdCBpcCAqaXAgPQorCQkJCSAgICAoc3RydWN0IGlwICopKG1faGVh ZC0+bV9kYXRhICsgZXRoZXJfbGVuKTsKKwkJCQl1bnNpZ25lZCBsb25nIGlwaF9sZW4gPSBpcC0+ aXBfaGwgPDwgMjsKKwkJCQlzdHJ1Y3QgdGNwaGRyICp0aCA9CisJCQkJICAgIChzdHJ1Y3QgdGNw aGRyICopKChjYWRkcl90KWlwICsgaXBoX2xlbik7CisJCQkKKwkJCQl0c29faW5mby0+bHNvX3Yy X3htaXQuaXBfdmVyc2lvbiA9CisJCQkJICAgIFJORElTX1RDUF9MQVJHRV9TRU5EX09GRkxPQURf SVBWNDsKKwkJCQlpcC0+aXBfbGVuID0gMDsKKwkJCQlpcC0+aXBfc3VtID0gMDsKKwkJCQorCQkJ CXRoLT50aF9zdW0gPSBpbl9wc2V1ZG8oaXAtPmlwX3NyYy5zX2FkZHIsCisJCQkJICAgIGlwLT5p cF9kc3Quc19hZGRyLCBodG9ucyhJUFBST1RPX1RDUCkpOworCQkJfQogI2VuZGlmCiAjaWYgZGVm aW5lZChJTkVUNikgJiYgZGVmaW5lZChJTkVUKQotCQllbHNlCisJCQllbHNlCiAjZW5kaWYKICNp ZmRlZiBJTkVUNgotCQl7Ci0JCQlzdHJ1Y3QgaXA2X2hkciAqaXA2ID0KLQkJCSAgICAoc3RydWN0 IGlwNl9oZHIgKikobV9oZWFkLT5tX2RhdGEgKyBldGhlcl9sZW4pOwotCQkJc3RydWN0IHRjcGhk ciAqdGggPSAoc3RydWN0IHRjcGhkciAqKShpcDYgKyAxKTsKLQotCQkJdHNvX2luZm8tPmxzb192 Ml94bWl0LmlwX3ZlcnNpb24gPQotCQkJICAgIFJORElTX1RDUF9MQVJHRV9TRU5EX09GRkxPQURf SVBWNjsKLQkJCWlwNi0+aXA2X3BsZW4gPSAwOwotCQkJdGgtPnRoX3N1bSA9IGluNl9ja3N1bV9w c2V1ZG8oaXA2LCAwLCBJUFBST1RPX1RDUCwgMCk7Ci0JCX0KKwkJCXsKKwkJCQlzdHJ1Y3QgaXA2 X2hkciAqaXA2ID0gKHN0cnVjdCBpcDZfaGRyICopCisJCQkJICAgIChtX2hlYWQtPm1fZGF0YSAr IGV0aGVyX2xlbik7CisJCQkJc3RydWN0IHRjcGhkciAqdGggPSAoc3RydWN0IHRjcGhkciAqKShp cDYgKyAxKTsKKworCQkJCXRzb19pbmZvLT5sc29fdjJfeG1pdC5pcF92ZXJzaW9uID0KKwkJCQkg ICAgUk5ESVNfVENQX0xBUkdFX1NFTkRfT0ZGTE9BRF9JUFY2OworCQkJCWlwNi0+aXA2X3BsZW4g PSAwOworCQkJCXRoLT50aF9zdW0gPQorCQkJCSAgICBpbjZfY2tzdW1fcHNldWRvKGlwNiwgMCwg SVBQUk9UT19UQ1AsIDApOworCQkJfQogI2VuZGlmCi0JCXRzb19pbmZvLT5sc29fdjJfeG1pdC50 Y3BfaGVhZGVyX29mZnNldCA9IDA7Ci0JCXRzb19pbmZvLT5sc29fdjJfeG1pdC5tc3MgPSBtX2hl YWQtPm1fcGt0aGRyLnRzb19zZWdzejsKKwkJCXRzb19pbmZvLT5sc29fdjJfeG1pdC50Y3BfaGVh ZGVyX29mZnNldCA9IDA7CisJCQl0c29faW5mby0+bHNvX3YyX3htaXQubXNzID0gbV9oZWFkLT5t X3BrdGhkci50c29fc2Vnc3o7CisJCX0gZWxzZSBpZiAobV9oZWFkLT5tX3BrdGhkci5jc3VtX2Zs YWdzICYgc2MtPmhuX2NzdW1fYXNzaXN0KSB7CisJCQlybmRpc19tc2dfc2l6ZSArPSBSTkRJU19D U1VNX1BQSV9TSVpFOworCQkJcnBwaSA9IGh2X3NldF9ycHBpX2RhdGEocm5kaXNfbWVzZywgUk5E SVNfQ1NVTV9QUElfU0laRSwKKwkJCSAgICB0Y3BpcF9jaGtzdW1faW5mbyk7CisJCQljc3VtX2lu Zm8gPSAocm5kaXNfdGNwX2lwX2NzdW1faW5mbyAqKSgoY2hhciopcnBwaSArCisJCQkgICAgcnBw aS0+cGVyX3BhY2tldF9pbmZvX29mZnNldCk7CisKKwkJCWNzdW1faW5mby0+eG1pdC5pc19pcHY0 ID0gMTsKKwkJCWlmIChtX2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBDU1VNX1RDUCkgewor CQkJCWNzdW1faW5mby0+eG1pdC50Y3BfY3N1bSA9IDE7CisJCQkJY3N1bV9pbmZvLT54bWl0LnRj cF9oZWFkZXJfb2Zmc2V0ID0gMDsKKwkJCX0gZWxzZSBpZiAobV9oZWFkLT5tX3BrdGhkci5jc3Vt X2ZsYWdzICYgQ1NVTV9VRFApIHsKKwkJCQljc3VtX2luZm8tPnhtaXQudWRwX2NzdW0gPSAxOwor CQkJfQorCQl9CiAKLXByZV9zZW5kOgogCQlybmRpc19tZXNnLT5tc2dfbGVuID0gcGFja2V0LT50 b3RfZGF0YV9idWZfbGVuICsgcm5kaXNfbXNnX3NpemU7CiAJCXBhY2tldC0+dG90X2RhdGFfYnVm X2xlbiA9IHJuZGlzX21lc2ctPm1zZ19sZW47CiAKZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9o eXBlcnYvbmV0dnNjL2h2X25ldF92c2MuaCBiL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2 X25ldF92c2MuaAotLS0gYS9oZWFkL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmgK KysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5oCkBAIC0xMDE1LDYg KzEwMTUsNyBAQAogCWJ1c19kbWFfdGFnX3QJaG5fdHhfcm5kaXNfZHRhZzsKIAlpbnQJCWhuX3R4 X2NoaW1uZXlfc2l6ZTsKIAlpbnQJCWhuX3R4X2NoaW1uZXlfbWF4OworCXVpbnQ2NF90CWhuX2Nz dW1fYXNzaXN0OwogCiAJc3RydWN0IG10eAlobl90eGxpc3Rfc3BpbjsKIAlzdHJ1Y3QgaG5fdHhk ZXNjX2xpc3QgaG5fdHhsaXN0OwpAQCAtMTA0Myw4ICsxMDQ0LDYgQEAKIAl1X2xvbmcJCWhuX3R4 ZG1hX2ZhaWxlZDsKIAl1X2xvbmcJCWhuX3R4X2NvbGxhcHNlZDsKIAl1X2xvbmcJCWhuX3R4X2No aW1uZXk7Ci0KLQl1aW50NjRfdAlobl9jc3VtX2Fzc2lzdDsKIH0gaG5fc29mdGNfdDsKIAogCgo= --b1_cdd90e305a9c697abe3d4ccee544f60b-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:01:18 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DFCE1A9CDEF for ; Fri, 5 Feb 2016 05:01:17 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id C1F4E184B for ; Fri, 5 Feb 2016 05:01:17 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id C0FF410763A; Fri, 5 Feb 2016 05:01:17 +0000 (UTC) Date: Fri, 5 Feb 2016 05:01:17 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5099+325+848a83c1598839c9@reviews.freebsd.org Subject: [Differential] [Closed] D5099: hyperv/hn: Enable IP header checksum offloading Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5099: hyperv/hn: Enable IP header checksum offloading X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: YmJjZTQyNjA4YTZjZjY2YWRlYTUwZDRiZThkIFa0LJ0= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_b0cdd3fe745e279cfefc402b3159ad84" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:01:18 -0000 --b1_b0cdd3fe745e279cfefc402b3159ad84 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295298: hyperv/hn: Enable IP header checksum offloading (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5099?vs=12775&id=13032#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5099?vs=12775&id=13032 REVISION DETAIL https://reviews.freebsd.org/D5099 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -172,7 +172,7 @@ * Windows releases. */ #define HN_CSUM_ASSIST_WIN8 (CSUM_TCP) -#define HN_CSUM_ASSIST (CSUM_UDP | CSUM_TCP) +#define HN_CSUM_ASSIST (CSUM_IP | CSUM_UDP | CSUM_TCP) /* XXX move to netinet/tcp_lro.h */ #define HN_LRO_HIWAT_MAX 65535 @@ -867,6 +867,9 @@ rppi->per_packet_info_offset); csum_info->xmit.is_ipv4 = 1; + if (m_head->m_pkthdr.csum_flags & CSUM_IP) + csum_info->xmit.ip_header_csum = 1; + if (m_head->m_pkthdr.csum_flags & CSUM_TCP) { csum_info->xmit.tcp_csum = 1; csum_info->xmit.tcp_header_offset = 0; EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list --b1_b0cdd3fe745e279cfefc402b3159ad84 Content-Type: text/x-patch; charset=utf-8; name="D5099.13032.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5099.13032.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTE3Miw3ICsxNzIsNyBAQAogICogV2luZG93cyByZWxlYXNlcy4KICAqLwogI2RlZmlu ZSBITl9DU1VNX0FTU0lTVF9XSU44CShDU1VNX1RDUCkKLSNkZWZpbmUgSE5fQ1NVTV9BU1NJU1QJ CShDU1VNX1VEUCB8IENTVU1fVENQKQorI2RlZmluZSBITl9DU1VNX0FTU0lTVAkJKENTVU1fSVAg fCBDU1VNX1VEUCB8IENTVU1fVENQKQogCiAvKiBYWFggbW92ZSB0byBuZXRpbmV0L3RjcF9scm8u aCAqLwogI2RlZmluZSBITl9MUk9fSElXQVRfTUFYCQkJCTY1NTM1CkBAIC04NjcsNiArODY3LDkg QEAKIAkJCSAgICBycHBpLT5wZXJfcGFja2V0X2luZm9fb2Zmc2V0KTsKIAogCQkJY3N1bV9pbmZv LT54bWl0LmlzX2lwdjQgPSAxOworCQkJaWYgKG1faGVhZC0+bV9wa3RoZHIuY3N1bV9mbGFncyAm IENTVU1fSVApCisJCQkJY3N1bV9pbmZvLT54bWl0LmlwX2hlYWRlcl9jc3VtID0gMTsKKwogCQkJ aWYgKG1faGVhZC0+bV9wa3RoZHIuY3N1bV9mbGFncyAmIENTVU1fVENQKSB7CiAJCQkJY3N1bV9p bmZvLT54bWl0LnRjcF9jc3VtID0gMTsKIAkJCQljc3VtX2luZm8tPnhtaXQudGNwX2hlYWRlcl9v ZmZzZXQgPSAwOwoK --b1_b0cdd3fe745e279cfefc402b3159ad84-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:06:39 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48B20A9B037 for ; Fri, 5 Feb 2016 05:06:39 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 282E91BE5 for ; Fri, 5 Feb 2016 05:06:39 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 27D151078D6; Fri, 5 Feb 2016 05:06:39 +0000 (UTC) Date: Fri, 5 Feb 2016 05:06:39 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5102+325+25f4243c1505a8d7@reviews.freebsd.org Subject: [Differential] [Closed] D5102: hyperv/hn: Enable UDP RXCSUM Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5102: hyperv/hn: Enable UDP RXCSUM X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: ZTIzNGRiYzMzNDRhNGZkYzBiMzY2ODVlMDA5IFa0Ld8= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_cba97515a92adb9f0f2369428003e91b" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:06:39 -0000 --b1_cba97515a92adb9f0f2369428003e91b Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295299: hyperv/hn: Enable UDP RXCSUM (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5102?vs=12780&id=13033#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5102?vs=12780&id=13033 REVISION DETAIL https://reviews.freebsd.org/D5102 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_net_vsc.h head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -456,6 +456,8 @@ CTLFLAG_RW, &sc->hn_csum_ip, "RXCSUM IP"); SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "csum_tcp", CTLFLAG_RW, &sc->hn_csum_tcp, "RXCSUM TCP"); + SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "csum_udp", + CTLFLAG_RW, &sc->hn_csum_udp, "RXCSUM UDP"); SYSCTL_ADD_ULONG(ctx, child, OID_AUTO, "csum_trusted", CTLFLAG_RW, &sc->hn_csum_trusted, "# of TCP segements that we trust host's csum verification"); @@ -1156,20 +1158,24 @@ m_new->m_pkthdr.rcvif = ifp; /* receive side checksum offload */ - if (NULL != csum_info) { + if (csum_info != NULL) { /* IP csum offload */ if (csum_info->receive.ip_csum_succeeded) { m_new->m_pkthdr.csum_flags |= (CSUM_IP_CHECKED | CSUM_IP_VALID); sc->hn_csum_ip++; } - /* TCP csum offload */ - if (csum_info->receive.tcp_csum_succeeded) { + /* TCP/UDP csum offload */ + if (csum_info->receive.tcp_csum_succeeded || + csum_info->receive.udp_csum_succeeded) { m_new->m_pkthdr.csum_flags |= (CSUM_DATA_VALID | CSUM_PSEUDO_HDR); m_new->m_pkthdr.csum_data = 0xffff; - sc->hn_csum_tcp++; + if (csum_info->receive.tcp_csum_succeeded) + sc->hn_csum_tcp++; + else + sc->hn_csum_udp++; } if (csum_info->receive.ip_csum_succeeded && diff --git a/head/sys/dev/hyperv/netvsc/hv_net_vsc.h b/head/sys/dev/hyperv/netvsc/hv_net_vsc.h --- a/head/sys/dev/hyperv/netvsc/hv_net_vsc.h +++ b/head/sys/dev/hyperv/netvsc/hv_net_vsc.h @@ -1036,6 +1036,7 @@ u_long hn_csum_ip; u_long hn_csum_tcp; + u_long hn_csum_udp; u_long hn_csum_trusted; u_long hn_lro_tried; u_long hn_small_pkts; EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list --b1_cba97515a92adb9f0f2369428003e91b Content-Type: text/x-patch; charset=utf-8; name="D5102.13033.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5102.13033.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTQ1Niw2ICs0NTYsOCBAQAogCSAgICBDVExGTEFHX1JXLCAmc2MtPmhuX2NzdW1faXAs ICJSWENTVU0gSVAiKTsKIAlTWVNDVExfQUREX1VMT05HKGN0eCwgY2hpbGQsIE9JRF9BVVRPLCAi Y3N1bV90Y3AiLAogCSAgICBDVExGTEFHX1JXLCAmc2MtPmhuX2NzdW1fdGNwLCAiUlhDU1VNIFRD UCIpOworCVNZU0NUTF9BRERfVUxPTkcoY3R4LCBjaGlsZCwgT0lEX0FVVE8sICJjc3VtX3VkcCIs CisJICAgIENUTEZMQUdfUlcsICZzYy0+aG5fY3N1bV91ZHAsICJSWENTVU0gVURQIik7CiAJU1lT Q1RMX0FERF9VTE9ORyhjdHgsIGNoaWxkLCBPSURfQVVUTywgImNzdW1fdHJ1c3RlZCIsCiAJICAg IENUTEZMQUdfUlcsICZzYy0+aG5fY3N1bV90cnVzdGVkLAogCSAgICAiIyBvZiBUQ1Agc2VnZW1l bnRzIHRoYXQgd2UgdHJ1c3QgaG9zdCdzIGNzdW0gdmVyaWZpY2F0aW9uIik7CkBAIC0xMTU2LDIw ICsxMTU4LDI0IEBACiAJbV9uZXctPm1fcGt0aGRyLnJjdmlmID0gaWZwOwogCiAJLyogcmVjZWl2 ZSBzaWRlIGNoZWNrc3VtIG9mZmxvYWQgKi8KLQlpZiAoTlVMTCAhPSBjc3VtX2luZm8pIHsKKwlp ZiAoY3N1bV9pbmZvICE9IE5VTEwpIHsKIAkJLyogSVAgY3N1bSBvZmZsb2FkICovCiAJCWlmIChj c3VtX2luZm8tPnJlY2VpdmUuaXBfY3N1bV9zdWNjZWVkZWQpIHsKIAkJCW1fbmV3LT5tX3BrdGhk ci5jc3VtX2ZsYWdzIHw9CiAJCQkgICAgKENTVU1fSVBfQ0hFQ0tFRCB8IENTVU1fSVBfVkFMSUQp OwogCQkJc2MtPmhuX2NzdW1faXArKzsKIAkJfQogCi0JCS8qIFRDUCBjc3VtIG9mZmxvYWQgKi8K LQkJaWYgKGNzdW1faW5mby0+cmVjZWl2ZS50Y3BfY3N1bV9zdWNjZWVkZWQpIHsKKwkJLyogVENQ L1VEUCBjc3VtIG9mZmxvYWQgKi8KKwkJaWYgKGNzdW1faW5mby0+cmVjZWl2ZS50Y3BfY3N1bV9z dWNjZWVkZWQgfHwKKwkJICAgIGNzdW1faW5mby0+cmVjZWl2ZS51ZHBfY3N1bV9zdWNjZWVkZWQp IHsKIAkJCW1fbmV3LT5tX3BrdGhkci5jc3VtX2ZsYWdzIHw9CiAJCQkgICAgKENTVU1fREFUQV9W QUxJRCB8IENTVU1fUFNFVURPX0hEUik7CiAJCQltX25ldy0+bV9wa3RoZHIuY3N1bV9kYXRhID0g MHhmZmZmOwotCQkJc2MtPmhuX2NzdW1fdGNwKys7CisJCQlpZiAoY3N1bV9pbmZvLT5yZWNlaXZl LnRjcF9jc3VtX3N1Y2NlZWRlZCkKKwkJCQlzYy0+aG5fY3N1bV90Y3ArKzsKKwkJCWVsc2UKKwkJ CQlzYy0+aG5fY3N1bV91ZHArKzsKIAkJfQogCiAJCWlmIChjc3VtX2luZm8tPnJlY2VpdmUuaXBf Y3N1bV9zdWNjZWVkZWQgJiYKZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNj L2h2X25ldF92c2MuaCBiL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuaAot LS0gYS9oZWFkL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmgKKysrIGIvaGVhZC9z eXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5oCkBAIC0xMDM2LDYgKzEwMzYsNyBAQAog CiAJdV9sb25nCQlobl9jc3VtX2lwOwogCXVfbG9uZwkJaG5fY3N1bV90Y3A7CisJdV9sb25nCQlo bl9jc3VtX3VkcDsKIAl1X2xvbmcJCWhuX2NzdW1fdHJ1c3RlZDsKIAl1X2xvbmcJCWhuX2xyb190 cmllZDsKIAl1X2xvbmcJCWhuX3NtYWxsX3BrdHM7Cgo= --b1_cba97515a92adb9f0f2369428003e91b-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:13:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B39FBA9B381 for ; Fri, 5 Feb 2016 05:13:04 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 9E2B6DB for ; Fri, 5 Feb 2016 05:13:04 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 99811107CB0; Fri, 5 Feb 2016 05:13:04 +0000 (UTC) Date: Fri, 5 Feb 2016 05:13:04 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5103+325+9f7cd214eb126a32@reviews.freebsd.org Subject: [Differential] [Closed] D5103: hyperv/hn: Add sysctl to trust host side UDP and IP csum verification Message-ID: <409a684ed8fe599a6694d71fe34b36f0@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5103: hyperv/hn: Add sysctl to trust host side UDP and IP csum verification X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: YTc1N2JhYmJkMzBkNmVlOWEyYjZiYzZjY2FjIFa0L2A= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_409a684ed8fe599a6694d71fe34b36f0" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:13:04 -0000 --b1_409a684ed8fe599a6694d71fe34b36f0 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295300: hyperv/hn: Add sysctls to trust host side UDP and IP csum verification (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5103?vs=12782&id=13034#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5103?vs=12782&id=13034 REVISION DETAIL https://reviews.freebsd.org/D5103 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_net_vsc.h head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, howard0su_gmail.com, honzhan_microsoft.com, adrian, network Cc: freebsd-net-list --b1_409a684ed8fe599a6694d71fe34b36f0 Content-Type: text/x-patch; charset=utf-8; name="D5103.13034.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5103.13034.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTIxMCw2ICsyMTAsMTQgQEAKIHN0YXRpYyBpbnQgaG5fdHJ1c3RfaG9zdHRjcCA9IDE7 CiBUVU5BQkxFX0lOVCgiZGV2LmhuLnRydXN0X2hvc3R0Y3AiLCAmaG5fdHJ1c3RfaG9zdHRjcCk7 CiAKKy8qIFRydXN0IHVkcCBkYXRhZ3JhbXMgdmVyaWZpY2F0aW9uIG9uIGhvc3Qgc2lkZS4gKi8K K3N0YXRpYyBpbnQgaG5fdHJ1c3RfaG9zdHVkcCA9IDE7CitUVU5BQkxFX0lOVCgiZGV2LmhuLnRy dXN0X2hvc3R1ZHAiLCAmaG5fdHJ1c3RfaG9zdHVkcCk7CisKKy8qIFRydXN0IGlwIHBhY2tldHMg dmVyaWZpY2F0aW9uIG9uIGhvc3Qgc2lkZS4gKi8KK3N0YXRpYyBpbnQgaG5fdHJ1c3RfaG9zdGlw ID0gMTsKK1RVTkFCTEVfSU5UKCJkZXYuaG4udHJ1c3RfaG9zdGlwIiwgJmhuX3RydXN0X2hvc3Rp cCk7CisKICNpZiBfX0ZyZWVCU0RfdmVyc2lvbiA+PSAxMTAwMDQ1CiAvKiBMaW1pdCBUU08gYnVy c3Qgc2l6ZSAqLwogc3RhdGljIGludCBobl90c29fbWF4bGVuID0gMDsKQEAgLTIzOSw2ICsyNDcs NyBAQAogI2lmZGVmIEhOX0xST19ISVdBVAogc3RhdGljIGludCBobl9scm9faGl3YXRfc3lzY3Rs KFNZU0NUTF9IQU5ETEVSX0FSR1MpOwogI2VuZGlmCitzdGF0aWMgaW50IGhuX3RydXN0X2hjc3Vt X3N5c2N0bChTWVNDVExfSEFORExFUl9BUkdTKTsKIHN0YXRpYyBpbnQgaG5fdHhfY2hpbW5leV9z aXplX3N5c2N0bChTWVNDVExfSEFORExFUl9BUkdTKTsKIHN0YXRpYyBpbnQgaG5fY2hlY2tfaXBs ZW4oY29uc3Qgc3RydWN0IG1idWYgKiwgaW50KTsKIHN0YXRpYyBpbnQgaG5fY3JlYXRlX3R4X3Jp bmcoc3RydWN0IGhuX3NvZnRjICpzYyk7CkBAIC0zMzUsOCArMzQ0LDEzIEBACiAJc2MtPmhuX3Vu aXQgPSB1bml0OwogCXNjLT5obl9kZXYgPSBkZXY7CiAJc2MtPmhuX2xyb19oaXdhdCA9IEhOX0xS T19ISVdBVF9ERUY7Ci0Jc2MtPmhuX3RydXN0X2hvc3R0Y3AgPSBobl90cnVzdF9ob3N0dGNwOwog CXNjLT5obl9kaXJlY3RfdHhfc2l6ZSA9IGhuX2RpcmVjdF90eF9zaXplOworCWlmIChobl90cnVz dF9ob3N0dGNwKQorCQlzYy0+aG5fdHJ1c3RfaGNzdW0gfD0gSE5fVFJVU1RfSENTVU1fVENQOwor CWlmIChobl90cnVzdF9ob3N0dWRwKQorCQlzYy0+aG5fdHJ1c3RfaGNzdW0gfD0gSE5fVFJVU1Rf SENTVU1fVURQOworCWlmIChobl90cnVzdF9ob3N0aXApCisJCXNjLT5obl90cnVzdF9oY3N1bSB8 PSBITl9UUlVTVF9IQ1NVTV9JUDsKIAogCXNjLT5obl90eF90YXNrcSA9IHRhc2txdWV1ZV9jcmVh dGVfZmFzdCgiaG5fdHgiLCBNX1dBSVRPSywKIAkgICAgdGFza3F1ZXVlX3RocmVhZF9lbnF1ZXVl LCAmc2MtPmhuX3R4X3Rhc2txKTsKQEAgLTQ0OCwxOSArNDYyLDMwIEBACiAJICAgIENUTFRZUEVf SU5UIHwgQ1RMRkxBR19SVywgc2MsIDAsIGhuX2xyb19oaXdhdF9zeXNjdGwsCiAJICAgICJJIiwg IkxSTyBoaWdoIHdhdGVybWFyayIpOwogI2VuZGlmCi0JU1lTQ1RMX0FERF9JTlQoY3R4LCBjaGls ZCwgT0lEX0FVVE8sICJ0cnVzdF9ob3N0dGNwIiwKLQkgICAgQ1RMRkxBR19SVywgJnNjLT5obl90 cnVzdF9ob3N0dGNwLCAwLAorCVNZU0NUTF9BRERfUFJPQyhjdHgsIGNoaWxkLCBPSURfQVVUTywg InRydXN0X2hvc3R0Y3AiLAorCSAgICBDVExUWVBFX0lOVCB8IENUTEZMQUdfUlcsIHNjLCBITl9U UlVTVF9IQ1NVTV9UQ1AsCisJICAgIGhuX3RydXN0X2hjc3VtX3N5c2N0bCwgIkkiLAogCSAgICAi VHJ1c3QgdGNwIHNlZ2VtZW50IHZlcmlmaWNhdGlvbiBvbiBob3N0IHNpZGUsICIKIAkgICAgIndo ZW4gY3N1bSBpbmZvIGlzIG1pc3NpbmciKTsKKwlTWVNDVExfQUREX1BST0MoY3R4LCBjaGlsZCwg T0lEX0FVVE8sICJ0cnVzdF9ob3N0dWRwIiwKKwkgICAgQ1RMVFlQRV9JTlQgfCBDVExGTEFHX1JX LCBzYywgSE5fVFJVU1RfSENTVU1fVURQLAorCSAgICBobl90cnVzdF9oY3N1bV9zeXNjdGwsICJJ IiwKKwkgICAgIlRydXN0IHVkcCBkYXRhZ3JhbSB2ZXJpZmljYXRpb24gb24gaG9zdCBzaWRlLCAi CisJICAgICJ3aGVuIGNzdW0gaW5mbyBpcyBtaXNzaW5nIik7CisJU1lTQ1RMX0FERF9QUk9DKGN0 eCwgY2hpbGQsIE9JRF9BVVRPLCAidHJ1c3RfaG9zdGlwIiwKKwkgICAgQ1RMVFlQRV9JTlQgfCBD VExGTEFHX1JXLCBzYywgSE5fVFJVU1RfSENTVU1fSVAsCisJICAgIGhuX3RydXN0X2hjc3VtX3N5 c2N0bCwgIkkiLAorCSAgICAiVHJ1c3QgaXAgcGFja2V0IHZlcmlmaWNhdGlvbiBvbiBob3N0IHNp ZGUsICIKKwkgICAgIndoZW4gY3N1bSBpbmZvIGlzIG1pc3NpbmciKTsKIAlTWVNDVExfQUREX1VM T05HKGN0eCwgY2hpbGQsIE9JRF9BVVRPLCAiY3N1bV9pcCIsCiAJICAgIENUTEZMQUdfUlcsICZz Yy0+aG5fY3N1bV9pcCwgIlJYQ1NVTSBJUCIpOwogCVNZU0NUTF9BRERfVUxPTkcoY3R4LCBjaGls ZCwgT0lEX0FVVE8sICJjc3VtX3RjcCIsCiAJICAgIENUTEZMQUdfUlcsICZzYy0+aG5fY3N1bV90 Y3AsICJSWENTVU0gVENQIik7CiAJU1lTQ1RMX0FERF9VTE9ORyhjdHgsIGNoaWxkLCBPSURfQVVU TywgImNzdW1fdWRwIiwKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9jc3VtX3VkcCwgIlJYQ1NV TSBVRFAiKTsKIAlTWVNDVExfQUREX1VMT05HKGN0eCwgY2hpbGQsIE9JRF9BVVRPLCAiY3N1bV90 cnVzdGVkIiwKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9jc3VtX3RydXN0ZWQsCi0JICAgICIj IG9mIFRDUCBzZWdlbWVudHMgdGhhdCB3ZSB0cnVzdCBob3N0J3MgY3N1bSB2ZXJpZmljYXRpb24i KTsKKwkgICAgIiMgb2YgcGFja2V0cyB0aGF0IHdlIHRydXN0IGhvc3QncyBjc3VtIHZlcmlmaWNh dGlvbiIpOwogCVNZU0NUTF9BRERfVUxPTkcoY3R4LCBjaGlsZCwgT0lEX0FVVE8sICJzbWFsbF9w a3RzIiwKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9zbWFsbF9wa3RzLCAiIyBvZiBzbWFsbCBw YWNrZXRzIHJlY2VpdmVkIik7CiAJU1lTQ1RMX0FERF9VTE9ORyhjdHgsIGNoaWxkLCBPSURfQVVU TywgIm5vX3R4ZGVzY3MiLApAQCAtNTAzLDYgKzUyOCwxNCBAQAogCQkgICAgQ1RMRkxBR19SRCwg JmhuX3RydXN0X2hvc3R0Y3AsIDAsCiAJCSAgICAiVHJ1c3QgdGNwIHNlZ2VtZW50IHZlcmlmaWNh dGlvbiBvbiBob3N0IHNpZGUsICIKIAkJICAgICJ3aGVuIGNzdW0gaW5mbyBpcyBtaXNzaW5nIChn bG9iYWwgc2V0dGluZykiKTsKKwkJU1lTQ1RMX0FERF9JTlQoZGNfY3R4LCBkY19jaGlsZCwgT0lE X0FVVE8sICJ0cnVzdF9ob3N0dWRwIiwKKwkJICAgIENUTEZMQUdfUkQsICZobl90cnVzdF9ob3N0 dWRwLCAwLAorCQkgICAgIlRydXN0IHVkcCBkYXRhZ3JhbSB2ZXJpZmljYXRpb24gb24gaG9zdCBz aWRlLCAiCisJCSAgICAid2hlbiBjc3VtIGluZm8gaXMgbWlzc2luZyAoZ2xvYmFsIHNldHRpbmcp Iik7CisJCVNZU0NUTF9BRERfSU5UKGRjX2N0eCwgZGNfY2hpbGQsIE9JRF9BVVRPLCAidHJ1c3Rf aG9zdGlwIiwKKwkJICAgIENUTEZMQUdfUkQsICZobl90cnVzdF9ob3N0aXAsIDAsCisJCSAgICAi VHJ1c3QgaXAgcGFja2V0IHZlcmlmaWNhdGlvbiBvbiBob3N0IHNpZGUsICIKKwkJICAgICJ3aGVu IGNzdW0gaW5mbyBpcyBtaXNzaW5nIChnbG9iYWwgc2V0dGluZykiKTsKIAkJU1lTQ1RMX0FERF9J TlQoZGNfY3R4LCBkY19jaGlsZCwgT0lEX0FVVE8sICJ0eF9jaGltbmV5X3NpemUiLAogCQkgICAg Q1RMRkxBR19SRCwgJmhuX3R4X2NoaW1uZXlfc2l6ZSwgMCwKIAkJICAgICJDaGltbmV5IHNlbmQg cGFja2V0IHNpemUgbGltaXQiKTsKQEAgLTEyMDYsMTUgKzEyMzksMjggQEAKIAogCQkJcHIgPSBo bl9jaGVja19pcGxlbihtX25ldywgaG9mZik7CiAJCQlpZiAocHIgPT0gSVBQUk9UT19UQ1ApIHsK LQkJCQlpZiAoc2MtPmhuX3RydXN0X2hvc3R0Y3ApIHsKKwkJCQlpZiAoc2MtPmhuX3RydXN0X2hj c3VtICYgSE5fVFJVU1RfSENTVU1fVENQKSB7CiAJCQkJCXNjLT5obl9jc3VtX3RydXN0ZWQrKzsK IAkJCQkJbV9uZXctPm1fcGt0aGRyLmNzdW1fZmxhZ3MgfD0KIAkJCQkJICAgKENTVU1fSVBfQ0hF Q0tFRCB8IENTVU1fSVBfVkFMSUQgfAogCQkJCQkgICAgQ1NVTV9EQVRBX1ZBTElEIHwgQ1NVTV9Q U0VVRE9fSERSKTsKIAkJCQkJbV9uZXctPm1fcGt0aGRyLmNzdW1fZGF0YSA9IDB4ZmZmZjsKIAkJ CQl9CiAJCQkJLyogUmVseSBvbiBTVyBjc3VtIHZlcmlmaWNhdGlvbiB0aG91Z2guLi4gKi8KIAkJ CQlkb19scm8gPSAxOworCQkJfSBlbHNlIGlmIChwciA9PSBJUFBST1RPX1VEUCkgeworCQkJCWlm IChzYy0+aG5fdHJ1c3RfaGNzdW0gJiBITl9UUlVTVF9IQ1NVTV9VRFApIHsKKwkJCQkJc2MtPmhu X2NzdW1fdHJ1c3RlZCsrOworCQkJCQltX25ldy0+bV9wa3RoZHIuY3N1bV9mbGFncyB8PQorCQkJ CQkgICAoQ1NVTV9JUF9DSEVDS0VEIHwgQ1NVTV9JUF9WQUxJRCB8CisJCQkJCSAgICBDU1VNX0RB VEFfVkFMSUQgfCBDU1VNX1BTRVVET19IRFIpOworCQkJCQltX25ldy0+bV9wa3RoZHIuY3N1bV9k YXRhID0gMHhmZmZmOworCQkJCX0KKwkJCX0gZWxzZSBpZiAocHIgIT0gSVBQUk9UT19ET05FICYm CisJCQkgICAgKHNjLT5obl90cnVzdF9oY3N1bSAmIEhOX1RSVVNUX0hDU1VNX0lQKSkgeworCQkJ CXNjLT5obl9jc3VtX3RydXN0ZWQrKzsKKwkJCQltX25ldy0+bV9wa3RoZHIuY3N1bV9mbGFncyB8 PQorCQkJCSAgICAoQ1NVTV9JUF9DSEVDS0VEIHwgQ1NVTV9JUF9WQUxJRCk7CiAJCQl9CiAJCX0K IAl9CkBAIC0xNjUwLDYgKzE2OTYsMzAgQEAKICNlbmRpZgkvKiBITl9MUk9fSElXQVQgKi8KIAog c3RhdGljIGludAoraG5fdHJ1c3RfaGNzdW1fc3lzY3RsKFNZU0NUTF9IQU5ETEVSX0FSR1MpCit7 CisJc3RydWN0IGhuX3NvZnRjICpzYyA9IGFyZzE7CisJaW50IGhjc3VtID0gYXJnMjsKKwlpbnQg b24sIGVycm9yOworCisJb24gPSAwOworCWlmIChzYy0+aG5fdHJ1c3RfaGNzdW0gJiBoY3N1bSkK KwkJb24gPSAxOworCisJZXJyb3IgPSBzeXNjdGxfaGFuZGxlX2ludChvaWRwLCAmb24sIDAsIHJl cSk7CisJaWYgKGVycm9yIHx8IHJlcS0+bmV3cHRyID09IE5VTEwpCisJCXJldHVybiBlcnJvcjsK KworCU5WX0xPQ0soc2MpOworCWlmIChvbikKKwkJc2MtPmhuX3RydXN0X2hjc3VtIHw9IGhjc3Vt OworCWVsc2UKKwkJc2MtPmhuX3RydXN0X2hjc3VtICY9IH5oY3N1bTsKKwlOVl9VTkxPQ0soc2Mp OworCXJldHVybiAwOworfQorCitzdGF0aWMgaW50CiBobl90eF9jaGltbmV5X3NpemVfc3lzY3Rs KFNZU0NUTF9IQU5ETEVSX0FSR1MpCiB7CiAJc3RydWN0IGhuX3NvZnRjICpzYyA9IGFyZzE7CmRp ZmYgLS1naXQgYS9oZWFkL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmggYi9oZWFk L3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmgKLS0tIGEvaGVhZC9zeXMvZGV2L2h5 cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5oCisrKyBiL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNj L2h2X25ldF92c2MuaApAQCAtMTAzMSw4ICsxMDMxLDggQEAKIAlzdHJ1Y3QgbHJvX2N0cmwJaG5f bHJvOwogCWludAkJaG5fbHJvX2hpd2F0OwogCi0JLyogVHJ1c3QgdGNwIHNlZ21lbnRzIHZlcmlm aWNhdGlvbiBvbiBob3N0IHNpZGUgKi8KLQlpbnQJCWhuX3RydXN0X2hvc3R0Y3A7CisJLyogVHJ1 c3QgY3N1bSB2ZXJpZmljYXRpb24gb24gaG9zdCBzaWRlICovCisJaW50CQlobl90cnVzdF9oY3N1 bTsJLyogSE5fVFJVU1RfSENTVU1fICovCiAKIAl1X2xvbmcJCWhuX2NzdW1faXA7CiAJdV9sb25n CQlobl9jc3VtX3RjcDsKQEAgLTEwNDcsNiArMTA0Nyw5IEBACiAJdV9sb25nCQlobl90eF9jaGlt bmV5OwogfSBobl9zb2Z0Y190OwogCisjZGVmaW5lIEhOX1RSVVNUX0hDU1VNX0lQCTB4MDAwMQor I2RlZmluZSBITl9UUlVTVF9IQ1NVTV9UQ1AJMHgwMDAyCisjZGVmaW5lIEhOX1RSVVNUX0hDU1VN X1VEUAkweDAwMDQKIAogLyoKICAqIEV4dGVybnMKCg== --b1_409a684ed8fe599a6694d71fe34b36f0-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:18:21 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3272A9B540 for ; Fri, 5 Feb 2016 05:18:21 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id B1CEC61B for ; Fri, 5 Feb 2016 05:18:21 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id B11971070B5; Fri, 5 Feb 2016 05:18:21 +0000 (UTC) Date: Fri, 5 Feb 2016 05:18:21 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5104+325+bc513568574341fc@reviews.freebsd.org Subject: [Differential] [Closed] D5104: hyperv/hn: Obey IFCAP_RXCSUM Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5104: hyperv/hn: Obey IFCAP_RXCSUM X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: MTc0NTllYjE2MmY5YWYyODFiZDZkYjcyYmUwIFa0MJ0= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_c096b50a40499c6e120ba9aa3d7955b4" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:18:21 -0000 --b1_c096b50a40499c6e120ba9aa3d7955b4 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295301: hyperv/hn: Obey IFCAP_RXCSUM configure (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5104?vs=12783&id=13035#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5104?vs=12783&id=13035 REVISION DETAIL https://reviews.freebsd.org/D5104 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -1142,7 +1142,7 @@ struct mbuf *m_new; struct ifnet *ifp; device_t dev = device_ctx->device; - int size, do_lro = 0; + int size, do_lro = 0, do_csum = 1; if (sc == NULL) { return (0); /* TODO: KYS how can this be! */ @@ -1190,18 +1190,21 @@ } m_new->m_pkthdr.rcvif = ifp; + if (__predict_false((ifp->if_capenable & IFCAP_RXCSUM) == 0)) + do_csum = 0; + /* receive side checksum offload */ if (csum_info != NULL) { /* IP csum offload */ - if (csum_info->receive.ip_csum_succeeded) { + if (csum_info->receive.ip_csum_succeeded && do_csum) { m_new->m_pkthdr.csum_flags |= (CSUM_IP_CHECKED | CSUM_IP_VALID); sc->hn_csum_ip++; } /* TCP/UDP csum offload */ - if (csum_info->receive.tcp_csum_succeeded || - csum_info->receive.udp_csum_succeeded) { + if ((csum_info->receive.tcp_csum_succeeded || + csum_info->receive.udp_csum_succeeded) && do_csum) { m_new->m_pkthdr.csum_flags |= (CSUM_DATA_VALID | CSUM_PSEUDO_HDR); m_new->m_pkthdr.csum_data = 0xffff; @@ -1239,7 +1242,8 @@ pr = hn_check_iplen(m_new, hoff); if (pr == IPPROTO_TCP) { - if (sc->hn_trust_hcsum & HN_TRUST_HCSUM_TCP) { + if (do_csum && + (sc->hn_trust_hcsum & HN_TRUST_HCSUM_TCP)) { sc->hn_csum_trusted++; m_new->m_pkthdr.csum_flags |= (CSUM_IP_CHECKED | CSUM_IP_VALID | @@ -1249,14 +1253,15 @@ /* Rely on SW csum verification though... */ do_lro = 1; } else if (pr == IPPROTO_UDP) { - if (sc->hn_trust_hcsum & HN_TRUST_HCSUM_UDP) { + if (do_csum && + (sc->hn_trust_hcsum & HN_TRUST_HCSUM_UDP)) { sc->hn_csum_trusted++; m_new->m_pkthdr.csum_flags |= (CSUM_IP_CHECKED | CSUM_IP_VALID | CSUM_DATA_VALID | CSUM_PSEUDO_HDR); m_new->m_pkthdr.csum_data = 0xffff; } - } else if (pr != IPPROTO_DONE && + } else if (pr != IPPROTO_DONE && do_csum && (sc->hn_trust_hcsum & HN_TRUST_HCSUM_IP)) { sc->hn_csum_trusted++; m_new->m_pkthdr.csum_flags |= EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list --b1_c096b50a40499c6e120ba9aa3d7955b4 Content-Type: text/x-patch; charset=utf-8; name="D5104.13035.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5104.13035.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTExNDIsNyArMTE0Miw3IEBACiAJc3RydWN0IG1idWYgKm1fbmV3OwogCXN0cnVjdCBp Zm5ldCAqaWZwOwogCWRldmljZV90IGRldiA9IGRldmljZV9jdHgtPmRldmljZTsKLQlpbnQgc2l6 ZSwgZG9fbHJvID0gMDsKKwlpbnQgc2l6ZSwgZG9fbHJvID0gMCwgZG9fY3N1bSA9IDE7CiAKIAlp ZiAoc2MgPT0gTlVMTCkgewogCQlyZXR1cm4gKDApOyAvKiBUT0RPOiBLWVMgaG93IGNhbiB0aGlz IGJlISAqLwpAQCAtMTE5MCwxOCArMTE5MCwyMSBAQAogCX0KIAltX25ldy0+bV9wa3RoZHIucmN2 aWYgPSBpZnA7CiAKKwlpZiAoX19wcmVkaWN0X2ZhbHNlKChpZnAtPmlmX2NhcGVuYWJsZSAmIElG Q0FQX1JYQ1NVTSkgPT0gMCkpCisJCWRvX2NzdW0gPSAwOworCiAJLyogcmVjZWl2ZSBzaWRlIGNo ZWNrc3VtIG9mZmxvYWQgKi8KIAlpZiAoY3N1bV9pbmZvICE9IE5VTEwpIHsKIAkJLyogSVAgY3N1 bSBvZmZsb2FkICovCi0JCWlmIChjc3VtX2luZm8tPnJlY2VpdmUuaXBfY3N1bV9zdWNjZWVkZWQp IHsKKwkJaWYgKGNzdW1faW5mby0+cmVjZWl2ZS5pcF9jc3VtX3N1Y2NlZWRlZCAmJiBkb19jc3Vt KSB7CiAJCQltX25ldy0+bV9wa3RoZHIuY3N1bV9mbGFncyB8PQogCQkJICAgIChDU1VNX0lQX0NI RUNLRUQgfCBDU1VNX0lQX1ZBTElEKTsKIAkJCXNjLT5obl9jc3VtX2lwKys7CiAJCX0KIAogCQkv KiBUQ1AvVURQIGNzdW0gb2ZmbG9hZCAqLwotCQlpZiAoY3N1bV9pbmZvLT5yZWNlaXZlLnRjcF9j c3VtX3N1Y2NlZWRlZCB8fAotCQkgICAgY3N1bV9pbmZvLT5yZWNlaXZlLnVkcF9jc3VtX3N1Y2Nl ZWRlZCkgeworCQlpZiAoKGNzdW1faW5mby0+cmVjZWl2ZS50Y3BfY3N1bV9zdWNjZWVkZWQgfHwK KwkJICAgICBjc3VtX2luZm8tPnJlY2VpdmUudWRwX2NzdW1fc3VjY2VlZGVkKSAmJiBkb19jc3Vt KSB7CiAJCQltX25ldy0+bV9wa3RoZHIuY3N1bV9mbGFncyB8PQogCQkJICAgIChDU1VNX0RBVEFf VkFMSUQgfCBDU1VNX1BTRVVET19IRFIpOwogCQkJbV9uZXctPm1fcGt0aGRyLmNzdW1fZGF0YSA9 IDB4ZmZmZjsKQEAgLTEyMzksNyArMTI0Miw4IEBACiAKIAkJCXByID0gaG5fY2hlY2tfaXBsZW4o bV9uZXcsIGhvZmYpOwogCQkJaWYgKHByID09IElQUFJPVE9fVENQKSB7Ci0JCQkJaWYgKHNjLT5o bl90cnVzdF9oY3N1bSAmIEhOX1RSVVNUX0hDU1VNX1RDUCkgeworCQkJCWlmIChkb19jc3VtICYm CisJCQkJICAgIChzYy0+aG5fdHJ1c3RfaGNzdW0gJiBITl9UUlVTVF9IQ1NVTV9UQ1ApKSB7CiAJ CQkJCXNjLT5obl9jc3VtX3RydXN0ZWQrKzsKIAkJCQkJbV9uZXctPm1fcGt0aGRyLmNzdW1fZmxh Z3MgfD0KIAkJCQkJICAgKENTVU1fSVBfQ0hFQ0tFRCB8IENTVU1fSVBfVkFMSUQgfApAQCAtMTI0 OSwxNCArMTI1MywxNSBAQAogCQkJCS8qIFJlbHkgb24gU1cgY3N1bSB2ZXJpZmljYXRpb24gdGhv dWdoLi4uICovCiAJCQkJZG9fbHJvID0gMTsKIAkJCX0gZWxzZSBpZiAocHIgPT0gSVBQUk9UT19V RFApIHsKLQkJCQlpZiAoc2MtPmhuX3RydXN0X2hjc3VtICYgSE5fVFJVU1RfSENTVU1fVURQKSB7 CisJCQkJaWYgKGRvX2NzdW0gJiYKKwkJCQkgICAgKHNjLT5obl90cnVzdF9oY3N1bSAmIEhOX1RS VVNUX0hDU1VNX1VEUCkpIHsKIAkJCQkJc2MtPmhuX2NzdW1fdHJ1c3RlZCsrOwogCQkJCQltX25l dy0+bV9wa3RoZHIuY3N1bV9mbGFncyB8PQogCQkJCQkgICAoQ1NVTV9JUF9DSEVDS0VEIHwgQ1NV TV9JUF9WQUxJRCB8CiAJCQkJCSAgICBDU1VNX0RBVEFfVkFMSUQgfCBDU1VNX1BTRVVET19IRFIp OwogCQkJCQltX25ldy0+bV9wa3RoZHIuY3N1bV9kYXRhID0gMHhmZmZmOwogCQkJCX0KLQkJCX0g ZWxzZSBpZiAocHIgIT0gSVBQUk9UT19ET05FICYmCisJCQl9IGVsc2UgaWYgKHByICE9IElQUFJP VE9fRE9ORSAmJiBkb19jc3VtICYmCiAJCQkgICAgKHNjLT5obl90cnVzdF9oY3N1bSAmIEhOX1RS VVNUX0hDU1VNX0lQKSkgewogCQkJCXNjLT5obl9jc3VtX3RydXN0ZWQrKzsKIAkJCQltX25ldy0+ bV9wa3RoZHIuY3N1bV9mbGFncyB8PQoK --b1_c096b50a40499c6e120ba9aa3d7955b4-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:25:33 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF019A9B934 for ; Fri, 5 Feb 2016 05:25:33 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id C9F91B53 for ; Fri, 5 Feb 2016 05:25:33 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id C92D11074AD; Fri, 5 Feb 2016 05:25:33 +0000 (UTC) Date: Fri, 5 Feb 2016 05:25:33 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5158+325+84c30785c1b48e01@reviews.freebsd.org Subject: [Differential] [Closed] D5158: hyperv/hn: Factor out hn_encap from hn_start_locked() Message-ID: <7d83666eeeadbf8cd33a405b6c1d7cae@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5158: hyperv/hn: Factor out hn_encap from hn_start_locked() X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: MjFmOWM1NjYwZmMxOTMyMjJlM2E1YTQ1MjllIFa0Mk0= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_7d83666eeeadbf8cd33a405b6c1d7cae" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:25:34 -0000 --b1_7d83666eeeadbf8cd33a405b6c1d7cae Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295302: hyperv/hn: Factor out hn_encap() from hn_start_locked() (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5158?vs=12925&id=13036#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5158?vs=12925&id=13036 REVISION DETAIL https://reviews.freebsd.org/D5158 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list --b1_7d83666eeeadbf8cd33a405b6c1d7cae Content-Type: text/x-patch; charset=utf-8; name="D5158.13036.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5158.13036.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTI1NCw2ICsyNTQsNyBAQAogc3RhdGljIHZvaWQgaG5fZGVzdHJveV90eF9yaW5nKHN0 cnVjdCBobl9zb2Z0YyAqc2MpOwogc3RhdGljIHZvaWQgaG5fc3RhcnRfdGFza2Z1bmModm9pZCAq eHNjLCBpbnQgcGVuZGluZyk7CiBzdGF0aWMgdm9pZCBobl90eGVvZl90YXNrZnVuYyh2b2lkICp4 c2MsIGludCBwZW5kaW5nKTsKK3N0YXRpYyBpbnQgaG5fZW5jYXAoc3RydWN0IGhuX3NvZnRjICos IHN0cnVjdCBobl90eGRlc2MgKiwgc3RydWN0IG1idWYgKiopOwogCiBzdGF0aWMgX19pbmxpbmUg dm9pZAogaG5fc2V0X2xyb19oaXdhdChzdHJ1Y3QgaG5fc29mdGMgKnNjLCBpbnQgaGl3YXQpCkBA IC03NDQsMzEgKzc0NSwyMzUgQEAKIH0KIAogLyoKLSAqIFN0YXJ0IGEgdHJhbnNtaXQgb2Ygb25l IG9yIG1vcmUgcGFja2V0cworICogTk9URToKKyAqIFRoaXMgdGhpcyBmdW5jdGlvbiBmYWlscywg dGhlbiBib3RoIHR4ZCBhbmQgbV9oZWFkMCB3aWxsIGJlIGZyZWVkCiAgKi8KIHN0YXRpYyBpbnQK LWhuX3N0YXJ0X2xvY2tlZChzdHJ1Y3QgaWZuZXQgKmlmcCwgaW50IGxlbikKK2huX2VuY2FwKHN0 cnVjdCBobl9zb2Z0YyAqc2MsIHN0cnVjdCBobl90eGRlc2MgKnR4ZCwgc3RydWN0IG1idWYgKipt X2hlYWQwKQogewotCWhuX3NvZnRjX3QgKnNjID0gaWZwLT5pZl9zb2Z0YzsKLQlzdHJ1Y3QgaHZf ZGV2aWNlICpkZXZpY2VfY3R4ID0gdm1idXNfZ2V0X2RldmN0eChzYy0+aG5fZGV2KTsKLQluZXR2 c2NfZGV2ICpuZXRfZGV2ID0gc2MtPm5ldF9kZXY7CisJYnVzX2RtYV9zZWdtZW50X3Qgc2Vnc1tI Tl9UWF9EQVRBX1NFR0NOVF9NQVhdOworCWludCBlcnJvciwgbnNlZ3MsIGk7CisJc3RydWN0IG1i dWYgKm1faGVhZCA9ICptX2hlYWQwOworCW5ldHZzY19wYWNrZXQgKnBhY2tldDsKIAlybmRpc19t c2cgKnJuZGlzX21lc2c7CiAJcm5kaXNfcGFja2V0ICpybmRpc19wa3Q7CiAJcm5kaXNfcGVyX3Bh Y2tldF9pbmZvICpycHBpOwotCW5kaXNfODAyMXFfaW5mbyAqcnBwaV92bGFuX2luZm87Ci0Jcm5k aXNfdGNwX2lwX2NzdW1faW5mbyAqY3N1bV9pbmZvOwotCXJuZGlzX3RjcF90c29faW5mbyAqdHNv X2luZm87CQotCXVpbnQzMl90IHJuZGlzX21zZ19zaXplID0gMDsKKwl1aW50MzJfdCBybmRpc19t c2dfc2l6ZTsKKworCXBhY2tldCA9ICZ0eGQtPm5ldHZzY19wa3Q7CisJcGFja2V0LT5pc19kYXRh X3BrdCA9IFRSVUU7CisJcGFja2V0LT50b3RfZGF0YV9idWZfbGVuID0gbV9oZWFkLT5tX3BrdGhk ci5sZW47CisKKwkvKgorCSAqIGV4dGVuc2lvbiBwb2ludHMgdG8gdGhlIGFyZWEgcmVzZXJ2ZWQg Zm9yIHRoZQorCSAqIHJuZGlzX2ZpbHRlcl9wYWNrZXQsIHdoaWNoIGlzIHBsYWNlZCBqdXN0IGFm dGVyCisJICogdGhlIG5ldHZzY19wYWNrZXQgKGFuZCBycHBpIHN0cnVjdCwgaWYgcHJlc2VudDsK KwkgKiBsZW5ndGggaXMgdXBkYXRlZCBsYXRlcikuCisJICovCisJcm5kaXNfbWVzZyA9IHR4ZC0+ cm5kaXNfbXNnOworCS8qIFhYWCBub3QgbmVjZXNzYXJ5ICovCisJbWVtc2V0KHJuZGlzX21lc2cs IDAsIEhOX1JORElTX01TR19MRU4pOworCXJuZGlzX21lc2ctPm5kaXNfbXNnX3R5cGUgPSBSRU1P VEVfTkRJU19QQUNLRVRfTVNHOworCisJcm5kaXNfcGt0ID0gJnJuZGlzX21lc2ctPm1zZy5wYWNr ZXQ7CisJcm5kaXNfcGt0LT5kYXRhX29mZnNldCA9IHNpemVvZihybmRpc19wYWNrZXQpOworCXJu ZGlzX3BrdC0+ZGF0YV9sZW5ndGggPSBwYWNrZXQtPnRvdF9kYXRhX2J1Zl9sZW47CisJcm5kaXNf cGt0LT5wZXJfcGt0X2luZm9fb2Zmc2V0ID0gc2l6ZW9mKHJuZGlzX3BhY2tldCk7CisKKwlybmRp c19tc2dfc2l6ZSA9IFJORElTX01FU1NBR0VfU0laRShybmRpc19wYWNrZXQpOworCisJaWYgKG1f aGVhZC0+bV9mbGFncyAmIE1fVkxBTlRBRykgeworCQluZGlzXzgwMjFxX2luZm8gKnJwcGlfdmxh bl9pbmZvOworCisJCXJuZGlzX21zZ19zaXplICs9IFJORElTX1ZMQU5fUFBJX1NJWkU7CisJCXJw cGkgPSBodl9zZXRfcnBwaV9kYXRhKHJuZGlzX21lc2csIFJORElTX1ZMQU5fUFBJX1NJWkUsCisJ CSAgICBpZWVlXzgwMjFxX2luZm8pOworCisJCXJwcGlfdmxhbl9pbmZvID0gKG5kaXNfODAyMXFf aW5mbyAqKSgodWludDhfdCAqKXJwcGkgKworCQkgICAgcnBwaS0+cGVyX3BhY2tldF9pbmZvX29m ZnNldCk7CisJCXJwcGlfdmxhbl9pbmZvLT51MS5zMS52bGFuX2lkID0KKwkJICAgIG1faGVhZC0+ bV9wa3RoZHIuZXRoZXJfdnRhZyAmIDB4ZmZmOworCX0KKworCWlmIChtX2hlYWQtPm1fcGt0aGRy LmNzdW1fZmxhZ3MgJiBDU1VNX1RTTykgeworCQlybmRpc190Y3BfdHNvX2luZm8gKnRzb19pbmZv OwkKKwkJc3RydWN0IGV0aGVyX3ZsYW5faGVhZGVyICplaDsKKwkJaW50IGV0aGVyX2xlbjsKKwor CQkvKgorCQkgKiBYWFggbmVlZCBtX3B1bGx1cCBhbmQgdXNlIG10b2RvCisJCSAqLworCQllaCA9 IG10b2QobV9oZWFkLCBzdHJ1Y3QgZXRoZXJfdmxhbl9oZWFkZXIqKTsKKwkJaWYgKGVoLT5ldmxf ZW5jYXBfcHJvdG8gPT0gaHRvbnMoRVRIRVJUWVBFX1ZMQU4pKQorCQkJZXRoZXJfbGVuID0gRVRI RVJfSERSX0xFTiArIEVUSEVSX1ZMQU5fRU5DQVBfTEVOOworCQllbHNlCisJCQlldGhlcl9sZW4g PSBFVEhFUl9IRFJfTEVOOworCisJCXJuZGlzX21zZ19zaXplICs9IFJORElTX1RTT19QUElfU0la RTsKKwkJcnBwaSA9IGh2X3NldF9ycHBpX2RhdGEocm5kaXNfbWVzZywgUk5ESVNfVFNPX1BQSV9T SVpFLAorCQkgICAgdGNwX2xhcmdlX3NlbmRfaW5mbyk7CisKKwkJdHNvX2luZm8gPSAocm5kaXNf dGNwX3Rzb19pbmZvICopKCh1aW50OF90ICopcnBwaSArCisJCSAgICBycHBpLT5wZXJfcGFja2V0 X2luZm9fb2Zmc2V0KTsKKwkJdHNvX2luZm8tPmxzb192Ml94bWl0LnR5cGUgPQorCQkgICAgUk5E SVNfVENQX0xBUkdFX1NFTkRfT0ZGTE9BRF9WMl9UWVBFOworCisjaWZkZWYgSU5FVAorCQlpZiAo bV9oZWFkLT5tX3BrdGhkci5jc3VtX2ZsYWdzICYgQ1NVTV9JUF9UU08pIHsKKwkJCXN0cnVjdCBp cCAqaXAgPQorCQkJICAgIChzdHJ1Y3QgaXAgKikobV9oZWFkLT5tX2RhdGEgKyBldGhlcl9sZW4p OworCQkJdW5zaWduZWQgbG9uZyBpcGhfbGVuID0gaXAtPmlwX2hsIDw8IDI7CisJCQlzdHJ1Y3Qg dGNwaGRyICp0aCA9CisJCQkgICAgKHN0cnVjdCB0Y3BoZHIgKikoKGNhZGRyX3QpaXAgKyBpcGhf bGVuKTsKKworCQkJdHNvX2luZm8tPmxzb192Ml94bWl0LmlwX3ZlcnNpb24gPQorCQkJICAgIFJO RElTX1RDUF9MQVJHRV9TRU5EX09GRkxPQURfSVBWNDsKKwkJCWlwLT5pcF9sZW4gPSAwOworCQkJ aXAtPmlwX3N1bSA9IDA7CisKKwkJCXRoLT50aF9zdW0gPSBpbl9wc2V1ZG8oaXAtPmlwX3NyYy5z X2FkZHIsCisJCQkgICAgaXAtPmlwX2RzdC5zX2FkZHIsIGh0b25zKElQUFJPVE9fVENQKSk7CisJ CX0KKyNlbmRpZgorI2lmIGRlZmluZWQoSU5FVDYpICYmIGRlZmluZWQoSU5FVCkKKwkJZWxzZQor I2VuZGlmCisjaWZkZWYgSU5FVDYKKwkJeworCQkJc3RydWN0IGlwNl9oZHIgKmlwNiA9IChzdHJ1 Y3QgaXA2X2hkciAqKQorCQkJICAgIChtX2hlYWQtPm1fZGF0YSArIGV0aGVyX2xlbik7CisJCQlz dHJ1Y3QgdGNwaGRyICp0aCA9IChzdHJ1Y3QgdGNwaGRyICopKGlwNiArIDEpOworCisJCQl0c29f aW5mby0+bHNvX3YyX3htaXQuaXBfdmVyc2lvbiA9CisJCQkgICAgUk5ESVNfVENQX0xBUkdFX1NF TkRfT0ZGTE9BRF9JUFY2OworCQkJaXA2LT5pcDZfcGxlbiA9IDA7CisJCQl0aC0+dGhfc3VtID0g aW42X2Nrc3VtX3BzZXVkbyhpcDYsIDAsIElQUFJPVE9fVENQLCAwKTsKKwkJfQorI2VuZGlmCisJ CXRzb19pbmZvLT5sc29fdjJfeG1pdC50Y3BfaGVhZGVyX29mZnNldCA9IDA7CisJCXRzb19pbmZv LT5sc29fdjJfeG1pdC5tc3MgPSBtX2hlYWQtPm1fcGt0aGRyLnRzb19zZWdzejsKKwl9IGVsc2Ug aWYgKG1faGVhZC0+bV9wa3RoZHIuY3N1bV9mbGFncyAmIHNjLT5obl9jc3VtX2Fzc2lzdCkgewor CQlybmRpc190Y3BfaXBfY3N1bV9pbmZvICpjc3VtX2luZm87CisKKwkJcm5kaXNfbXNnX3NpemUg Kz0gUk5ESVNfQ1NVTV9QUElfU0laRTsKKwkJcnBwaSA9IGh2X3NldF9ycHBpX2RhdGEocm5kaXNf bWVzZywgUk5ESVNfQ1NVTV9QUElfU0laRSwKKwkJICAgIHRjcGlwX2Noa3N1bV9pbmZvKTsKKwkJ Y3N1bV9pbmZvID0gKHJuZGlzX3RjcF9pcF9jc3VtX2luZm8gKikoKHVpbnQ4X3QgKilycHBpICsK KwkJICAgIHJwcGktPnBlcl9wYWNrZXRfaW5mb19vZmZzZXQpOworCisJCWNzdW1faW5mby0+eG1p dC5pc19pcHY0ID0gMTsKKwkJaWYgKG1faGVhZC0+bV9wa3RoZHIuY3N1bV9mbGFncyAmIENTVU1f SVApCisJCQljc3VtX2luZm8tPnhtaXQuaXBfaGVhZGVyX2NzdW0gPSAxOworCisJCWlmIChtX2hl YWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBDU1VNX1RDUCkgeworCQkJY3N1bV9pbmZvLT54bWl0 LnRjcF9jc3VtID0gMTsKKwkJCWNzdW1faW5mby0+eG1pdC50Y3BfaGVhZGVyX29mZnNldCA9IDA7 CisJCX0gZWxzZSBpZiAobV9oZWFkLT5tX3BrdGhkci5jc3VtX2ZsYWdzICYgQ1NVTV9VRFApIHsK KwkJCWNzdW1faW5mby0+eG1pdC51ZHBfY3N1bSA9IDE7CisJCX0KKwl9CisKKwlybmRpc19tZXNn LT5tc2dfbGVuID0gcGFja2V0LT50b3RfZGF0YV9idWZfbGVuICsgcm5kaXNfbXNnX3NpemU7CisJ cGFja2V0LT50b3RfZGF0YV9idWZfbGVuID0gcm5kaXNfbWVzZy0+bXNnX2xlbjsKKworCS8qCisJ ICogQ2hpbW5leSBzZW5kLCBpZiB0aGUgcGFja2V0IGNvdWxkIGZpdCBpbnRvIG9uZSBjaGltbmV5 IGJ1ZmZlci4KKwkgKi8KKwlpZiAocGFja2V0LT50b3RfZGF0YV9idWZfbGVuIDwgc2MtPmhuX3R4 X2NoaW1uZXlfc2l6ZSkgeworCQluZXR2c2NfZGV2ICpuZXRfZGV2ID0gc2MtPm5ldF9kZXY7CisJ CXVpbnQzMl90IHNlbmRfYnVmX3NlY3Rpb25faWR4OworCisJCXNlbmRfYnVmX3NlY3Rpb25faWR4 ID0KKwkJICAgIGh2X252X2dldF9uZXh0X3NlbmRfc2VjdGlvbihuZXRfZGV2KTsKKwkJaWYgKHNl bmRfYnVmX3NlY3Rpb25faWR4ICE9CisJCSAgICBOVlNQXzFfQ0hJTU5FWV9TRU5EX0lOVkFMSURf U0VDVElPTl9JTkRFWCkgeworCQkJdWludDhfdCAqZGVzdCA9ICgodWludDhfdCAqKW5ldF9kZXYt PnNlbmRfYnVmICsKKwkJCSAgICAoc2VuZF9idWZfc2VjdGlvbl9pZHggKgorCQkJICAgICBuZXRf ZGV2LT5zZW5kX3NlY3Rpb25fc2l6ZSkpOworCisJCQltZW1jcHkoZGVzdCwgcm5kaXNfbWVzZywg cm5kaXNfbXNnX3NpemUpOworCQkJZGVzdCArPSBybmRpc19tc2dfc2l6ZTsKKwkJCW1fY29weWRh dGEobV9oZWFkLCAwLCBtX2hlYWQtPm1fcGt0aGRyLmxlbiwgZGVzdCk7CisKKwkJCXBhY2tldC0+ c2VuZF9idWZfc2VjdGlvbl9pZHggPSBzZW5kX2J1Zl9zZWN0aW9uX2lkeDsKKwkJCXBhY2tldC0+ c2VuZF9idWZfc2VjdGlvbl9zaXplID0KKwkJCSAgICBwYWNrZXQtPnRvdF9kYXRhX2J1Zl9sZW47 CisJCQlwYWNrZXQtPnBhZ2VfYnVmX2NvdW50ID0gMDsKKwkJCXNjLT5obl90eF9jaGltbmV5Kys7 CisJCQlnb3RvIGRvbmU7CisJCX0KKwl9CisKKwllcnJvciA9IGhuX3R4ZGVzY19kbWFtYXBfbG9h ZChzYywgdHhkLCAmbV9oZWFkLCBzZWdzLCAmbnNlZ3MpOworCWlmIChlcnJvcikgeworCQlpbnQg ZnJlZWQ7CisKKwkJLyoKKwkJICogVGhpcyBtYnVmIGlzIG5vdCBsaW5rZWQgdy8gdGhlIHR4ZCB5 ZXQsIHNvIGZyZWUgaXQgbm93LgorCQkgKi8KKwkJbV9mcmVlbShtX2hlYWQpOworCQkqbV9oZWFk MCA9IE5VTEw7CisKKwkJZnJlZWQgPSBobl90eGRlc2NfcHV0KHNjLCB0eGQpOworCQlLQVNTRVJU KGZyZWVkICE9IDAsCisJCSAgICAoImZhaWwgdG8gZnJlZSB0eGQgdXBvbiB0eGRtYSBlcnJvciIp KTsKKworCQlzYy0+aG5fdHhkbWFfZmFpbGVkKys7CisJCWlmX2luY19jb3VudGVyKHNjLT5obl9p ZnAsIElGQ09VTlRFUl9PRVJST1JTLCAxKTsKKwkJcmV0dXJuIGVycm9yOworCX0KKwkqbV9oZWFk MCA9IG1faGVhZDsKKworCXBhY2tldC0+cGFnZV9idWZfY291bnQgPSBuc2VncyArIEhWX1JGX05V TV9UWF9SRVNFUlZFRF9QQUdFX0JVRlM7CisKKwkvKiBzZW5kIHBhY2tldCB3aXRoIHBhZ2UgYnVm ZmVyICovCisJcGFja2V0LT5wYWdlX2J1ZmZlcnNbMF0ucGZuID0gYXRvcCh0eGQtPnJuZGlzX21z Z19wYWRkcik7CisJcGFja2V0LT5wYWdlX2J1ZmZlcnNbMF0ub2Zmc2V0ID0gdHhkLT5ybmRpc19t c2dfcGFkZHIgJiBQQUdFX01BU0s7CisJcGFja2V0LT5wYWdlX2J1ZmZlcnNbMF0ubGVuZ3RoID0g cm5kaXNfbXNnX3NpemU7CisKKwkvKgorCSAqIEZpbGwgdGhlIHBhZ2UgYnVmZmVycyB3aXRoIG1i dWYgaW5mbyBzdGFydGluZyBhdCBpbmRleAorCSAqIEhWX1JGX05VTV9UWF9SRVNFUlZFRF9QQUdF X0JVRlMuCisJICovCisJZm9yIChpID0gMDsgaSA8IG5zZWdzOyArK2kpIHsKKwkJaHZfdm1idXNf cGFnZV9idWZmZXIgKnBiID0gJnBhY2tldC0+cGFnZV9idWZmZXJzWworCQkgICAgaSArIEhWX1JG X05VTV9UWF9SRVNFUlZFRF9QQUdFX0JVRlNdOworCisJCXBiLT5wZm4gPSBhdG9wKHNlZ3NbaV0u ZHNfYWRkcik7CisJCXBiLT5vZmZzZXQgPSBzZWdzW2ldLmRzX2FkZHIgJiBQQUdFX01BU0s7CisJ CXBiLT5sZW5ndGggPSBzZWdzW2ldLmRzX2xlbjsKKwl9CisKKwlwYWNrZXQtPnNlbmRfYnVmX3Nl Y3Rpb25faWR4ID0KKwkgICAgTlZTUF8xX0NISU1ORVlfU0VORF9JTlZBTElEX1NFQ1RJT05fSU5E RVg7CisJcGFja2V0LT5zZW5kX2J1Zl9zZWN0aW9uX3NpemUgPSAwOworZG9uZToKKwl0eGQtPm0g PSBtX2hlYWQ7CisKKwkvKiBTZXQgdGhlIGNvbXBsZXRpb24gcm91dGluZSAqLworCXBhY2tldC0+ Y29tcGwuc2VuZC5vbl9zZW5kX2NvbXBsZXRpb24gPSBuZXR2c2NfeG1pdF9jb21wbGV0aW9uOwor CXBhY2tldC0+Y29tcGwuc2VuZC5zZW5kX2NvbXBsZXRpb25fY29udGV4dCA9IHBhY2tldDsKKwlw YWNrZXQtPmNvbXBsLnNlbmQuc2VuZF9jb21wbGV0aW9uX3RpZCA9ICh1aW50NjRfdCkodWludHB0 cl90KXR4ZDsKKworCXJldHVybiAwOworfQorCisvKgorICogU3RhcnQgYSB0cmFuc21pdCBvZiBv bmUgb3IgbW9yZSBwYWNrZXRzCisgKi8KK3N0YXRpYyBpbnQKK2huX3N0YXJ0X2xvY2tlZChzdHJ1 Y3QgaWZuZXQgKmlmcCwgaW50IGxlbikKK3sKKwlzdHJ1Y3QgaG5fc29mdGMgKnNjID0gaWZwLT5p Zl9zb2Z0YzsKKwlzdHJ1Y3QgaHZfZGV2aWNlICpkZXZpY2VfY3R4ID0gdm1idXNfZ2V0X2RldmN0 eChzYy0+aG5fZGV2KTsKIAogCWlmICgoaWZwLT5pZl9kcnZfZmxhZ3MgJiAoSUZGX0RSVl9SVU5O SU5HIHwgSUZGX0RSVl9PQUNUSVZFKSkgIT0KIAkgICAgSUZGX0RSVl9SVU5OSU5HKQogCQlyZXR1 cm4gMDsKIAogCXdoaWxlICghSUZRX0RSVl9JU19FTVBUWSgmaWZwLT5pZl9zbmQpKSB7Ci0JCWJ1 c19kbWFfc2VnbWVudF90IHNlZ3NbSE5fVFhfREFUQV9TRUdDTlRfTUFYXTsKLQkJaW50IGVycm9y LCBuc2VncywgaSwgc2VuZF9mYWlsZWQgPSAwOworCQlpbnQgZXJyb3IsIHNlbmRfZmFpbGVkID0g MDsKIAkJc3RydWN0IGhuX3R4ZGVzYyAqdHhkOwotCQluZXR2c2NfcGFja2V0ICpwYWNrZXQ7CiAJ CXN0cnVjdCBtYnVmICptX2hlYWQ7CiAKIAkJSUZRX0RSVl9ERVFVRVVFKCZpZnAtPmlmX3NuZCwg bV9oZWFkKTsKQEAgLTc5MywyMTYgKzk5OCwxNyBAQAogCQkJYnJlYWs7CiAJCX0KIAotCQlwYWNr ZXQgPSAmdHhkLT5uZXR2c2NfcGt0OwotCQlwYWNrZXQtPmlzX2RhdGFfcGt0ID0gVFJVRTsKLQkJ LyogSW5pdGlhbGl6ZSBpdCBmcm9tIHRoZSBtYnVmICovCi0JCXBhY2tldC0+dG90X2RhdGFfYnVm X2xlbiA9IG1faGVhZC0+bV9wa3RoZHIubGVuOwotCi0JCS8qCi0JCSAqIGV4dGVuc2lvbiBwb2lu dHMgdG8gdGhlIGFyZWEgcmVzZXJ2ZWQgZm9yIHRoZQotCQkgKiBybmRpc19maWx0ZXJfcGFja2V0 LCB3aGljaCBpcyBwbGFjZWQganVzdCBhZnRlcgotCQkgKiB0aGUgbmV0dnNjX3BhY2tldCAoYW5k IHJwcGkgc3RydWN0LCBpZiBwcmVzZW50OwotCQkgKiBsZW5ndGggaXMgdXBkYXRlZCBsYXRlciku Ci0JCSAqLwotCQlybmRpc19tZXNnID0gdHhkLT5ybmRpc19tc2c7Ci0JCS8qIFhYWCBub3QgbmVj ZXNzYXJ5ICovCi0JCW1lbXNldChybmRpc19tZXNnLCAwLCBITl9STkRJU19NU0dfTEVOKTsKLQkJ cm5kaXNfbWVzZy0+bmRpc19tc2dfdHlwZSA9IFJFTU9URV9ORElTX1BBQ0tFVF9NU0c7Ci0KLQkJ cm5kaXNfcGt0ID0gJnJuZGlzX21lc2ctPm1zZy5wYWNrZXQ7Ci0JCXJuZGlzX3BrdC0+ZGF0YV9v ZmZzZXQgPSBzaXplb2Yocm5kaXNfcGFja2V0KTsKLQkJcm5kaXNfcGt0LT5kYXRhX2xlbmd0aCA9 IHBhY2tldC0+dG90X2RhdGFfYnVmX2xlbjsKLQkJcm5kaXNfcGt0LT5wZXJfcGt0X2luZm9fb2Zm c2V0ID0gc2l6ZW9mKHJuZGlzX3BhY2tldCk7Ci0KLQkJcm5kaXNfbXNnX3NpemUgPSBSTkRJU19N RVNTQUdFX1NJWkUocm5kaXNfcGFja2V0KTsKLQotCQkvKgotCQkgKiBJZiB0aGUgSHlwZXItViBp bmZyYXN0cnVjdHVyZSBuZWVkcyB0byBlbWJlZCBhIFZMQU4gdGFnLAotCQkgKiBpbml0aWFsaXpl IG5ldHZzY19wYWNrZXQgYW5kIHJwcGkgc3RydWN0IHZhbHVlcyBhcyBuZWVkZWQuCi0JCSAqLwot CQlpZiAobV9oZWFkLT5tX2ZsYWdzICYgTV9WTEFOVEFHKSB7Ci0JCQkvKgotCQkJICogc2V0IHVw IHNvbWUgYWRkaXRpb25hbCBmaWVsZHMgc28gdGhlIEh5cGVyLVYgaW5mcmFzdHJ1Y3R1cmUgd2ls bCBzdHVmZiB0aGUgVkxBTiB0YWcKLQkJCSAqIGludG8gdGhlIGZyYW1lLgotCQkJICovCi0JCQly bmRpc19tc2dfc2l6ZSArPSBSTkRJU19WTEFOX1BQSV9TSVpFOwotCi0JCQlycHBpID0gaHZfc2V0 X3JwcGlfZGF0YShybmRpc19tZXNnLCBSTkRJU19WTEFOX1BQSV9TSVpFLAotCQkJICAgIGllZWVf ODAyMXFfaW5mbyk7Ci0JCQotCQkJLyogVkxBTiBpbmZvIGltbWVkaWF0ZWx5IGZvbGxvd3MgcnBw aSBzdHJ1Y3QgKi8KLQkJCXJwcGlfdmxhbl9pbmZvID0gKG5kaXNfODAyMXFfaW5mbyAqKSgoY2hh ciopcnBwaSArIAotCQkJICAgIHJwcGktPnBlcl9wYWNrZXRfaW5mb19vZmZzZXQpOwotCQkJLyog RnJlZUJTRCBkb2VzIG5vdCBzdXBwb3J0IENGSSBvciBwcmlvcml0eSAqLwotCQkJcnBwaV92bGFu X2luZm8tPnUxLnMxLnZsYW5faWQgPQotCQkJICAgIG1faGVhZC0+bV9wa3RoZHIuZXRoZXJfdnRh ZyAmIDB4ZmZmOwotCQl9Ci0KLQkJaWYgKG1faGVhZC0+bV9wa3RoZHIuY3N1bV9mbGFncyAmIENT VU1fVFNPKSB7Ci0JCQlzdHJ1Y3QgZXRoZXJfdmxhbl9oZWFkZXIgKmVoOwotCQkJaW50IGV0aGVy X2xlbjsKLQotCQkJZWggPSBtdG9kKG1faGVhZCwgc3RydWN0IGV0aGVyX3ZsYW5faGVhZGVyKik7 Ci0JCQlpZiAoZWgtPmV2bF9lbmNhcF9wcm90byA9PSBodG9ucyhFVEhFUlRZUEVfVkxBTikpIHsK LQkJCQlldGhlcl9sZW4gPSBFVEhFUl9IRFJfTEVOICsKLQkJCQkgICAgRVRIRVJfVkxBTl9FTkNB UF9MRU47Ci0JCQl9IGVsc2UgewotCQkJCWV0aGVyX2xlbiA9IEVUSEVSX0hEUl9MRU47Ci0JCQl9 Ci0KLQkJCXJuZGlzX21zZ19zaXplICs9IFJORElTX1RTT19QUElfU0laRTsKLQkJCXJwcGkgPSBo dl9zZXRfcnBwaV9kYXRhKHJuZGlzX21lc2csIFJORElTX1RTT19QUElfU0laRSwKLQkJCSAgICB0 Y3BfbGFyZ2Vfc2VuZF9pbmZvKTsKLQotCQkJdHNvX2luZm8gPSAocm5kaXNfdGNwX3Rzb19pbmZv ICopKChjaGFyICopcnBwaSArCi0JCQkgICAgcnBwaS0+cGVyX3BhY2tldF9pbmZvX29mZnNldCk7 Ci0JCQl0c29faW5mby0+bHNvX3YyX3htaXQudHlwZSA9Ci0JCQkgICAgUk5ESVNfVENQX0xBUkdF X1NFTkRfT0ZGTE9BRF9WMl9UWVBFOwotCi0jaWZkZWYgSU5FVAotCQkJaWYgKG1faGVhZC0+bV9w a3RoZHIuY3N1bV9mbGFncyAmIENTVU1fSVBfVFNPKSB7Ci0JCQkJc3RydWN0IGlwICppcCA9Ci0J CQkJICAgIChzdHJ1Y3QgaXAgKikobV9oZWFkLT5tX2RhdGEgKyBldGhlcl9sZW4pOwotCQkJCXVu c2lnbmVkIGxvbmcgaXBoX2xlbiA9IGlwLT5pcF9obCA8PCAyOwotCQkJCXN0cnVjdCB0Y3BoZHIg KnRoID0KLQkJCQkgICAgKHN0cnVjdCB0Y3BoZHIgKikoKGNhZGRyX3QpaXAgKyBpcGhfbGVuKTsK LQkJCQotCQkJCXRzb19pbmZvLT5sc29fdjJfeG1pdC5pcF92ZXJzaW9uID0KLQkJCQkgICAgUk5E SVNfVENQX0xBUkdFX1NFTkRfT0ZGTE9BRF9JUFY0OwotCQkJCWlwLT5pcF9sZW4gPSAwOwotCQkJ CWlwLT5pcF9zdW0gPSAwOwotCQkJCi0JCQkJdGgtPnRoX3N1bSA9IGluX3BzZXVkbyhpcC0+aXBf c3JjLnNfYWRkciwKLQkJCQkgICAgaXAtPmlwX2RzdC5zX2FkZHIsIGh0b25zKElQUFJPVE9fVENQ KSk7Ci0JCQl9Ci0jZW5kaWYKLSNpZiBkZWZpbmVkKElORVQ2KSAmJiBkZWZpbmVkKElORVQpCi0J CQllbHNlCi0jZW5kaWYKLSNpZmRlZiBJTkVUNgotCQkJewotCQkJCXN0cnVjdCBpcDZfaGRyICpp cDYgPSAoc3RydWN0IGlwNl9oZHIgKikKLQkJCQkgICAgKG1faGVhZC0+bV9kYXRhICsgZXRoZXJf bGVuKTsKLQkJCQlzdHJ1Y3QgdGNwaGRyICp0aCA9IChzdHJ1Y3QgdGNwaGRyICopKGlwNiArIDEp OwotCi0JCQkJdHNvX2luZm8tPmxzb192Ml94bWl0LmlwX3ZlcnNpb24gPQotCQkJCSAgICBSTkRJ U19UQ1BfTEFSR0VfU0VORF9PRkZMT0FEX0lQVjY7Ci0JCQkJaXA2LT5pcDZfcGxlbiA9IDA7Ci0J CQkJdGgtPnRoX3N1bSA9Ci0JCQkJICAgIGluNl9ja3N1bV9wc2V1ZG8oaXA2LCAwLCBJUFBST1RP X1RDUCwgMCk7Ci0JCQl9Ci0jZW5kaWYKLQkJCXRzb19pbmZvLT5sc29fdjJfeG1pdC50Y3BfaGVh ZGVyX29mZnNldCA9IDA7Ci0JCQl0c29faW5mby0+bHNvX3YyX3htaXQubXNzID0gbV9oZWFkLT5t X3BrdGhkci50c29fc2Vnc3o7Ci0JCX0gZWxzZSBpZiAobV9oZWFkLT5tX3BrdGhkci5jc3VtX2Zs YWdzICYgc2MtPmhuX2NzdW1fYXNzaXN0KSB7Ci0JCQlybmRpc19tc2dfc2l6ZSArPSBSTkRJU19D U1VNX1BQSV9TSVpFOwotCQkJcnBwaSA9IGh2X3NldF9ycHBpX2RhdGEocm5kaXNfbWVzZywgUk5E SVNfQ1NVTV9QUElfU0laRSwKLQkJCSAgICB0Y3BpcF9jaGtzdW1faW5mbyk7Ci0JCQljc3VtX2lu Zm8gPSAocm5kaXNfdGNwX2lwX2NzdW1faW5mbyAqKSgoY2hhciopcnBwaSArCi0JCQkgICAgcnBw aS0+cGVyX3BhY2tldF9pbmZvX29mZnNldCk7Ci0KLQkJCWNzdW1faW5mby0+eG1pdC5pc19pcHY0 ID0gMTsKLQkJCWlmIChtX2hlYWQtPm1fcGt0aGRyLmNzdW1fZmxhZ3MgJiBDU1VNX0lQKQotCQkJ CWNzdW1faW5mby0+eG1pdC5pcF9oZWFkZXJfY3N1bSA9IDE7Ci0KLQkJCWlmIChtX2hlYWQtPm1f cGt0aGRyLmNzdW1fZmxhZ3MgJiBDU1VNX1RDUCkgewotCQkJCWNzdW1faW5mby0+eG1pdC50Y3Bf Y3N1bSA9IDE7Ci0JCQkJY3N1bV9pbmZvLT54bWl0LnRjcF9oZWFkZXJfb2Zmc2V0ID0gMDsKLQkJ CX0gZWxzZSBpZiAobV9oZWFkLT5tX3BrdGhkci5jc3VtX2ZsYWdzICYgQ1NVTV9VRFApIHsKLQkJ CQljc3VtX2luZm8tPnhtaXQudWRwX2NzdW0gPSAxOwotCQkJfQotCQl9Ci0KLQkJcm5kaXNfbWVz Zy0+bXNnX2xlbiA9IHBhY2tldC0+dG90X2RhdGFfYnVmX2xlbiArIHJuZGlzX21zZ19zaXplOwot CQlwYWNrZXQtPnRvdF9kYXRhX2J1Zl9sZW4gPSBybmRpc19tZXNnLT5tc2dfbGVuOwotCi0JCS8q IHNlbmQgcGFja2V0IHdpdGggc2VuZCBidWZmZXIgKi8KLQkJaWYgKHBhY2tldC0+dG90X2RhdGFf YnVmX2xlbiA8IHNjLT5obl90eF9jaGltbmV5X3NpemUpIHsKLQkJCXVpbnQzMl90IHNlbmRfYnVm X3NlY3Rpb25faWR4OwotCi0JCQlzZW5kX2J1Zl9zZWN0aW9uX2lkeCA9Ci0JCQkgICAgaHZfbnZf Z2V0X25leHRfc2VuZF9zZWN0aW9uKG5ldF9kZXYpOwotCQkJaWYgKHNlbmRfYnVmX3NlY3Rpb25f aWR4ICE9Ci0JCQkgICAgTlZTUF8xX0NISU1ORVlfU0VORF9JTlZBTElEX1NFQ1RJT05fSU5ERVgp IHsKLQkJCQl1aW50OF90ICpkZXN0ID0gKCh1aW50OF90ICopbmV0X2Rldi0+c2VuZF9idWYgKwot CQkJCSAgICAoc2VuZF9idWZfc2VjdGlvbl9pZHggKgotCQkJCSAgICAgbmV0X2Rldi0+c2VuZF9z ZWN0aW9uX3NpemUpKTsKLQotCQkJCW1lbWNweShkZXN0LCBybmRpc19tZXNnLCBybmRpc19tc2df c2l6ZSk7Ci0JCQkJZGVzdCArPSBybmRpc19tc2dfc2l6ZTsKLQotCQkJCW1fY29weWRhdGEobV9o ZWFkLCAwLCBtX2hlYWQtPm1fcGt0aGRyLmxlbiwKLQkJCQkgICAgZGVzdCk7Ci0KLQkJCQlwYWNr ZXQtPnNlbmRfYnVmX3NlY3Rpb25faWR4ID0KLQkJCQkgICAgc2VuZF9idWZfc2VjdGlvbl9pZHg7 Ci0JCQkJcGFja2V0LT5zZW5kX2J1Zl9zZWN0aW9uX3NpemUgPQotCQkJCSAgICBwYWNrZXQtPnRv dF9kYXRhX2J1Zl9sZW47Ci0JCQkJcGFja2V0LT5wYWdlX2J1Zl9jb3VudCA9IDA7Ci0JCQkJc2Mt PmhuX3R4X2NoaW1uZXkrKzsKLQkJCQlnb3RvIGRvX3NlbmQ7Ci0JCQl9Ci0JCX0KLQotCQllcnJv ciA9IGhuX3R4ZGVzY19kbWFtYXBfbG9hZChzYywgdHhkLCAmbV9oZWFkLCBzZWdzLCAmbnNlZ3Mp OworCQllcnJvciA9IGhuX2VuY2FwKHNjLCB0eGQsICZtX2hlYWQpOwogCQlpZiAoZXJyb3IpIHsK LQkJCWludCBmcmVlZDsKLQotCQkJLyoKLQkJCSAqIFRoaXMgbWJ1ZiBpcyBub3QgbGlua2VkIHcv IHRoZSB0eGQgeWV0LCBzbyBmcmVlCi0JCQkgKiBpdCBub3cuCi0JCQkgKi8KLQkJCW1fZnJlZW0o bV9oZWFkKTsKLQkJCWZyZWVkID0gaG5fdHhkZXNjX3B1dChzYywgdHhkKTsKLQkJCUtBU1NFUlQo ZnJlZWQgIT0gMCwKLQkJCSAgICAoImZhaWwgdG8gZnJlZSB0eGQgdXBvbiB0eGRtYSBlcnJvciIp KTsKLQotCQkJc2MtPmhuX3R4ZG1hX2ZhaWxlZCsrOwotCQkJaWZfaW5jX2NvdW50ZXIoaWZwLCBJ RkNPVU5URVJfT0VSUk9SUywgMSk7CisJCQkvKiBCb3RoIHR4ZCBhbmQgbV9oZWFkIGFyZSBmcmVl ZCAqLwogCQkJY29udGludWU7CiAJCX0KLQotCQlwYWNrZXQtPnBhZ2VfYnVmX2NvdW50ID0gbnNl Z3MgKwotCQkgICAgSFZfUkZfTlVNX1RYX1JFU0VSVkVEX1BBR0VfQlVGUzsKLQotCQkvKiBzZW5k IHBhY2tldCB3aXRoIHBhZ2UgYnVmZmVyICovCi0JCXBhY2tldC0+cGFnZV9idWZmZXJzWzBdLnBm biA9IGF0b3AodHhkLT5ybmRpc19tc2dfcGFkZHIpOwotCQlwYWNrZXQtPnBhZ2VfYnVmZmVyc1sw XS5vZmZzZXQgPQotCQkgICAgdHhkLT5ybmRpc19tc2dfcGFkZHIgJiBQQUdFX01BU0s7Ci0JCXBh Y2tldC0+cGFnZV9idWZmZXJzWzBdLmxlbmd0aCA9IHJuZGlzX21zZ19zaXplOwotCi0JCS8qCi0J CSAqIEZpbGwgdGhlIHBhZ2UgYnVmZmVycyB3aXRoIG1idWYgaW5mbyBzdGFydGluZyBhdCBpbmRl eAotCQkgKiBIVl9SRl9OVU1fVFhfUkVTRVJWRURfUEFHRV9CVUZTLgotCQkgKi8KLQkJZm9yIChp ID0gMDsgaSA8IG5zZWdzOyArK2kpIHsKLQkJCWh2X3ZtYnVzX3BhZ2VfYnVmZmVyICpwYiA9ICZw YWNrZXQtPnBhZ2VfYnVmZmVyc1sKLQkJCSAgICBpICsgSFZfUkZfTlVNX1RYX1JFU0VSVkVEX1BB R0VfQlVGU107Ci0KLQkJCXBiLT5wZm4gPSBhdG9wKHNlZ3NbaV0uZHNfYWRkcik7Ci0JCQlwYi0+ b2Zmc2V0ID0gc2Vnc1tpXS5kc19hZGRyICYgUEFHRV9NQVNLOwotCQkJcGItPmxlbmd0aCA9IHNl Z3NbaV0uZHNfbGVuOwotCQl9Ci0KLQkJcGFja2V0LT5zZW5kX2J1Zl9zZWN0aW9uX2lkeCA9IAot CQkgICAgTlZTUF8xX0NISU1ORVlfU0VORF9JTlZBTElEX1NFQ1RJT05fSU5ERVg7Ci0JCXBhY2tl dC0+c2VuZF9idWZfc2VjdGlvbl9zaXplID0gMDsKLQotZG9fc2VuZDoKLQkJdHhkLT5tID0gbV9o ZWFkOwotCi0JCS8qIFNldCB0aGUgY29tcGxldGlvbiByb3V0aW5lICovCi0JCXBhY2tldC0+Y29t cGwuc2VuZC5vbl9zZW5kX2NvbXBsZXRpb24gPSBuZXR2c2NfeG1pdF9jb21wbGV0aW9uOwotCQlw YWNrZXQtPmNvbXBsLnNlbmQuc2VuZF9jb21wbGV0aW9uX2NvbnRleHQgPSBwYWNrZXQ7Ci0JCXBh Y2tldC0+Y29tcGwuc2VuZC5zZW5kX2NvbXBsZXRpb25fdGlkID0KLQkJICAgICh1aW50NjRfdCko dWludHB0cl90KXR4ZDsKLQogYWdhaW46CiAJCS8qCiAJCSAqIE1ha2Ugc3VyZSB0aGF0IHR4ZCBp cyBub3QgZnJlZWQgYmVmb3JlIEVUSEVSX0JQRl9NVEFQLgogCQkgKi8KIAkJaG5fdHhkZXNjX2hv bGQodHhkKTsKLQkJZXJyb3IgPSBodl9udl9vbl9zZW5kKGRldmljZV9jdHgsIHBhY2tldCk7CisJ CWVycm9yID0gaHZfbnZfb25fc2VuZChkZXZpY2VfY3R4LCAmdHhkLT5uZXR2c2NfcGt0KTsKIAkJ aWYgKCFlcnJvcikgewogCQkJRVRIRVJfQlBGX01UQVAoaWZwLCBtX2hlYWQpOwogCQkJaWZfaW5j X2NvdW50ZXIoaWZwLCBJRkNPVU5URVJfT1BBQ0tFVFMsIDEpOwoK --b1_7d83666eeeadbf8cd33a405b6c1d7cae-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:31:59 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3736BA9BBD4 for ; Fri, 5 Feb 2016 05:31:59 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 21A96EA7 for ; Fri, 5 Feb 2016 05:31:59 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 1FD021077BD; Fri, 5 Feb 2016 05:31:59 +0000 (UTC) Date: Fri, 5 Feb 2016 05:31:59 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5159+325+0db17ec577dfe5ca@reviews.freebsd.org Subject: [Differential] [Closed] D5159: hyperv/hn: Recover half of the chimney sending space Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5159: hyperv/hn: Recover half of the chimney sending space X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: MzRmNWEyNTk4ODRlNWNkNGYzNTA2Yjk4Nzc1IFa0M88= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_e4d2b41832295454d9e2b119a14e0837" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:31:59 -0000 --b1_e4d2b41832295454d9e2b119a14e0837 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295303: hyperv/hn: Recover half of the chimney sending space (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5159?vs=12926&id=13037#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5159?vs=12926&id=13037 REVISION DETAIL https://reviews.freebsd.org/D5159 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_net_vsc.c CHANGE DETAILS diff --git a/head/sys/dev/hyperv/netvsc/hv_net_vsc.c b/head/sys/dev/hyperv/netvsc/hv_net_vsc.c --- a/head/sys/dev/hyperv/netvsc/hv_net_vsc.c +++ b/head/sys/dev/hyperv/netvsc/hv_net_vsc.c @@ -136,15 +136,15 @@ int i; for (i = 0; i < bitsmap_words; i++) { - idx = ffs(~bitsmap[i]); + idx = ffsl(~bitsmap[i]); if (0 == idx) continue; idx--; - if (i * BITS_PER_LONG + idx >= net_dev->send_section_count) - return (ret); + KASSERT(i * BITS_PER_LONG + idx < net_dev->send_section_count, + ("invalid i %d and idx %lu", i, idx)); - if (synch_test_and_set_bit(idx, &bitsmap[i])) + if (atomic_testandset_long(&bitsmap[i], idx)) continue; ret = i * BITS_PER_LONG + idx; @@ -789,8 +789,27 @@ if (NULL != net_vsc_pkt) { if (net_vsc_pkt->send_buf_section_idx != NVSP_1_CHIMNEY_SEND_INVALID_SECTION_INDEX) { - synch_change_bit(net_vsc_pkt->send_buf_section_idx, - net_dev->send_section_bitsmap); + u_long mask; + int idx; + + idx = net_vsc_pkt->send_buf_section_idx / + BITS_PER_LONG; + KASSERT(idx < net_dev->bitsmap_words, + ("invalid section index %u", + net_vsc_pkt->send_buf_section_idx)); + mask = 1UL << + (net_vsc_pkt->send_buf_section_idx % + BITS_PER_LONG); + + KASSERT(net_dev->send_section_bitsmap[idx] & + mask, + ("index bitmap 0x%lx, section index %u, " + "bitmap idx %d, bitmask 0x%lx", + net_dev->send_section_bitsmap[idx], + net_vsc_pkt->send_buf_section_idx, + idx, mask)); + atomic_clear_long( + &net_dev->send_section_bitsmap[idx], mask); } /* Notify the layer above us */ EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list --b1_e4d2b41832295454d9e2b119a14e0837 Content-Type: text/x-patch; charset=utf-8; name="D5159.13037.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5159.13037.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuYyBiL2hl YWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldF92c2MuYwotLS0gYS9oZWFkL3N5cy9kZXYv aHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2 c2MvaHZfbmV0X3ZzYy5jCkBAIC0xMzYsMTUgKzEzNiwxNSBAQAogCWludCBpOwogCiAJZm9yIChp ID0gMDsgaSA8IGJpdHNtYXBfd29yZHM7IGkrKykgewotCQlpZHggPSBmZnMofmJpdHNtYXBbaV0p OworCQlpZHggPSBmZnNsKH5iaXRzbWFwW2ldKTsKIAkJaWYgKDAgPT0gaWR4KQogCQkJY29udGlu dWU7CiAKIAkJaWR4LS07Ci0JCWlmIChpICogQklUU19QRVJfTE9ORyArIGlkeCA+PSBuZXRfZGV2 LT5zZW5kX3NlY3Rpb25fY291bnQpCi0JCQlyZXR1cm4gKHJldCk7CisJCUtBU1NFUlQoaSAqIEJJ VFNfUEVSX0xPTkcgKyBpZHggPCBuZXRfZGV2LT5zZW5kX3NlY3Rpb25fY291bnQsCisJCSAgICAo ImludmFsaWQgaSAlZCBhbmQgaWR4ICVsdSIsIGksIGlkeCkpOwogCi0JCWlmIChzeW5jaF90ZXN0 X2FuZF9zZXRfYml0KGlkeCwgJmJpdHNtYXBbaV0pKQorCQlpZiAoYXRvbWljX3Rlc3RhbmRzZXRf bG9uZygmYml0c21hcFtpXSwgaWR4KSkKIAkJCWNvbnRpbnVlOwogCiAJCXJldCA9IGkgKiBCSVRT X1BFUl9MT05HICsgaWR4OwpAQCAtNzg5LDggKzc4OSwyNyBAQAogCQlpZiAoTlVMTCAhPSBuZXRf dnNjX3BrdCkgewogCQkJaWYgKG5ldF92c2NfcGt0LT5zZW5kX2J1Zl9zZWN0aW9uX2lkeCAhPQog CQkJICAgIE5WU1BfMV9DSElNTkVZX1NFTkRfSU5WQUxJRF9TRUNUSU9OX0lOREVYKSB7Ci0JCQkJ c3luY2hfY2hhbmdlX2JpdChuZXRfdnNjX3BrdC0+c2VuZF9idWZfc2VjdGlvbl9pZHgsCi0JCQkJ ICAgIG5ldF9kZXYtPnNlbmRfc2VjdGlvbl9iaXRzbWFwKTsKKwkJCQl1X2xvbmcgbWFzazsKKwkJ CQlpbnQgaWR4OworCisJCQkJaWR4ID0gbmV0X3ZzY19wa3QtPnNlbmRfYnVmX3NlY3Rpb25faWR4 IC8KKwkJCQkgICAgQklUU19QRVJfTE9ORzsKKwkJCQlLQVNTRVJUKGlkeCA8IG5ldF9kZXYtPmJp dHNtYXBfd29yZHMsCisJCQkJICAgICgiaW52YWxpZCBzZWN0aW9uIGluZGV4ICV1IiwKKwkJCQkg ICAgIG5ldF92c2NfcGt0LT5zZW5kX2J1Zl9zZWN0aW9uX2lkeCkpOworCQkJCW1hc2sgPSAxVUwg PDwKKwkJCQkgICAgKG5ldF92c2NfcGt0LT5zZW5kX2J1Zl9zZWN0aW9uX2lkeCAlCisJCQkJICAg ICBCSVRTX1BFUl9MT05HKTsKKworCQkJCUtBU1NFUlQobmV0X2Rldi0+c2VuZF9zZWN0aW9uX2Jp dHNtYXBbaWR4XSAmCisJCQkJICAgIG1hc2ssCisJCQkJICAgICgiaW5kZXggYml0bWFwIDB4JWx4 LCBzZWN0aW9uIGluZGV4ICV1LCAiCisJCQkJICAgICAiYml0bWFwIGlkeCAlZCwgYml0bWFzayAw eCVseCIsCisJCQkJICAgICBuZXRfZGV2LT5zZW5kX3NlY3Rpb25fYml0c21hcFtpZHhdLAorCQkJ CSAgICAgbmV0X3ZzY19wa3QtPnNlbmRfYnVmX3NlY3Rpb25faWR4LAorCQkJCSAgICAgaWR4LCBt YXNrKSk7CisJCQkJYXRvbWljX2NsZWFyX2xvbmcoCisJCQkJICAgICZuZXRfZGV2LT5zZW5kX3Nl Y3Rpb25fYml0c21hcFtpZHhdLCBtYXNrKTsKIAkJCX0KIAkJCQogCQkJLyogTm90aWZ5IHRoZSBs YXllciBhYm92ZSB1cyAqLwoK --b1_e4d2b41832295454d9e2b119a14e0837-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:38:27 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C640A9BF05 for ; Fri, 5 Feb 2016 05:38:27 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 873DC13CB for ; Fri, 5 Feb 2016 05:38:27 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 861F8107AE2; Fri, 5 Feb 2016 05:38:27 +0000 (UTC) Date: Fri, 5 Feb 2016 05:38:27 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5166+325+a3471c38b880b49e@reviews.freebsd.org Subject: [Differential] [Closed] D5166: hyperv/hn: Increase LRO entry count to 128 by default Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5166: hyperv/hn: Increase LRO entry count to 128 by default X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NWRmOGI3NDFlMWM4OTg3OGUxYjZjYzIwNmVlIFa0NVM= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_e7481af8f9773ced40f18d8239d5177a" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:38:27 -0000 --b1_e7481af8f9773ced40f18d8239d5177a Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295304: hyperv/hn: Increase LRO entry count to 128 by default (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5166?vs=12947&id=13038#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5166?vs=12947&id=13038 REVISION DETAIL https://reviews.freebsd.org/D5166 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -132,6 +132,8 @@ /* YYY should get it from the underlying channel */ #define HN_TX_DESC_CNT 512 +#define HN_LROENT_CNT_DEF 128 + #define HN_RNDIS_MSG_LEN \ (sizeof(rndis_msg) + \ RNDIS_VLAN_PPI_SIZE + \ @@ -232,6 +234,13 @@ static int hn_direct_tx_size = HN_DIRECT_TX_SIZE_DEF; TUNABLE_INT("dev.hn.direct_tx_size", &hn_direct_tx_size); +#if defined(INET) || defined(INET6) +#if __FreeBSD_version >= 1100095 +static int hn_lro_entry_count = HN_LROENT_CNT_DEF; +TUNABLE_INT("dev.hn.lro_entry_count", &hn_lro_entry_count); +#endif +#endif + /* * Forward declarations */ @@ -335,6 +344,11 @@ #if __FreeBSD_version >= 1100045 int tso_maxlen; #endif +#if defined(INET) || defined(INET6) +#if __FreeBSD_version >= 1100095 + int lroent_cnt; +#endif +#endif sc = device_get_softc(dev); if (sc == NULL) { @@ -417,9 +431,17 @@ } #if defined(INET) || defined(INET6) +#if __FreeBSD_version >= 1100095 + lroent_cnt = hn_lro_entry_count; + if (lroent_cnt < TCP_LRO_ENTRIES) + lroent_cnt = TCP_LRO_ENTRIES; + tcp_lro_init_args(&sc->hn_lro, ifp, lroent_cnt, 0); + device_printf(dev, "LRO: entry count %d\n", lroent_cnt); +#else tcp_lro_init(&sc->hn_lro); /* Driver private LRO settings */ sc->hn_lro.ifp = ifp; +#endif #ifdef HN_LRO_HIWAT sc->hn_lro.lro_hiwat = sc->hn_lro_hiwat; #endif @@ -547,6 +569,12 @@ SYSCTL_ADD_INT(dc_ctx, dc_child, OID_AUTO, "direct_tx_size", CTLFLAG_RD, &hn_direct_tx_size, 0, "Size of the packet for direct transmission"); +#if defined(INET) || defined(INET6) +#if __FreeBSD_version >= 1100095 + SYSCTL_ADD_INT(dc_ctx, dc_child, OID_AUTO, "lro_entry_count", + CTLFLAG_RD, &hn_lro_entry_count, 0, "LRO entry count"); +#endif +#endif } return (0); EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list --b1_e7481af8f9773ced40f18d8239d5177a Content-Type: text/x-patch; charset=utf-8; name="D5166.13038.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5166.13038.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTEzMiw2ICsxMzIsOCBAQAogLyogWVlZIHNob3VsZCBnZXQgaXQgZnJvbSB0aGUgdW5k ZXJseWluZyBjaGFubmVsICovCiAjZGVmaW5lIEhOX1RYX0RFU0NfQ05UCQkJNTEyCiAKKyNkZWZp bmUgSE5fTFJPRU5UX0NOVF9ERUYJCTEyOAorCiAjZGVmaW5lIEhOX1JORElTX01TR19MRU4JCVwK ICAgICAoc2l6ZW9mKHJuZGlzX21zZykgKwkJXAogICAgICBSTkRJU19WTEFOX1BQSV9TSVpFICsJ CVwKQEAgLTIzMiw2ICsyMzQsMTMgQEAKIHN0YXRpYyBpbnQgaG5fZGlyZWN0X3R4X3NpemUgPSBI Tl9ESVJFQ1RfVFhfU0laRV9ERUY7CiBUVU5BQkxFX0lOVCgiZGV2LmhuLmRpcmVjdF90eF9zaXpl IiwgJmhuX2RpcmVjdF90eF9zaXplKTsKIAorI2lmIGRlZmluZWQoSU5FVCkgfHwgZGVmaW5lZChJ TkVUNikKKyNpZiBfX0ZyZWVCU0RfdmVyc2lvbiA+PSAxMTAwMDk1CitzdGF0aWMgaW50IGhuX2xy b19lbnRyeV9jb3VudCA9IEhOX0xST0VOVF9DTlRfREVGOworVFVOQUJMRV9JTlQoImRldi5obi5s cm9fZW50cnlfY291bnQiLCAmaG5fbHJvX2VudHJ5X2NvdW50KTsKKyNlbmRpZgorI2VuZGlmCisK IC8qCiAgKiBGb3J3YXJkIGRlY2xhcmF0aW9ucwogICovCkBAIC0zMzUsNiArMzQ0LDExIEBACiAj aWYgX19GcmVlQlNEX3ZlcnNpb24gPj0gMTEwMDA0NQogCWludCB0c29fbWF4bGVuOwogI2VuZGlm CisjaWYgZGVmaW5lZChJTkVUKSB8fCBkZWZpbmVkKElORVQ2KQorI2lmIF9fRnJlZUJTRF92ZXJz aW9uID49IDExMDAwOTUKKwlpbnQgbHJvZW50X2NudDsKKyNlbmRpZgorI2VuZGlmCiAKIAlzYyA9 IGRldmljZV9nZXRfc29mdGMoZGV2KTsKIAlpZiAoc2MgPT0gTlVMTCkgewpAQCAtNDE3LDkgKzQz MSwxNyBAQAogCX0KIAogI2lmIGRlZmluZWQoSU5FVCkgfHwgZGVmaW5lZChJTkVUNikKKyNpZiBf X0ZyZWVCU0RfdmVyc2lvbiA+PSAxMTAwMDk1CisJbHJvZW50X2NudCA9IGhuX2xyb19lbnRyeV9j b3VudDsKKwlpZiAobHJvZW50X2NudCA8IFRDUF9MUk9fRU5UUklFUykKKwkJbHJvZW50X2NudCA9 IFRDUF9MUk9fRU5UUklFUzsKKwl0Y3BfbHJvX2luaXRfYXJncygmc2MtPmhuX2xybywgaWZwLCBs cm9lbnRfY250LCAwKTsKKwlkZXZpY2VfcHJpbnRmKGRldiwgIkxSTzogZW50cnkgY291bnQgJWRc biIsIGxyb2VudF9jbnQpOworI2Vsc2UKIAl0Y3BfbHJvX2luaXQoJnNjLT5obl9scm8pOwogCS8q IERyaXZlciBwcml2YXRlIExSTyBzZXR0aW5ncyAqLwogCXNjLT5obl9scm8uaWZwID0gaWZwOwor I2VuZGlmCiAjaWZkZWYgSE5fTFJPX0hJV0FUCiAJc2MtPmhuX2xyby5scm9faGl3YXQgPSBzYy0+ aG5fbHJvX2hpd2F0OwogI2VuZGlmCkBAIC01NDcsNiArNTY5LDEyIEBACiAJCVNZU0NUTF9BRERf SU5UKGRjX2N0eCwgZGNfY2hpbGQsIE9JRF9BVVRPLCAiZGlyZWN0X3R4X3NpemUiLAogCQkgICAg Q1RMRkxBR19SRCwgJmhuX2RpcmVjdF90eF9zaXplLCAwLAogCQkgICAgIlNpemUgb2YgdGhlIHBh Y2tldCBmb3IgZGlyZWN0IHRyYW5zbWlzc2lvbiIpOworI2lmIGRlZmluZWQoSU5FVCkgfHwgZGVm aW5lZChJTkVUNikKKyNpZiBfX0ZyZWVCU0RfdmVyc2lvbiA+PSAxMTAwMDk1CisJCVNZU0NUTF9B RERfSU5UKGRjX2N0eCwgZGNfY2hpbGQsIE9JRF9BVVRPLCAibHJvX2VudHJ5X2NvdW50IiwKKwkJ ICAgIENUTEZMQUdfUkQsICZobl9scm9fZW50cnlfY291bnQsIDAsICJMUk8gZW50cnkgY291bnQi KTsKKyNlbmRpZgorI2VuZGlmCiAJfQogCiAJcmV0dXJuICgwKTsKCg== --b1_e7481af8f9773ced40f18d8239d5177a-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:44:57 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BCE4A9A242 for ; Fri, 5 Feb 2016 05:44:57 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id F03FB1AFA for ; Fri, 5 Feb 2016 05:44:56 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id EEA95107FDF; Fri, 5 Feb 2016 05:44:56 +0000 (UTC) Date: Fri, 5 Feb 2016 05:44:56 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5167+325+acc9bcb5dbcc28f2@reviews.freebsd.org Subject: [Differential] [Closed] D5167: hyperv/hn: Move LRO flush to the channel processing rollup Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5167: hyperv/hn: Move LRO flush to the channel processing rollup X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: N2NlNTE2YjI3MTBjYjEzNmM4OWIyM2FkMjc1IFa0Ntg= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_c9198d81d09fe437aa4521f8bf434026" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:44:57 -0000 --b1_c9198d81d09fe437aa4521f8bf434026 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295305: hyperv/hn: Move LRO flush to the channel processing rollup (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5167?vs=12948&id=13039#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5167?vs=12948&id=13039 REVISION DETAIL https://reviews.freebsd.org/D5167 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -764,6 +764,15 @@ netvsc_channel_rollup(struct hv_device *device_ctx) { struct hn_softc *sc = device_get_softc(device_ctx->device); +#if defined(INET) || defined(INET6) + struct lro_ctrl *lro = &sc->hn_lro; + struct lro_entry *queued; + + while ((queued = SLIST_FIRST(&lro->lro_active)) != NULL) { + SLIST_REMOVE_HEAD(&lro->lro_active, next); + tcp_lro_flush(lro, queued); + } +#endif if (!sc->hn_txeof) return; @@ -1338,18 +1347,8 @@ } void -netvsc_recv_rollup(struct hv_device *device_ctx) +netvsc_recv_rollup(struct hv_device *device_ctx __unused) { -#if defined(INET) || defined(INET6) - hn_softc_t *sc = device_get_softc(device_ctx->device); - struct lro_ctrl *lro = &sc->hn_lro; - struct lro_entry *queued; - - while ((queued = SLIST_FIRST(&lro->lro_active)) != NULL) { - SLIST_REMOVE_HEAD(&lro->lro_active, next); - tcp_lro_flush(lro, queued); - } -#endif } /* EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, adrian, network Cc: freebsd-net-list --b1_c9198d81d09fe437aa4521f8bf434026 Content-Type: text/x-patch; charset=utf-8; name="D5167.13039.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5167.13039.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTc2NCw2ICs3NjQsMTUgQEAKIG5ldHZzY19jaGFubmVsX3JvbGx1cChzdHJ1Y3QgaHZf ZGV2aWNlICpkZXZpY2VfY3R4KQogewogCXN0cnVjdCBobl9zb2Z0YyAqc2MgPSBkZXZpY2VfZ2V0 X3NvZnRjKGRldmljZV9jdHgtPmRldmljZSk7CisjaWYgZGVmaW5lZChJTkVUKSB8fCBkZWZpbmVk KElORVQ2KQorCXN0cnVjdCBscm9fY3RybCAqbHJvID0gJnNjLT5obl9scm87CisJc3RydWN0IGxy b19lbnRyeSAqcXVldWVkOworCisJd2hpbGUgKChxdWV1ZWQgPSBTTElTVF9GSVJTVCgmbHJvLT5s cm9fYWN0aXZlKSkgIT0gTlVMTCkgeworCQlTTElTVF9SRU1PVkVfSEVBRCgmbHJvLT5scm9fYWN0 aXZlLCBuZXh0KTsKKwkJdGNwX2xyb19mbHVzaChscm8sIHF1ZXVlZCk7CisJfQorI2VuZGlmCiAK IAlpZiAoIXNjLT5obl90eGVvZikKIAkJcmV0dXJuOwpAQCAtMTMzOCwxOCArMTM0Nyw4IEBACiB9 CiAKIHZvaWQKLW5ldHZzY19yZWN2X3JvbGx1cChzdHJ1Y3QgaHZfZGV2aWNlICpkZXZpY2VfY3R4 KQorbmV0dnNjX3JlY3Zfcm9sbHVwKHN0cnVjdCBodl9kZXZpY2UgKmRldmljZV9jdHggX191bnVz ZWQpCiB7Ci0jaWYgZGVmaW5lZChJTkVUKSB8fCBkZWZpbmVkKElORVQ2KQotCWhuX3NvZnRjX3Qg KnNjID0gZGV2aWNlX2dldF9zb2Z0YyhkZXZpY2VfY3R4LT5kZXZpY2UpOwotCXN0cnVjdCBscm9f Y3RybCAqbHJvID0gJnNjLT5obl9scm87Ci0Jc3RydWN0IGxyb19lbnRyeSAqcXVldWVkOwotCi0J d2hpbGUgKChxdWV1ZWQgPSBTTElTVF9GSVJTVCgmbHJvLT5scm9fYWN0aXZlKSkgIT0gTlVMTCkg ewotCQlTTElTVF9SRU1PVkVfSEVBRCgmbHJvLT5scm9fYWN0aXZlLCBuZXh0KTsKLQkJdGNwX2xy b19mbHVzaChscm8sIHF1ZXVlZCk7Ci0JfQotI2VuZGlmCiB9CiAKIC8qCgo= --b1_c9198d81d09fe437aa4521f8bf434026-- From owner-freebsd-net@freebsd.org Fri Feb 5 05:51:23 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77A18A9A455 for ; Fri, 5 Feb 2016 05:51:23 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 5833C1EC6 for ; Fri, 5 Feb 2016 05:51:23 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 57341107298; Fri, 5 Feb 2016 05:51:23 +0000 (UTC) Date: Fri, 5 Feb 2016 05:51:23 +0000 To: freebsd-net@freebsd.org From: Phabricator Reply-to: D5175+325+b48f3d05eeb00f73@reviews.freebsd.org Subject: [Differential] [Closed] D5175: hyperv/hn: Add an option to always do transmission scheduling Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5175: hyperv/hn: Add an option to always do transmission scheduling X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: OGFmNjgzYWFmZmVhMjQ0MTIwYzQ1OWJjZDg2IFa0OFs= MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_eb1832b700c57f22d641be7e0f253c8a" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 05:51:23 -0000 --b1_eb1832b700c57f22d641be7e0f253c8a Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS295306: hyperv/hn: Add an option to always do transmission scheduling (authored by sephe). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D5175?vs=12968&id=13040#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D5175?vs=12968&id=13040 REVISION DETAIL https://reviews.freebsd.org/D5175 AFFECTED FILES head/sys/dev/hyperv/netvsc/hv_net_vsc.h head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c CHANGE DETAILS diff --git a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c --- a/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c +++ b/head/sys/dev/hyperv/netvsc/hv_netvsc_drv_freebsd.c @@ -534,6 +534,10 @@ SYSCTL_ADD_INT(ctx, child, OID_AUTO, "direct_tx_size", CTLFLAG_RW, &sc->hn_direct_tx_size, 0, "Size of the packet for direct transmission"); + SYSCTL_ADD_INT(ctx, child, OID_AUTO, "sched_tx", + CTLFLAG_RW, &sc->hn_sched_tx, 0, + "Always schedule transmission " + "instead of doing direct transmission"); if (unit == 0) { struct sysctl_ctx_list *dc_ctx; @@ -1602,26 +1606,31 @@ static void hn_start(struct ifnet *ifp) { - hn_softc_t *sc; + struct hn_softc *sc = ifp->if_softc; + + if (sc->hn_sched_tx) + goto do_sched; - sc = ifp->if_softc; if (NV_TRYLOCK(sc)) { int sched; sched = hn_start_locked(ifp, sc->hn_direct_tx_size); NV_UNLOCK(sc); if (!sched) return; } +do_sched: taskqueue_enqueue_fast(sc->hn_tx_taskq, &sc->hn_start_task); } static void hn_start_txeof(struct ifnet *ifp) { - hn_softc_t *sc; + struct hn_softc *sc = ifp->if_softc; + + if (sc->hn_sched_tx) + goto do_sched; - sc = ifp->if_softc; if (NV_TRYLOCK(sc)) { int sched; @@ -1633,6 +1642,7 @@ &sc->hn_start_task); } } else { +do_sched: /* * Release the OACTIVE earlier, with the hope, that * others could catch up. The task will clear the diff --git a/head/sys/dev/hyperv/netvsc/hv_net_vsc.h b/head/sys/dev/hyperv/netvsc/hv_net_vsc.h --- a/head/sys/dev/hyperv/netvsc/hv_net_vsc.h +++ b/head/sys/dev/hyperv/netvsc/hv_net_vsc.h @@ -1023,6 +1023,7 @@ int hn_txdesc_avail; int hn_txeof; + int hn_sched_tx; int hn_direct_tx_size; struct taskqueue *hn_tx_taskq; struct task hn_start_task; EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, howard0su_gmail.com, adrian, network, honzhan_microsoft.com Cc: freebsd-virtualization-list, freebsd-net-list --b1_eb1832b700c57f22d641be7e0f253c8a Content-Type: text/x-patch; charset=utf-8; name="D5175.13040.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D5175.13040.patch" ZGlmZiAtLWdpdCBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25ldHZzY19kcnZfZnJl ZWJzZC5jIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKLS0tIGEvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKKysrIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0dnNjX2Rydl9mcmVlYnNk LmMKQEAgLTUzNCw2ICs1MzQsMTAgQEAKIAlTWVNDVExfQUREX0lOVChjdHgsIGNoaWxkLCBPSURf QVVUTywgImRpcmVjdF90eF9zaXplIiwKIAkgICAgQ1RMRkxBR19SVywgJnNjLT5obl9kaXJlY3Rf dHhfc2l6ZSwgMCwKIAkgICAgIlNpemUgb2YgdGhlIHBhY2tldCBmb3IgZGlyZWN0IHRyYW5zbWlz c2lvbiIpOworCVNZU0NUTF9BRERfSU5UKGN0eCwgY2hpbGQsIE9JRF9BVVRPLCAic2NoZWRfdHgi LAorCSAgICBDVExGTEFHX1JXLCAmc2MtPmhuX3NjaGVkX3R4LCAwLAorCSAgICAiQWx3YXlzIHNj aGVkdWxlIHRyYW5zbWlzc2lvbiAiCisJICAgICJpbnN0ZWFkIG9mIGRvaW5nIGRpcmVjdCB0cmFu c21pc3Npb24iKTsKIAogCWlmICh1bml0ID09IDApIHsKIAkJc3RydWN0IHN5c2N0bF9jdHhfbGlz dCAqZGNfY3R4OwpAQCAtMTYwMiwyNiArMTYwNiwzMSBAQAogc3RhdGljIHZvaWQKIGhuX3N0YXJ0 KHN0cnVjdCBpZm5ldCAqaWZwKQogewotCWhuX3NvZnRjX3QgKnNjOworCXN0cnVjdCBobl9zb2Z0 YyAqc2MgPSBpZnAtPmlmX3NvZnRjOworCisJaWYgKHNjLT5obl9zY2hlZF90eCkKKwkJZ290byBk b19zY2hlZDsKIAotCXNjID0gaWZwLT5pZl9zb2Z0YzsKIAlpZiAoTlZfVFJZTE9DSyhzYykpIHsK IAkJaW50IHNjaGVkOwogCiAJCXNjaGVkID0gaG5fc3RhcnRfbG9ja2VkKGlmcCwgc2MtPmhuX2Rp cmVjdF90eF9zaXplKTsKIAkJTlZfVU5MT0NLKHNjKTsKIAkJaWYgKCFzY2hlZCkKIAkJCXJldHVy bjsKIAl9Citkb19zY2hlZDoKIAl0YXNrcXVldWVfZW5xdWV1ZV9mYXN0KHNjLT5obl90eF90YXNr cSwgJnNjLT5obl9zdGFydF90YXNrKTsKIH0KIAogc3RhdGljIHZvaWQKIGhuX3N0YXJ0X3R4ZW9m KHN0cnVjdCBpZm5ldCAqaWZwKQogewotCWhuX3NvZnRjX3QgKnNjOworCXN0cnVjdCBobl9zb2Z0 YyAqc2MgPSBpZnAtPmlmX3NvZnRjOworCisJaWYgKHNjLT5obl9zY2hlZF90eCkKKwkJZ290byBk b19zY2hlZDsKIAotCXNjID0gaWZwLT5pZl9zb2Z0YzsKIAlpZiAoTlZfVFJZTE9DSyhzYykpIHsK IAkJaW50IHNjaGVkOwogCkBAIC0xNjMzLDYgKzE2NDIsNyBAQAogCQkJICAgICZzYy0+aG5fc3Rh cnRfdGFzayk7CiAJCX0KIAl9IGVsc2UgeworZG9fc2NoZWQ6CiAJCS8qCiAJCSAqIFJlbGVhc2Ug dGhlIE9BQ1RJVkUgZWFybGllciwgd2l0aCB0aGUgaG9wZSwgdGhhdAogCQkgKiBvdGhlcnMgY291 bGQgY2F0Y2ggdXAuICBUaGUgdGFzayB3aWxsIGNsZWFyIHRoZQpkaWZmIC0tZ2l0IGEvaGVhZC9z eXMvZGV2L2h5cGVydi9uZXR2c2MvaHZfbmV0X3ZzYy5oIGIvaGVhZC9zeXMvZGV2L2h5cGVydi9u ZXR2c2MvaHZfbmV0X3ZzYy5oCi0tLSBhL2hlYWQvc3lzL2Rldi9oeXBlcnYvbmV0dnNjL2h2X25l dF92c2MuaAorKysgYi9oZWFkL3N5cy9kZXYvaHlwZXJ2L25ldHZzYy9odl9uZXRfdnNjLmgKQEAg LTEwMjMsNiArMTAyMyw3IEBACiAJaW50CQlobl90eGRlc2NfYXZhaWw7CiAJaW50CQlobl90eGVv ZjsKIAorCWludAkJaG5fc2NoZWRfdHg7CiAJaW50CQlobl9kaXJlY3RfdHhfc2l6ZTsKIAlzdHJ1 Y3QgdGFza3F1ZXVlICpobl90eF90YXNrcTsKIAlzdHJ1Y3QgdGFzawlobl9zdGFydF90YXNrOwoK --b1_eb1832b700c57f22d641be7e0f253c8a-- From owner-freebsd-net@freebsd.org Fri Feb 5 07:53:34 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 313009D936A for ; Fri, 5 Feb 2016 07:53:34 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 1D37F1E2C for ; Fri, 5 Feb 2016 07:53:34 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 197E21073BD; Fri, 5 Feb 2016 07:53:34 +0000 (UTC) Date: Fri, 5 Feb 2016 07:53:34 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Reply-to: D4825+325+15e38f1ea6d5e28b@reviews.freebsd.org Subject: [Differential] [Updated] D4825: tcp/lro: Add network driver configurable LRO entry depth Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , Thread-Topic: D4825: tcp/lro: Add network driver configurable LRO entry depth X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NmVkZTllNGEwNGViMzMwZDAzYzdjMGY0Y2Y4IFa0VP4= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 07:53:34 -0000 hselasky added a comment. FYI https://reviews.freebsd.org/D1761 might be related to this one. Should you check that "lc->lro_hiwat" is greater or equal to "lc->ifp->if_mtu" ? --HPS REVISION DETAIL https://reviews.freebsd.org/D4825 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, transport, adrian, delphij, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, glebius Cc: hselasky, np, freebsd-net-list From owner-freebsd-net@freebsd.org Fri Feb 5 07:55:22 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD35E9D9422 for ; Fri, 5 Feb 2016 07:55:22 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 998E51F1C for ; Fri, 5 Feb 2016 07:55:22 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 960D71074BF; Fri, 5 Feb 2016 07:55:22 +0000 (UTC) Date: Fri, 5 Feb 2016 07:55:22 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D4825+325+15e38f1ea6d5e28b@reviews.freebsd.org Subject: [Differential] [Commented On] D4825: tcp/lro: Add network driver configurable LRO entry depth Message-ID: <6c36f266ce528c279fcc03757474360a@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D4825: tcp/lro: Add network driver configurable LRO entry depth X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NmVkZTllNGEwNGViMzMwZDAzYzdjMGY0Y2Y4IFa0VWo= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 07:55:22 -0000 sepherosa_gmail.com added a comment. In https://reviews.freebsd.org/D4825#110653, @hselasky wrote: > FYI > > https://reviews.freebsd.org/D1761 might be related to this one. > > Should you check that "lc->lro_hiwat" is greater or equal to "lc->ifp->if_mtu" ? > > --HPS I have discarded this one, please take a look at this: https://reviews.freebsd.org/D5185 Thanks, sephe REVISION DETAIL https://reviews.freebsd.org/D4825 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, transport, adrian, delphij, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, glebius Cc: hselasky, np, freebsd-net-list From owner-freebsd-net@freebsd.org Fri Feb 5 07:56:11 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC6D19D947C for ; Fri, 5 Feb 2016 07:56:11 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id C915E7D for ; Fri, 5 Feb 2016 07:56:11 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id BEF0310757C; Fri, 5 Feb 2016 07:56:11 +0000 (UTC) Date: Fri, 5 Feb 2016 07:56:11 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D4825+325+15e38f1ea6d5e28b@reviews.freebsd.org Subject: [Differential] [Abandoned] D4825: tcp/lro: Add network driver configurable LRO entry depth Message-ID: <68b51ce656c5947550f7ee91a397505d@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D4825: tcp/lro: Add network driver configurable LRO entry depth X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NmVkZTllNGEwNGViMzMwZDAzYzdjMGY0Y2Y4IFa0VZs= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 07:56:12 -0000 sepherosa_gmail.com abandoned this revision. sepherosa_gmail.com added a comment. Updated version at: https://reviews.freebsd.org/D5185 REVISION DETAIL https://reviews.freebsd.org/D4825 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, transport, adrian, delphij, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, glebius Cc: hselasky, np, freebsd-net-list From owner-freebsd-net@freebsd.org Fri Feb 5 11:09:32 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C47C6A9CD5A for ; Fri, 5 Feb 2016 11:09:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B56621A11 for ; Fri, 5 Feb 2016 11:09:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15B9Wac080634 for ; Fri, 5 Feb 2016 11:09:32 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206933] MFC of 264915 to 10 STABLE Date: Fri, 05 Feb 2016 11:09:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 11:09:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206933 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org CC| |glebius@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 11:10:20 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1550A9CE0A for ; Fri, 5 Feb 2016 11:10:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 924C91B31 for ; Fri, 5 Feb 2016 11:10:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15BAKKp082458 for ; Fri, 5 Feb 2016 11:10:20 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206934] MFC of commits r272695 and r288529 Date: Fri, 05 Feb 2016 11:10:20 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 11:10:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206934 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org CC| |ae@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 11:37:07 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 762B8A76854 for ; Fri, 5 Feb 2016 11:37:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 671C0BEB for ; Fri, 5 Feb 2016 11:37:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15Bb7r0062088 for ; Fri, 5 Feb 2016 11:37:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206934] MFC of commits r272695 and r288529 Date: Fri, 05 Feb 2016 11:37:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 11:37:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206934 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |zec@FreeBSD.org --- Comment #1 from Andrey V. Elsukov --- r288529 was merged as noted directly after 1 week. https://svnweb.freebsd.org/base?view=3Drevision&revision=3D289169 The second revision requires merging of new if_enc(4) from head/, because M= arco Zec noticed that this commit could introduce the problem with VIMAGE. I thi= nk it should be fixed in new if_enc(4). --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 15:38:00 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A7CFA77708 for ; Fri, 5 Feb 2016 15:38:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 519B124E for ; Fri, 5 Feb 2016 15:38:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15Fc0Cs098954 for ; Fri, 5 Feb 2016 15:38:00 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206932] Realtek 8111 card stops responding under high load in netmap mode Date: Fri, 05 Feb 2016 15:38:00 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: software-freebsd@interfasys.ch X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 15:38:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206932 --- Comment #2 from Olivier - interfaSys s=C3=A0rl --- Setting re0 to use a MTU of 9000 and the connection stays alive. Instead of timing out, the packet rate drops drastically once and things go back to normal.=20 The main difference in netstat is that the mbuf clusters are split between standard and jumbo frames ``` 768/2787/3555 mbufs in use (current/cache/total) 256/1524/1780/500200 mbuf clusters in use (current/cache/total/max) 256/1515 mbuf+clusters out of packet secondary zone in use (current/cache) 0/46/46/250099 4k (page size) jumbo clusters in use (current/cache/total/ma= x) 256/65/321/74103 9k jumbo clusters in use (current/cache/total/max) 0/0/0/41683 16k jumbo clusters in use (current/cache/total/max) 3008K/4513K/7521K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile ``` The rate in vmstat keeps rising, but that doesn't seem to be a problem ``` interrupt total rate irq16: sdhci_pci0 1 0 cpu0:timer 3008083 1113 irq256: ahci0 10125 3 irq257: xhci0 11363 4 irq258: hdac0 3 0 irq259: re0 13105929 4850 irq260: re1 101440 37 cpu2:timer 1095578 405 cpu1:timer 1083354 400 cpu3:timer 1123144 415 Total 19539020 7231 ``` --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 16:17:13 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B720CA9B823 for ; Fri, 5 Feb 2016 16:17:13 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id A247332D for ; Fri, 5 Feb 2016 16:17:13 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 9FBE56C03; Fri, 5 Feb 2016 16:17:13 +0000 (UTC) Date: Fri, 5 Feb 2016 16:17:13 +0000 To: freebsd-net@freebsd.org From: "gallatin (Andrew Gallatin)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Accepted] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFa0ywk= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 16:17:13 -0000 gallatin accepted this revision. gallatin added a comment. Thanks for addressing my concerns.. Does anybody else want to comment? REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, network, adrian, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, transport, gallatin Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Fri Feb 5 16:22:58 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40360A9BAA4 for ; Fri, 5 Feb 2016 16:22:58 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 2C000921 for ; Fri, 5 Feb 2016 16:22:58 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 29C586F5D; Fri, 5 Feb 2016 16:22:58 +0000 (UTC) Date: Fri, 5 Feb 2016 16:22:58 +0000 To: freebsd-net@freebsd.org From: "adrian (Adrian Chadd)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Accepted] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <3068f45692aeba96d3e69d6457667093@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFa0zGI= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 16:22:58 -0000 adrian accepted this revision. adrian added a comment. Nice! Thanks for all this work! REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, transport, gallatin, adrian, network Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Fri Feb 5 16:26:26 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 694C4A9BBC6 for ; Fri, 5 Feb 2016 16:26:26 +0000 (UTC) (envelope-from jim@ohlste.in) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F51AA80 for ; Fri, 5 Feb 2016 16:26:25 +0000 (UTC) (envelope-from jim@ohlste.in) Received: by mail-qg0-x230.google.com with SMTP id u30so71396511qge.1 for ; Fri, 05 Feb 2016 08:26:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ohlste-in.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:date:user-agent:mime-version :content-type:content-transfer-encoding; bh=8qx66NLJ1SJ+gqZeAsmOuDzxEdKMnUolMP3MhiS7vxQ=; b=qHwtCZgEkns46tf8cjobNF3h1eAOUPVPX20EmXpRQFSTL8vabz7RKC2UShQHKwiF5+ HvrmNd3Qcbph2nduSHBoecwj5eqQKJFAfvjILuZHNYyp0+2aPANLUToiMpjVNv+5fli6 /h0+fImK5QrHUepufOnJf+XbqgBmLOe4dB7p1it1Q/u4vCaQjMpo0ig2VRxXoO+pYJ7c 9lqynohNQ+vVHRYQD7ud7Mif2mqyN2jszJX7At+Bb+K1A6+d3jW1YLZlLvSXBRNKjGzb duHkB8FyP6WKhuchhA5GbEacQYZrHGpKgmSIr5NiFkpz645pEi9VVbPkulH1nkZg5xPs K27A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-type:content-transfer-encoding; bh=8qx66NLJ1SJ+gqZeAsmOuDzxEdKMnUolMP3MhiS7vxQ=; b=MKNVIy0EhMmI/AiK1rRUb7JpSYi2tfMtNLH6xc/8Jc4tsVJWVHGKvRMyazNNpyIUNW 0xntssiYQXCdO0dBx4DSbnWsNrwVlA6rq3Jo2tq88/x3tOj6DJaVrqsxFdZya7yWn+dl QE/62lA9UZjgGvXNq+Qu27G6ubeh40sYNnFEldJZqqy579cYVI6czsWUn2p1O0csNB+i mNfb4hS8JbO6BaSYChsQAX27wI2iY8hR//UiiPA+AjCB0bJF1B78xvUH8EKR7kzin5eX rmeFRn9zI6SwYhK0FZme18Op6aomzIb+yWUxN/tDJhanFUtd2pO2ryIzS3pazSohHcJW BdKQ== X-Gm-Message-State: AG10YOTQYjArrHGThRNfUu9HCNRQcRS31VJzjlESqYqBd4m93+NY2IeroJgfK5gZSs62mQ== X-Received: by 10.141.5.213 with SMTP id h204mr18044762qhd.48.1454689584952; Fri, 05 Feb 2016 08:26:24 -0800 (PST) Received: from [192.168.1.18] (pool-96-249-243-37.nrflva.fios.verizon.net. [96.249.243.37]) by smtp.googlemail.com with ESMTPSA id a197sm8080239qhc.25.2016.02.05.08.26.24 for (version=TLSv1/SSLv3 cipher=OTHER); Fri, 05 Feb 2016 08:26:24 -0800 (PST) To: freebsd-net@freebsd.org From: Jim Ohlstein Subject: asm.ca.com (formerly just-ping.com) and FreeBSD Message-ID: <56B4CD2E.5080009@ohlste.in> Date: Fri, 5 Feb 2016 11:26:22 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 16:26:26 -0000 Hello, I apologize if this has been asked or brought up here before but I couldn't find it, so here goes. I've used the above site to check ping times and troubleshoot connectivity issues in the past but hadn't used them in a bit. All of my FreeBSD servers (different data centers in different countries using different ethernet adapters) report 20-50% packet loss from this service from all of their IPv4 servers. IPv6 generally shows no packet loss. Using other ping services I see no packet loss, nor do I see them from one server to the next. Servers using Solaris or Linux do not have the same packet loss, even servers that are on the same /24 (or VM's in the same host!). Does anybody know the reason? More an intellectual curiosity than a functional issue as the problem does seem to be on their end. Thanks, and please Cc me on reply as I don't subscribe to this list. -- Jim Ohlstein "Never argue with a fool, onlookers may not be able to tell the difference." - Mark Twain From owner-freebsd-net@freebsd.org Fri Feb 5 16:47:20 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AFABA77579 for ; Fri, 5 Feb 2016 16:47:20 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 566BB1CD9 for ; Fri, 5 Feb 2016 16:47:20 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 5371F6414; Fri, 5 Feb 2016 16:47:20 +0000 (UTC) Date: Fri, 5 Feb 2016 16:47:20 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Commented On] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <5a4cc1a6242a256587eb59455cd3511e@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFa00hg= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 16:47:20 -0000 hselasky added inline comments. INLINE COMMENTS sys/netinet/tcp_lro.h:94 Might be worth set this limit to unsigned instead of unsigned short. Technically we can LRO more than 64KBytes worth of data! REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, transport, gallatin, adrian, network Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Fri Feb 5 16:52:56 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03E2CA777E7 for ; Fri, 5 Feb 2016 16:52:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E6CDD393 for ; Fri, 5 Feb 2016 16:52:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15GqtOl090850 for ; Fri, 5 Feb 2016 16:52:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206932] Realtek 8111 card stops responding under high load in netmap mode Date: Fri, 05 Feb 2016 16:52:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: software-freebsd@interfasys.ch X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 16:52:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206932 --- Comment #3 from Olivier - interfaSys s=C3=A0rl --- I think this is logged when things start failing ``` 231.020147 [2925] netmap_transmit re0 full hwcur 0 hwtail 0 qlen = 255 len 42 m 0xfffff80052005400 235.997171 [2925] netmap_transmit re0 full hwcur 0 hwtail 0 qlen = 255 len 42 m 0xfffff80023ec9c00 240.989245 [2925] netmap_transmit re0 full hwcur 0 hwtail 0 qlen = 255 len 42 m 0xfffff800521ad500 247.887586 [2925] netmap_transmit re0 full hwcur 0 hwtail 0 qlen = 255 len 42 m 0xfffff80023da9b00 253.069781 [2925] netmap_transmit re0 full hwcur 0 hwtail 0 qlen = 255 len 42 m 0xfffff80023ec7700 258.110746 [2925] netmap_transmit re0 full hwcur 0 hwtail 0 qlen = 255 len 42 m 0xfffff800521ade00 263.188076 [2925] netmap_transmit re0 full hwcur 0 hwtail 0 qlen = 255 len 42 m 0xfffff800237d6900 ``` --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 17:14:44 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53659A77EB2 for ; Fri, 5 Feb 2016 17:14:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 44B96209 for ; Fri, 5 Feb 2016 17:14:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u15HEiUN068754 for ; Fri, 5 Feb 2016 17:14:44 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206934] MFC of commits r272695 and r288529 Date: Fri, 05 Feb 2016 17:14:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mgrooms@shrew.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 17:14:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206934 --- Comment #2 from mgrooms@shrew.net --- Ahh. I see the MFC now. Thanks for checking and sorry to bother you with th= at. Is the new if_enc you refer to already in 10 STABLE? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Feb 5 21:29:25 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 789CDA9E663 for ; Fri, 5 Feb 2016 21:29:25 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32E6714C9 for ; Fri, 5 Feb 2016 21:29:25 +0000 (UTC) (envelope-from sunxiaoye07@gmail.com) Received: by mail-io0-x22f.google.com with SMTP id 9so143119354iom.1 for ; Fri, 05 Feb 2016 13:29:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jCatk2V5Kn4RKZU5+IBiKb8cycMZmfeMAhaKqfI0AtI=; b=lxl+fSu2xv4fUIHaHHgNQMYqk2uzHEE26vNsT/RW20hh8auIYZGQrLaCcLwmB9qj86 zAN0iRhu+as7obWZ3QA6WyS5pVTHgkByxiDG3n+cNCVvXIayJJJxdNI7sXSfgBFHdU7M h9yeF3MfGvWdJwqHOmGbcpfKcE82Oe4px7krX5fB4xWQn3bj0CCxqGHE8R4TD8ScYNkT 0HMysY2VXaEYHMp+nKiG8R0VNKPSz0tjhcaYta3R6wkMAXW6EbNFp1QrOo26fdMQcK4s qFsNk5MNHmSroPVIZohTrxHg1K+7Bh9R8iqu0B4KrAjKbh74S2xM85+vi1N5XsIK4Nf7 BIWQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rice-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jCatk2V5Kn4RKZU5+IBiKb8cycMZmfeMAhaKqfI0AtI=; b=Xd7yPO/u6bLcPUV9fWKoLTna8GPP2cgc/Qzq8AqfOG7ulca+bfBl8RndNre+f5SSHp gkQBlTMIWnbIBl8PYR8Mt3vAi5AiW6D4qIGlGLph9dFf4FDzgtrDN3JWRBjZLySUvegU 7KWDAu8UNqLYeOTmRXfyFJM6QZgliA180xgST2rzzsBAygi8vmG12OGDN3OqASprMjY1 skLG7ptp7llFdv1a/ULBus1xYAwt5EfK/gwDOaIH+SYgcbHch1V3HNnAEwfJ0PYFouw8 t+e1T0bclXj1FLvsqKWeQvAvj3JPrwqaE7+C1FcSOoCNk8WkmVRp/HvcWJjXfwXgPrLc cVQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=jCatk2V5Kn4RKZU5+IBiKb8cycMZmfeMAhaKqfI0AtI=; b=lAuh/SaIs3CFgBzVPz7GHprFZgmDaWpK++DCgUROH2AFMM26aE/jgqrmmNUpob7R3n 2d1Ep0Ox4dE2X0Gv3hVgf9CcfPzU0PS15Cn6w9SiUElylpqlBQ4RW5z2v4T2evnk2Otm eW1pFwXUD0WdCFCcvogd4vjQ0pnPVVyc3P9NmgFkTDu/TSTgmZ6j5lxZJkWeTQWILuvg MTYdkmkn91PfNNccgeJIkRubV9naBVX4w+NoZ+6Ow0d/K+pni/kz6BrrFmDU3tpn8ef9 UL/QsLsUnZJgyIVdEXDhXTqKPRJZN/g6ZrLePF5LoMEb2fzyFSFahgDcmMW26rkFbIj6 YbYA== X-Gm-Message-State: AG10YORxAeFUS5I3rzbGU2G9gWvKy67deQNTBNmCPKky5Igs1HyjQWbO1LDu5UnP9v/+KELV4Z/Gnj1aq3kEmA== MIME-Version: 1.0 X-Received: by 10.107.31.17 with SMTP id f17mr15691726iof.68.1454707764465; Fri, 05 Feb 2016 13:29:24 -0800 (PST) Sender: sunxiaoye07@gmail.com Received: by 10.36.98.82 with HTTP; Fri, 5 Feb 2016 13:29:24 -0800 (PST) In-Reply-To: References: Date: Fri, 5 Feb 2016 15:29:24 -0600 X-Google-Sender-Auth: 8ha9qO3LSN5Fb3Vhq2S8BTGEENc Message-ID: Subject: Re: swaping ring slots between NIC ring and Host ring does not always success From: Xiaoye Sun To: Victor Detoni Cc: Luigi Rizzo , Pavel Odintsov , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2016 21:29:25 -0000 Hi Victor, Thanks for the help. The command you provided worked perfectly for me. Hi Luigi, Thanks for your clarification. The experiment I did was NOT running on 3 nodes. They ran on two nodes. node 1 ran [1. sender]; node 2 ran [2. bridge.c] and [3. receiver (not using netmap)]; [2. bridge.c ] saw packets inorder. [3. receiver] saw packets out-of-order. I saw replication packets (even corrupted packets) in the setup I mentioned in my first email in this threads. I did not see replication packet in the sender-bridge-receiver setup. Let's solve the reorder problem first and then solve the replication packet problem. I also tried the experiment setup having 3 nodes running sender, bridge, receiver( both non-netmap based and netmap based XYZ) respectively. In the 3 nodes experiment, there is NO packet reorder no any node. The difference between the 2 nodes experiment and the 3 nodes experiment is that in the bridge of node 2 in the 2-nodes experiment the bridge interact with the host stack, while netmap does not interact with host stack in the 3-node experiment. This makes me make the conclusion that there might be some problem with the interaction between netmap and host stack. What is your opinion? Thanks! Xiaoye On Thu, Feb 4, 2016 at 6:26 PM, Victor Detoni wrote: > I'm sorry, I made mistake. To workaround this try `ip link set $IFACE > promisc on` > > > > On Thu, Feb 4, 2016 at 10:04 PM, Xiaoye Sun wrote: > >> Yes. all the interfaces are up. Are you able to get ARP request when the >> interfaces are down? >> >> >> On Thursday, February 4, 2016, Victor Detoni >> wrote: >> >>> Both interfaces are up? Like ifconfig... up >>> >>> I had this the same problem and I solve with commands above >>> >>> Em quinta-feira, 4 de fevereiro de 2016, Xiaoye Sun >>> escreveu: >>> >>>> Hi Luigi, >>>> >>>> Thanks for your explanation. >>>> >>>> I used three machines to do this experiment. They are directly >>>> connected. >>>> >>>> [(machine1) eth1]---[eth2 (machine2) eth3]---[eth4 (machine3)]. >>>> >>>> First, I tried to run bridge.c on machine2 using the command *bridge -i >>>> netmap:eth2 -i netmap:eth3*. (sender receiver or XYZ were not running on >>>> machine 1or3) >>>> >>>> For my understanding, in this setup, machine2 will be transparent to >>>> machine1&3 since it forwards packet from its eth2 to eth3 and vice versa >>>> without any modification to the packets. >>>> >>>> I tried to ping machine 3 from machine 1 using the command like *ping >>>> 10.11.10.3*. However, it still does not success. >>>> This is because that before machine1 sends ping message to machine3, it >>>> will first send a ARP request message to get the mac address of >>>> machine3. >>>> machine3 gets that ARP request, and send the reply back (I use tcpdump >>>> to >>>> verify that machine3 gets the ARP request and send out the ARP reply). >>>> However, machine1 does not get the ARP reply. >>>> >>>> I checked that the bridge can only forwarding packet in one direction at >>>> the same time. it gets the ARP request but doesn't see the ARP reply >>>> (*pkt_queued* always returns 0 for one nic...). >>>> >>>> This behavior looks very weird to me. Do you think there is a >>>> compatibility >>>> issues between netmap and the os I am using? Is there a verified linux >>>> distribution (also the version) that perfectly works well with netmap? >>>> >>>> The OS I use is 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) >>>> x86_64 GNU/Linux. >>>> Linux kernel version is *3.16.0-4-amd64* >>>> >>>> >>>> Thanks! >>>> Xiaoye >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Feb 3, 2016 at 2:12 AM, Luigi Rizzo wrote: >>>> >>>> > On Tue, Feb 2, 2016 at 10:48 PM, Xiaoye Sun >>>> wrote: >>>> > > >>>> > > >>>> > > On Mon, Feb 1, 2016 at 11:34 PM, Luigi Rizzo >>>> wrote: >>>> > >> >>>> > >> On Tue, Feb 2, 2016 at 6:23 AM, Xiaoye Sun >>>> wrote: >>>> > >> > Hi Luigi, >>>> > >> > >>>> > >> > I have to clarify about the *jumping issue* about the slot >>>> indexes. >>>> > >> > In the bridge.c program, the slot index never jumps and it >>>> increases >>>> > >> > sequentially. >>>> > >> > In the receiver.c program, the udp packet seq jumps and I showed >>>> the >>>> > >> > slot >>>> > >> > index that each udp packet uses. So the slot index jumps >>>> together with >>>> > >> > the >>>> > >> > udp seq (at the receiver program only). >>>> > >> >>>> > >> So let me understand, is the "slot" some information written >>>> > >> in the packet by bridge.c (referring to the rx or tx slot, >>>> > >> I am not sure) and then read and printed by receiver.c >>>> > >> (which gets the packet through recvfrom so there isn't >>>> > >> really any slot index) ? >>>> > >> >>>> > > It works in the other way: >>>> > > The bridge.c checks the seq numbers of the udp packets in netmap >>>> slots >>>> > (in >>>> > > nic rx ring) before the swap; then it records the seq number, slot >>>> > > number(both rx and tx (tx indexes were not shown in the previous >>>> email >>>> > since >>>> > > they all look correct)) and buf_idx (rx and tx). The bridge.c does >>>> not >>>> > > change anything in the buffer and it knows the slot and buf_idx >>>> that a >>>> > > packet uses. Please refer to the added code in *process_rings* >>>> function >>>> > > http://www.owlnet.rice.edu/~xs6/bridge.c >>>> > > The receiver.c checks the seq numbers only and print out the seq >>>> numbers >>>> > it >>>> > > receive sequentially. >>>> > > With these information, I manually match the seq number I got from >>>> > > receiver.c and the seq number I got from bridge.c. So we know what >>>> is the >>>> > > seq order the receiver sees and which slot a packet uses when >>>> bridge.c >>>> > swaps >>>> > > the buf_idxs. >>>> > > >>>> > >> Do you see any ordering inversion when the receiver >>>> > >> gets packets through the NETMAP API (e.g. using bridge.c >>>> > >> instead of receiver.c) ? >>>> > >> >>>> > > There is no ordering inversion seen by bridge.c (As I said in the >>>> > previous >>>> > > paragraph, the bridge.c checks the seq number and I did not see any >>>> order >>>> > > inversion in THIS simple experiment (In my multicast protocol >>>> (mentioned >>>> > in >>>> > > the first email), there is ordering inversion. But let us solve the >>>> > simple >>>> > > bridge.c's problem first. I think they are two relatively >>>> independent >>>> > > issues.)). >>>> > >>>> > Sorry there was a misunderstanding. >>>> > I wanted you to check the following setup: >>>> > >>>> > [1: send.c] ->- [2: bridge.c] ->- [3: XYZ] >>>> > >>>> > where in XYZ you replace your receiver.c with some >>>> > netmap-based receiver (it could be pkt-gen in rx mode, >>>> > or possibly even another instance of bridge.c where >>>> > you connect the output port to a vale switch so >>>> > traffic is dropped), and then in XYZ print the content >>>> > of the packets. >>>> > >>>> > From your previous report we know that node 2: sees packets >>>> > in order, and node 3: sees packets out of order. >>>> > However, if the problem were due to bridge.c sending >>>> > the old buffer and not the new one, you'd see not only >>>> > reordering but also replication of packets. >>>> > >>>> > The fact that you see only the reordering in 3: makes >>>> > me think that the problem is in that node, and it could >>>> > be the network stack in 3: that does something strange. >>>> > So if you can run something netmap based in 3: and make >>>> > sure there is only one queue to read from, we could >>>> > at least figure out what is going on. >>>> > >>>> > cheers >>>> > luigi >>>> > >>>> > >>>> > is that >>>> > > >>>> > >> >>>> > >> Are you using native netmap drivers or the emulated mode ? >>>> > >> You can check that by playing with the "admode" sysctl entry >>>> > >> (or sysfs on linux) - try setting to 1 and 2 and see if >>>> > >> the behaviour changes. >>>> > >> >>>> > >> dev.netmap.admode: 0 >>>> > >> Controls the use of native or emulated adapter mode. >>>> > >> 0 uses the best available option, >>>> > >> 1 forces native and fails if not available, >>>> > >> 2 forces emulated hence never fails. >>>> > >> >>>> > > I was using admode 0. I changed the admode to 1 and 2 using the >>>> command >>>> > like >>>> > > *echo 1 > /sys/module/netmap/parameters/admode* and restart the >>>> bridge >>>> > > program. The behavior keeps the same. >>>> > > >>>> > >> >>>> > >> cheers >>>> > >> luigi >>>> > >> >>>> > >> > >>>> > >> > There is really one ring (tx and rx) for NIC and one ring (tx >>>> and rx) >>>> > >> > for >>>> > >> > the host. >>>> > >> > I also doubt that there might be multiple tx rings for the host. >>>> It >>>> > >> > seems >>>> > >> > like that bridge program swap packet to multiple host rings and >>>> the >>>> > udp >>>> > >> > recv >>>> > >> > program drains packets from these rings. But this is not the case >>>> > here. >>>> > >> > >>>> > >> > The bridge program prints a line like this >>>> > >> > *515.277263 main [277] Ready to go, eth3 0x1/1 <-> eth3 0x0/1.* >>>> > >> > this is printed by the following line the original program >>>> > >> > *D("Ready to go, %s 0x%x/%d <-> %s 0x%x/%d.", pa->req.nr_name, >>>> > >> > pa->first_rx_ring, pa->req.nr_rx_rings, pb->req.nr_name, >>>> > >> > pb->first_rx_ring, >>>> > >> > pb->req.nr_rx_rings);* >>>> > >> > >>>> > >> > I think this shows that there is really one NIC ring and one HOST >>>> > ring. >>>> > >> > >>>> > >> > Is there another way to verify the number of ring that netmap >>>> has? >>>> > >> > >>>> > >> > Thanks! >>>> > >> > Xiaoye >>>> > >> > >>>> > >> > On Mon, Feb 1, 2016 at 10:48 PM, Luigi Rizzo >>> > >>>> > wrote: >>>> > >> >> >>>> > >> >> Hi, >>>> > >> >> there must be some wrong with your setting because >>>> > >> >> slot indexes must be sequential and in your case they >>>> > >> >> are not (see the jump from 295 to 474 and then >>>> > >> >> back from 485 to 296, and the numerous interleavings >>>> > >> >> that you are seeing later). >>>> > >> >> >>>> > >> >> I have no idea of the cause but typically this pattern >>>> > >> >> is what you see when there are multiple input rings and >>>> > >> >> not just one. >>>> > >> >> >>>> > >> >> Cheers >>>> > >> >> Luigi >>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> > >> >> On Tue, Feb 2, 2016 at 12:24 AM, Xiaoye Sun < >>>> Xiaoye.Sun@rice.edu> >>>> > >> >> wrote: >>>> > >> >> > Hi Luigi, >>>> > >> >> > >>>> > >> >> > Thanks for the detailed advice. >>>> > >> >> > >>>> > >> >> > With more detailed experiments, actually I found that the udp >>>> > >> >> > sender/receiver packet reorder issue *might* be irrelevant to >>>> the >>>> > >> >> > original >>>> > >> >> > issue I posted. However, I think we should solve the udp >>>> > >> >> > sender/receiver >>>> > >> >> > issue first. >>>> > >> >> > I run the experiment with more detailed log. Here is my >>>> findings. >>>> > >> >> > >>>> > >> >> > 1. I am running a netmap version available since about Oct >>>> 13rd >>>> > from >>>> > >> >> > github >>>> > >> >> > (https://github.com/luigirizzo/netmap). So I think this is >>>> not the >>>> > >> >> > one >>>> > >> >> > related to the buffer allocation issue. I tried to running the >>>> > newest >>>> > >> >> > version, however, that version causes problem when I exit the >>>> > bridge >>>> > >> >> > program >>>> > >> >> > (something like kernel error which make the os crash). >>>> > >> >> > >>>> > >> >> > 2 & 3. I changed the receiver.c & bridge.c so that I can get >>>> more >>>> > >> >> > information (more detailed log). >>>> > >> >> > The reorder happens multiple times (about 10 times) within a >>>> > second. >>>> > >> >> > Here is >>>> > >> >> > one example trace collected from the above two programs. >>>> > (remembering >>>> > >> >> > that >>>> > >> >> > we have udp sender running on one machine; netmap bridge and >>>> udp >>>> > >> >> > receiver >>>> > >> >> > are running on another machine). >>>> > >> >> > There is only one pair of rings each with 512 slots (511 slot >>>> > usable) >>>> > >> >> > on >>>> > >> >> > the >>>> > >> >> > receiver machine. >>>> > >> >> > >>>> > >> >> > =================== packet trace collected from receiver.c >>>> > >> >> > =================== >>>> > >> >> > ===== together with the slot and buf_idx of the corresponding >>>> > netmap >>>> > >> >> > ring >>>> > >> >> > slots ====== >>>> > >> >> > [seq] [slot] [buf_idx] >>>> > >> >> > 8208 294 1833 >>>> > >> >> > 8209 295 1834 >>>> > >> >> > 8388 474 2013 >>>> > >> >> > ... (packet received in order) >>>> > >> >> > 8398 484 2023 >>>> > >> >> > 8399 485 2024 >>>> > >> >> > 8210 296 1835 >>>> > >> >> > 8211 297 1836 >>>> > >> >> > ... (packet received in order) >>>> > >> >> > ... >>>> > >> >> > 8222 308 1847 >>>> > >> >> > 8400 486 2025 >>>> > >> >> > 8223 309 1848 >>>> > >> >> > 8401 487 2026 >>>> > >> >> > 8224 310 1849 >>>> > >> >> > 8402 488 2027 >>>> > >> >> > 8225 311 1850 >>>> > >> >> > 8403 489 2028 >>>> > >> >> > 8226 312 1851 >>>> > >> >> > 8404 450 2029 >>>> > >> >> > 8227 313 1852 >>>> > >> >> > 8228 314 1853 >>>> > >> >> > >>>> =================================================================== >>>> > >> >> > As we can see that the udp receiver got packet 8210 after it >>>> got >>>> > >> >> > 8399, >>>> > >> >> > which >>>> > >> >> > is the first reorder. Then, the receiver got 8211 to 8222 >>>> > >> >> > sequentially. >>>> > >> >> > Then >>>> > >> >> > it got packet from 8223-8227 and 8400-8404 interleaved. >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > ==================== event order seen by netmap bridge >>>> > >> >> > ================== >>>> > >> >> > get 8209 >>>> > >> >> > poll called >>>> > >> >> > get 8210 >>>> > >> >> > ... >>>> > >> >> > ... >>>> > >> >> > get 8228 >>>> > >> >> > poll called >>>> > >> >> > get 8229 >>>> > >> >> > ... >>>> > >> >> > ... >>>> > >> >> > get 8383 >>>> > >> >> > poll called >>>> > >> >> > get 8384 >>>> > >> >> > ... >>>> > >> >> > get 8387 >>>> > >> >> > poll called >>>> > >> >> > get 8388 >>>> > >> >> > ... >>>> > >> >> > get 8393 >>>> > >> >> > poll called >>>> > >> >> > get 8394 >>>> > >> >> > ... >>>> > >> >> > get 8399 >>>> > >> >> > poll called >>>> > >> >> > get 8400 >>>> > >> >> > ... >>>> > >> >> > get 8404 >>>> > >> >> > poll called >>>> > >> >> > get 8405 >>>> > >> >> > >>>> =================================================================== >>>> > >> >> > As we can see, from the event ordering see by the bridge.c, >>>> all the >>>> > >> >> > packets >>>> > >> >> > are receiver in order, which means the the reorder happens >>>> when the >>>> > >> >> > bridge >>>> > >> >> > code swap the buf_idx between the nic ring(slot) and the host >>>> > >> >> > ring(slot). >>>> > >> >> > The reordered seq usually right before or after the poll >>>> function >>>> > >> >> > call. >>>> > >> >> > >>>> > >> >> > Best, >>>> > >> >> > Xiaoye >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > >>>> > >> >> > On Fri, Jan 29, 2016 at 4:27 PM, Luigi Rizzo < >>>> rizzo@iet.unipi.it> >>>> > >> >> > wrote: >>>> > >> >> >> >>>> > >> >> >> On Fri, Jan 29, 2016 at 2:12 PM, Xiaoye Sun < >>>> Xiaoye.Sun@rice.edu> >>>> > >> >> >> wrote: >>>> > >> >> >> > Hi Luigi, >>>> > >> >> >> > >>>> > >> >> >> > Thanks for your advice. >>>> > >> >> >> > I forgot to mention that I use the command "ethtool -L eth1 >>>> > >> >> >> > combined >>>> > >> >> >> > 1" >>>> > >> >> >> > to >>>> > >> >> >> > set the number of rings of the nic to 1. The host also >>>> only has >>>> > >> >> >> > one >>>> > >> >> >> > ring. >>>> > >> >> >> > I understand the situation where the first tx ring is full >>>> so >>>> > the >>>> > >> >> >> > bridge >>>> > >> >> >> > will swap the packets to the second tx ring and then the >>>> > host/nic >>>> > >> >> >> > might >>>> > >> >> >> > drain either rings. But this is not the case in the >>>> experiment. >>>> > >> >> >> >>>> > >> >> >> ok good to know that. >>>> > >> >> >> >>>> > >> >> >> So if we have ruled out multiqueue and iommu, let's look at >>>> > >> >> >> the internal allocator and at bridge.c >>>> > >> >> >> >>>> > >> >> >> 1. are you running the most recent version of netmap ? >>>> > >> >> >> Some older version (probably 1-2 years ago) had a bug >>>> > >> >> >> in the buffer allocator and some buffers were allocated >>>> > >> >> >> twice. >>>> > >> >> >> >>>> > >> >> >> 2. can you tweak your receiver.c to report some more info >>>> > >> >> >> on how often you get out of sequence packets, how much >>>> > >> >> >> out of sequence they are ? >>>> > >> >> >> Also it would be useful to report gaps on the increasing >>>> side >>>> > >> >> >> (i.e. new_seq != old_seq +1 ) >>>> > >> >> >> >>>> > >> >> >> 3. can you tweak bridge.c so that it writes into the packet >>>> > >> >> >> the netmap buffer indexes and slots on the rx and tx side, >>>> > >> >> >> so when you detect a sequence error we can figure out >>>> > >> >> >> where it is happening. >>>> > >> >> >> Ideally you could also add the sequence number detection >>>> > >> >> >> code in bridge.c so we can check whether the errors appear >>>> > >> >> >> on the input or output sides. >>>> > >> >> >> >>>> > >> >> >> cheers >>>> > >> >> >> luigi >>>> > >> >> >> >>>> > >> >> > >>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> > >> >> -- >>>> > >> >> >>>> > >> >> >>>> > >>>> -----------------------------------------+------------------------------- >>>> > >> >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >>>> > >> >> dell'Informazione >>>> > >> >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>>> > >> >> TEL +39-050-2217533 . via Diotisalvi 2 >>>> > >> >> Mobile +39-338-6809875 . 56122 PISA (Italy) >>>> > >> >> >>>> > >> >> >>>> > >>>> -----------------------------------------+------------------------------- >>>> > >> >> >>>> > >> > >>>> > >> >>>> > >> >>>> > >> >>>> > >> -- >>>> > >> >>>> > >>>> -----------------------------------------+------------------------------- >>>> > >> Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >>>> > dell'Informazione >>>> > >> http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>>> > >> TEL +39-050-2217533 . via Diotisalvi 2 >>>> > >> Mobile +39-338-6809875 . 56122 PISA (Italy) >>>> > >> >>>> > >>>> -----------------------------------------+------------------------------- >>>> > >> >>>> > > >>>> > >>>> > >>>> > >>>> > -- >>>> > >>>> -----------------------------------------+------------------------------- >>>> > Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. >>>> dell'Informazione >>>> > http://www.iet.unipi.it/~luigi/ . Universita` di Pisa >>>> > TEL +39-050-2217533 . via Diotisalvi 2 >>>> > Mobile +39-338-6809875 . 56122 PISA (Italy) >>>> > >>>> -----------------------------------------+------------------------------- >>>> > >>>> > >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>>> >>> > From owner-freebsd-net@freebsd.org Sat Feb 6 01:12:31 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 769B4A9EEAA for ; Sat, 6 Feb 2016 01:12:31 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 626C0DC5 for ; Sat, 6 Feb 2016 01:12:31 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 5F82C6B88; Sat, 6 Feb 2016 01:12:31 +0000 (UTC) Date: Sat, 6 Feb 2016 01:12:31 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Commented On] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFa1SH8= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2016 01:12:31 -0000 sepherosa_gmail.com added inline comments. INLINE COMMENTS sys/netinet/tcp_lro.h:94 My intention here is too keep the size of lro_ctrl unchanged on amd64 (I think there is an implicit 4 bytes padding after lro_mbuf_max :). But I am fine to change them into unsigned int. Does anyone know any drawbacks to change these two fields into unsigned int? If not, I would change them into unsigned int after Chinese New Year :) REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, hselasky, np, transport, gallatin, adrian, network Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Sat Feb 6 08:07:13 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4AC7AA9F24D for ; Sat, 6 Feb 2016 08:07:13 +0000 (UTC) (envelope-from free@oneex.me) Received: from mail.oneex.me (mail.oneex.me [91.193.143.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 02830912 for ; Sat, 6 Feb 2016 08:07:11 +0000 (UTC) (envelope-from free@oneex.me) Received: from [192.168.0.110] (unknown [85.12.216.123]) by mail.oneex.me (Postfix) with ESMTPSA id 9A033C3F50; Sat, 6 Feb 2016 12:57:43 +0500 (YEKT) Authentication-Results: mail.oneex.me; dmarc=fail header.from=oneex.me Authentication-Results: mail.oneex.me; spf=pass smtp.mailfrom=free@oneex.me DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oneex.me; s=mail; t=1454745463; bh=B60Htbg7Y21jiZazMccq0lMRBBj+3sHCKTba/H3hZcM=; h=To:References:Subject:From:Cc:Date:In-Reply-To; b=DAIzTJ2KBTPXqimXTH9Oe8gttmjgQ4rT+GjzjXODmwHpyJyl86F1mJKNzKnjqyMdz eJpOjVnR+mVhedIZj1qljt82uzw1S3ZYRkyQyTXFDEXhhZSxFVlFrcOfGe5wz9C453 MG1Jrh1V8vH3xh9qvxupY/7f6rmdAJ5eIuuUwLuo= To: freebsd-net@freebsd.org References: Subject: Re: Problem with ipfw, in-kernel NAT and port redirection to jails From: Alexey Roslyakov Cc: wow@0x89.net Message-ID: <56B5A77B.2010108@oneex.me> Date: Sat, 6 Feb 2016 12:57:47 +0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2016 08:07:13 -0000 Hello. I have same problem when I'm trying redirect incoming traffic into the jailed web server. I repeated my installation few times on different releases - problem with redirected ports was here all time (except 9.3 - there was random result). As a temporary solution am using pf nat for redirect ports. My test configuration: /etc/rc.conf: ifconfig_vtnet0="inet 192.168.1.18/24" defaultrouter="192.168.1.1" cloned_interfaces="lo1" /etc/jail.conf: exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; exec.clean; j1 { path = /home/jail1; mount.devfs; host.hostname = j1; interface = "lo1"; ip4.addr = 10.8.0.1; persist; } rc.firewall: ipfw nat 1 config if vtnet0 redirect_port tcp 10.8.0.1:80 80 ipfw add 500 nat 1 ip from any to 192.168.1.18 in via vtnet0 ipfw add 600 nat 1 ip from 10.8.0.1 to any out via vtnet0 ipfw add allow ip from any to any pf.conf: ext_if = "vtnet0" int_if = "lo1" jail_net = $int_if:network nat on $ext_if from $jail_net to any -> ($ext_if) rdr pass on $ext_if inet proto tcp from any to ($ext_if:0) port 80 -> 10.8.0.1 port 80 In jail I'm try nginx, apache24 and nc as source for redirection. Test file was generated: dd if/dev/random of=tmp.raw bs=1M count=2 On 10.1 and 10.2 there is no big differences, when using ipfw nat we can get only part of file (I'm using curl on different machine: curl http://192.168.1.18/tmp.raw > /dev/null): with nginx: Received = 33045 with apache: Received = 33092 with nc: Received = 16384 and result seems to be very stable in numbers. On 9.3: nginx: random bytes received, has no successful downloads apache: random bytes received, sometimes download entire file nc: entire file received My virtual environment is proxmox 3. Maybe it's related to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137346 or just not properly configured ipfw nat? From owner-freebsd-net@freebsd.org Sat Feb 6 08:59:40 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2301A9E33E for ; Sat, 6 Feb 2016 08:59:40 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id AE6191EF1 for ; Sat, 6 Feb 2016 08:59:40 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id AB783653B; Sat, 6 Feb 2016 08:59:40 +0000 (UTC) Date: Sat, 6 Feb 2016 08:59:40 +0000 To: freebsd-net@freebsd.org From: "hselasky (Hans Petter Selasky)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Updated] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFa1tfw= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2016 08:59:40 -0000 hselasky added a comment. The size of lro_ctrl already changed when the statistics was made 64-bit. Just remember to bump the FreeBSD_version. Might not be possible to MFC. REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, np, transport, gallatin, adrian, network, hselasky Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Sat Feb 6 09:02:09 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5015AA9E779 for ; Sat, 6 Feb 2016 09:02:09 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.rbsd.freebsd.org (unknown [IPv6:2607:fc50:2000:101::1bb:73]) by mx1.freebsd.org (Postfix) with ESMTP id 3CD2F663 for ; Sat, 6 Feb 2016 09:02:09 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.rbsd.freebsd.org (Postfix, from userid 1346) id 396CD6738; Sat, 6 Feb 2016 09:02:09 +0000 (UTC) Date: Sat, 6 Feb 2016 09:02:09 +0000 To: freebsd-net@freebsd.org From: "sepherosa_gmail.com (Sepherosa Ziehau)" Reply-to: D5185+325+ec92cccb639ec0fa@reviews.freebsd.org Subject: [Differential] [Commented On] D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit Message-ID: <9041b159e79b49b22614d13388476c67@localhost.localdomain> X-Priority: 3 X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: Thread-Topic: D5185: tcp/lro: Allow network drivers to set the limit for TCP ACK/data segment aggregation limit X-Herald-Rules: <64> X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: Precedence: bulk In-Reply-To: References: Thread-Index: NTU0NmM0Mjk2NjdmNzVhNmM3MzlkMWQyNTdmIFa1tpE= MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2016 09:02:09 -0000 sepherosa_gmail.com added a comment. In https://reviews.freebsd.org/D5185#110952, @hselasky wrote: > The size of lro_ctrl already changed when the statistics was made 64-bit. Just remember to bump the FreeBSD_version. Might not be possible to MFC. OK, let's wait for others inputs (I will be away next week). If no objection comes, I will change the limits to unsigned int. REVISION DETAIL https://reviews.freebsd.org/D5185 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: sepherosa_gmail.com, delphij, royger, decui_microsoft.com, honzhan_microsoft.com, howard0su_gmail.com, np, transport, gallatin, adrian, network, hselasky Cc: freebsd-virtualization-list, freebsd-net-list From owner-freebsd-net@freebsd.org Sat Feb 6 20:47:05 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83CC9AA0E38 for ; Sat, 6 Feb 2016 20:47:05 +0000 (UTC) (envelope-from guyyur@gmail.com) Received: from mail-oi0-x229.google.com (mail-oi0-x229.google.com [IPv6:2607:f8b0:4003:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F8B7973; Sat, 6 Feb 2016 20:47:05 +0000 (UTC) (envelope-from guyyur@gmail.com) Received: by mail-oi0-x229.google.com with SMTP id s2so61467285oie.2; Sat, 06 Feb 2016 12:47:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ChBIuoyVofn5d0OyKtyEF+IaBQU/rT1gPO1e74GljZM=; b=KYSmigzJPW/kmT5LMOEe7IJkGg5E3/7D1Cc1MvvmDHWVQrMkCZOlXrXTfB5cJs8OPT ytbFrafYu85V/q2H4iLDnQZX/PmJa/sKd25/+hTEPUKcugbh+OYqag9LdoVpigRKZLU6 GiEvCbYhuGT+D0WSQAfHu5E0Dq95CDpTRWeQrVmNtAiqea5gfii2JvZHaNhyPCCNdP0k 3qMjYHVIDP/HYOGpmtm37a+Uc3/javMuut1mcNkei6b8cdYKwez8naW8Fgd9h42rrWby pLm4tqMcvgKvgEA5D0iJ+nnV1oZfeuZGwQUsJo66W/Id4V6/tfFTDHD4O2mqg5YgGUWR n6fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=ChBIuoyVofn5d0OyKtyEF+IaBQU/rT1gPO1e74GljZM=; b=LEWEtPGB8rxVZiiMQnBDyzfnl2yi8VFwJXTx0S/oqtt0W4/S4TY3o+ia+ql2coYnld MbfTX6hik6YS0n7knJxH9AGV9CkPZrxjz5ft/wuaO6XQPISfgsqHRvd9qLo09Zha5t+r q+8M3flwX442/Z6or2AHXVr+CbJ/eAluMYMA/ZFiPO5o86SgRL9wbDWl40hvF67ux/7H B2NsQJwTMSqM1ci/aspuyD9xuL5h2GxDRLuV8r+rLJr6Hq6sOXpGCZRy+ZFzA4k1E9h7 kecFaP6PMes/d8AuCYy0WW+lG8YthrSlwUl33BKl7QsCDkFLFckr0tk11O8Sah35qPX8 s8lA== X-Gm-Message-State: AG10YOTcQk5IvkjsmCpzZSGea1kIWeSOedvCoU684HxRTZE3qUs4TlZYhUZxYnIjgckF7guSOuGH/UUCR2T09w== MIME-Version: 1.0 X-Received: by 10.202.85.88 with SMTP id j85mr13214890oib.28.1454791623922; Sat, 06 Feb 2016 12:47:03 -0800 (PST) Received: by 10.76.34.202 with HTTP; Sat, 6 Feb 2016 12:47:03 -0800 (PST) Date: Sat, 6 Feb 2016 22:47:03 +0200 Message-ID: Subject: openvpn tunnel subnet route netif is lo0 instead of tun0 From: Guy Yur To: freebsd-net@freebsd.org, melifaro@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2016 20:47:05 -0000 Hi, Between r286965 and r294555 openvpn ipv4 route added for subnet topology on the server started being associated with lo0 instead of tun0. This causes routing problems for clients other than the first. Reverting r293159 solves the problem. With r293159 the RTF_GATEWAY flag is not removed before calling rtrequest1_fib. I added some prints and I see rib_lookup_info returns 0 and ss.ss_family is 0. Commands to replicate the issue manually: ifconfig tun1 create ifconfig tun1 192.168.170.1 192.168.170.2 mtu 1500 netmask 255.255.255.0 up route add -net 192.168.170.0 192.168.170.1 255.255.255.0 Bad route for 192.168.170.0/24 with r293159: # netstat -rnf inet | grep -e Destination -e 192.168.170 Destination Gateway Flags Netif Expire 192.168.170.0/24 192.168.170.1 UGS lo0 192.168.170.1 link#4 UHS lo0 192.168.170.2 link#4 UH tun1 Good route for 192.168.170.0/24 with r293159 reverted: # netstat -rnf inet | grep -e Destination -e 192.168.170 Destination Gateway Flags Netif Expire 192.168.170.0/24 192.168.170.1 UGS tun1 192.168.170.1 link#4 UHS lo0 192.168.170.2 link#4 UH tun1 -- Guy