From owner-freebsd-xen@freebsd.org Mon Feb 19 09:58:37 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75358F14FE9; Mon, 19 Feb 2018 09:58:37 +0000 (UTC) (envelope-from laurence.pawling@globalsign.com) Received: from APC01-PU1-obe.outbound.protection.outlook.com (mail-pu1apc01on0108.outbound.protection.outlook.com [104.47.126.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DC306AB59; Mon, 19 Feb 2018 09:58:35 +0000 (UTC) (envelope-from laurence.pawling@globalsign.com) Received: from HK2PR0302MB2545.apcprd03.prod.outlook.com (10.170.152.14) by HK2PR0302MB2420.apcprd03.prod.outlook.com (10.170.151.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.527.6; Mon, 19 Feb 2018 09:58:31 +0000 Received: from HK2PR0302MB2545.apcprd03.prod.outlook.com ([fe80::997e:7c67:c760:d087]) by HK2PR0302MB2545.apcprd03.prod.outlook.com ([fe80::997e:7c67:c760:d087%2]) with mapi id 15.20.0527.012; Mon, 19 Feb 2018 09:58:30 +0000 From: Laurence Pawling To: "freebsd-xen@freebsd.org" , "freebsd-virtualization@freebsd.org" , "freebsd-net@freebsd.org" CC: David King , Vlad Galu Subject: multi-vCPU networking issues as client OS under Xen Thread-Topic: multi-vCPU networking issues as client OS under Xen Thread-Index: AQHTqWgyLMPz8qfMa0GcAhXNzUcYuA== Date: Mon, 19 Feb 2018 09:58:30 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [89.197.152.162] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HK2PR0302MB2420; 7:Jy+0LdTqz0VYXfNf2pZl3s/C/jZ7SQsE14zFpVs2YO7URYRr3lv64eqTlJ5Ilz2qWeeh1PFJjOMxA8yU88jfPy5t0GuT+KTTCQUMPSbpbDixZq8p4UpVVLsHp5rShIaw3PA/AvOQSQCsM1P3t/73JmFpJjDHvh8zeiJzcNPjYu6Lb4PFwTjltzb/NqVRRMflzoXYMdFY88MVMSYH82N4ubFyuTttyrznO13sa9zgG3HZgUO3dbIa0rgIPPSEyMYV x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10019020)(6019001)(7966004)(376002)(366004)(39850400004)(39380400002)(396003)(346002)(269900001)(189003)(199004)(53754006)(3280700002)(102836004)(105586002)(99936001)(6506007)(68736007)(55236004)(316002)(26005)(6116002)(3846002)(3660700001)(110136005)(36756003)(186003)(54906003)(4326008)(450100002)(6306002)(54896002)(6512007)(53936002)(82746002)(6436002)(6486002)(107886003)(106356001)(2501003)(5250100002)(5660300001)(2906002)(25786009)(2900100001)(33656002)(97736004)(66066001)(8936002)(83716003)(86362001)(508600001)(8676002)(81166006)(81156014)(7736002)(99286004)(14454004)(2201001)(966005); DIR:OUT; SFP:1102; SCL:1; SRVR:HK2PR0302MB2420; H:HK2PR0302MB2545.apcprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; x-ms-office365-filtering-correlation-id: 2dd732c0-f776-41c5-28b3-08d5777f54b4 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(5600026)(4604075)(3008032)(2017052603307)(7153060)(49563074)(7193020); SRVR:HK2PR0302MB2420; x-ms-traffictypediagnostic: HK2PR0302MB2420: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(158342451672863)(21748063052155); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(102415395)(6040501)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231101)(944501161)(10201501046)(3002001)(6041288)(20161123562045)(20161123564045)(20161123560045)(20161123558120)(201703131423095)(201703011903075)(201702281528075)(20161123555045)(201703061421075)(6072148)(201708071742011); SRVR:HK2PR0302MB2420; BCL:0; PCL:0; RULEID:; SRVR:HK2PR0302MB2420; x-forefront-prvs: 0588B2BD96 received-spf: None (protection.outlook.com: globalsign.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: 6f5xtPZRz6fb2IJcO2+F9eGVS6+UNiCiVRflUTJOaoFdG9F1sDQ1zLt52Z4PY+zILxt63HYDDis3FzwZkscoueHEW4+vOk5k3jAcN77A4PM5+EPR+YDQSDGrwfbcKiLrmG2emJQ1F8zPLRgd1FJUv1d5pMlhw6qTGcPWI+IfL9I= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha256; boundary="B_3601879107_1679118586" MIME-Version: 1.0 X-OriginatorOrg: globalsign.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2dd732c0-f776-41c5-28b3-08d5777f54b4 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Feb 2018 09:58:30.7188 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8fff67c1-8281-4635-b62f-93106cb7a9a8 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HK2PR0302MB2420 X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Feb 2018 09:58:37 -0000 --B_3601879107_1679118586 Content-type: text/plain; charset="UTF-8" Content-transfer-encoding: quoted-printable Hi all, =20 I=E2=80=99m wondering if anyone here has seen this issue before, I=E2=80=99ve spent the= last couple of days troubleshooting: =20 Platform: Host: XenServer 7.0 running on 2 x E2660-v4, 256GB RAM Server VM: FreeBSD 11 (tested on 11.0-p15 and 11.1-p6), 2GB RAM (also teste= d with 32GB RAM), 1x50GB HDD, 1 x NIC, 2 or more vCPUs in any combination (2= sockets x 1 core, 1 socket x 2 cores, =E2=80=A6) Client VM: FreeBSD 11, any configuration of vCPUs, RAM and HDD. =20 Behaviour: Sporadic interruption of TCP sessions when utilising the above machine as a= =E2=80=9Cserver=E2=80=9D with =E2=80=9Cclients=E2=80=9D connecting. Looking into the communication = with pcap/Wireshark, you see a TCP Dup Ack sent from both ends, followed by = the client sending an RST packet, terminating the TCP session. We have also = seen evidence of the client sending a Keepalive packet, which is ACK=E2=80=99d by = the server before the RST is sent from the client end. =20 To recreate: On the above VM, perform a vanilla install of nginx: pkg install nginx service nginx onestart Then on a client VM (currently only tested with FreeBSD), run the following= (or similar): for i in {1..10000}; do if [ $(curl -s -o /dev/null -w "%{http_code}" http:= //10.2.122.71) !=3D 200 ] ; then echo "error"; fi; done When vCPUs=3D1 on the server, I get no errors, when vCPUs>1 I get errors repo= rted. The frequency of errors *seems* to be proportional to the number of vC= PUs, but they are sporadic with no clear periodicity or pattern, so that is = just anecdotal. Also, the problem seems by far the most prevalent when commu= nicating between two VMs on the same host, in the same VLAN. Xen still sends= packets via the switch rather than bridging internally between the interfac= es. Note that we have not had a chance to investigate the effect of different n= umbers of CPUs on the *client* end, however it does seem to be governed enti= rely by the server end. =20 We cannot recreate this issue using the same FreeBSD image and same configu= ration, but using KVM as a hypervisor. =20 Has anyone met this before? =20 Thanks, =20 Laurence --B_3601879107_1679118586 Content-type: application/pkcs7-signature; name="smime.p7s" Content-transfer-encoding: base64 Content-disposition: attachment; filename="smime.p7s" MIIP9AYJKoZIhvcNAQcCoIIP5TCCD+ECAQExDzANBglghkgBZQMEAgEFADALBgkqhkiG9w0B BwGggg21MIIFYjCCBEqgAwIBAgIMSX2V7ElwCIrABZuQMA0GCSqGSIb3DQEBCwUAMF0xCzAJ BgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMTMwMQYDVQQDEypHbG9iYWxT aWduIFBlcnNvbmFsU2lnbiAyIENBIC0gU0hBMjU2IC0gRzMwHhcNMTYwODI2MTQzNjA4WhcN MTkwODI3MTQzNjA4WjCBnDELMAkGA1UEBhMCR0IxDTALBgNVBAgTBEtlbnQxEjAQBgNVBAcT CU1haWRzdG9uZTEfMB0GA1UEChMWR01PIEdsb2JhbFNpZ24gTGltaXRlZDEZMBcGA1UEAxMQ TGF1cmVuY2UgUGF3bGluZzEuMCwGCSqGSIb3DQEJARYfbGF1cmVuY2UucGF3bGluZ0BnbG9i YWxzaWduLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALBcjCf6ajlJBRGN ptXjlTf3Wwv5TuzE6iW5j3YOxxGE2uRSoisepRd5pHkQ3ffF1u4Rjfc3aXcHvlmipxf3pA3e TrGJGrlLhvpnY2Zg7ZXDYHEqZIfgASxl/5EXeshGqVi9uIoCjHng+oyfXU0GWuKPgqh88oZN Wm45QoeH4t96TRdlY/dpFjK8ea6vFvZApvbum8bYJZGlJOtgPl0j8/uF1akM8SNxciDQfT39 2quhOd6LxK0LeX9qIe5lXjbhojmyb0IUf4HKZanSuFMer5OY9MnsFgMsUdLs/D5LWw2dsqcF YfQgGs2FEolsnBwmogblGogYxpqy8qIK2JZwJp0CAwEAAaOCAeAwggHcMA4GA1UdDwEB/wQE AwIFoDCBngYIKwYBBQUHAQEEgZEwgY4wTQYIKwYBBQUHMAKGQWh0dHA6Ly9zZWN1cmUuZ2xv YmFsc2lnbi5jb20vY2FjZXJ0L2dzcGVyc29uYWxzaWduMnNoYTJnM29jc3AuY3J0MD0GCCsG AQUFBzABhjFodHRwOi8vb2NzcDIuZ2xvYmFsc2lnbi5jb20vZ3NwZXJzb25hbHNpZ24yc2hh MmczME0GA1UdIARGMEQwQgYKKwYBBAGgMgEoCjA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3 dy5nbG9iYWxzaWduLmNvbS9yZXBvc2l0b3J5LzAJBgNVHRMEAjAAMEQGA1UdHwQ9MDswOaA3 oDWGM2h0dHA6Ly9jcmwuZ2xvYmFsc2lnbi5jb20vZ3NwZXJzb25hbHNpZ24yc2hhMmczLmNy bDAqBgNVHREEIzAhgR9sYXVyZW5jZS5wYXdsaW5nQGdsb2JhbHNpZ24uY29tMB0GA1UdJQQW MBQGCCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQU5SIdmwI3kfJf98Y87lFl62amcP0w HwYDVR0jBBgwFoAUaXKCYjFnlUSFd5GAxAQ2SZ17C2EwDQYJKoZIhvcNAQELBQADggEBACMg 3mWAnD3BBOGpqvrPNem5D3G0GMKlVWSsRax251QsJwogpkoFDgbyX4QtHmkooPRGNydJGwhV TjvWu5BMle9UuZbcDw6zno3/hLD7kDIUlNJOuU+IgRhWtZwp6ELxaSvIBkiDKPRmpue3P73X gMTq2VnDwUx5Sxk9FkZcyFNfZ1wvZGWA03sZAZXj1UkmBrsGZ/dOpfsLr1Z0N4AR2ZVx+H4C MyWzVGJzp5Lvqw54WDY4KL4XRmkw8SrdujgIA5SsoAl4wEmSl1GZiXdb88koxbQ7yME+roNa q86YT7L4n4cWSBsf5lQTodjeYLWN1mmlBGzBxcv2ceTcy541lbwwggToMIID0KADAgECAg5I G2oJE72AJMpdIvAU8zANBgkqhkiG9w0BAQsFADBMMSAwHgYDVQQLExdHbG9iYWxTaWduIFJv b3QgQ0EgLSBSMzETMBEGA1UEChMKR2xvYmFsU2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjAe Fw0xNjA2MTUwMDAwMDBaFw0yNDA2MTUwMDAwMDBaMF0xCzAJBgNVBAYTAkJFMRkwFwYDVQQK ExBHbG9iYWxTaWduIG52LXNhMTMwMQYDVQQDEypHbG9iYWxTaWduIFBlcnNvbmFsU2lnbiAy IENBIC0gU0hBMjU2IC0gRzMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC2lmiT Zf0sAexiow1Uv4vLpEORopqvsYPytW1v2fDq3M8We9cZ44QDhfnGH3CPH2xJxWkZHnCRODV+ AkhdOWsYRKZqpch09F31hD5jH3FEggg+7cmn9uluJkEpgMlJuv/ZdqCjuxv2ed9LKNNKe5xR wg9lUKGfqEqd5UYEjYNP3LLIPc+YIQmYTMDxj3qpPzcmZUfYZo9JJsLDQL5mPWz/Oq0pRvAT nOHy65mni8LTX1Btog5vxwaXOC9OoY5HArSDANik47pBB2Dl3Tda8gfBO6ecl2gut++pSDa8 6WmomapH6cf2UdL5sSy2xUm1mJ5TU9r7cvN8D/hxPtcD+mfDAgMBAAGjggG1MIIBsTAOBgNV HQ8BAf8EBAMCAQYwagYDVR0lBGMwYQYIKwYBBQUHAwIGCCsGAQUFBwMEBggrBgEFBQcDCQYK KwYBBAGCNxQCAgYKKwYBBAGCNwoDBAYJKwYBBAGCNxUGBgorBgEEAYI3CgMMBggrBgEFBQcD BwYIKwYBBQUHAxEwEgYDVR0TAQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQUaXKCYjFnlUSFd5GA xAQ2SZ17C2EwHwYDVR0jBBgwFoAUj/BLf6guRSSuTVD6Y5qL3uLdG7wwPgYIKwYBBQUHAQEE MjAwMC4GCCsGAQUFBzABhiJodHRwOi8vb2NzcDIuZ2xvYmFsc2lnbi5jb20vcm9vdHIzMDYG A1UdHwQvMC0wK6ApoCeGJWh0dHA6Ly9jcmwuZ2xvYmFsc2lnbi5jb20vcm9vdC1yMy5jcmww ZwYDVR0gBGAwXjALBgkrBgEEAaAyASgwDAYKKwYBBAGgMgEoCjBBBgkrBgEEAaAyAV8wNDAy BggrBgEFBQcCARYmaHR0cHM6Ly93d3cuZ2xvYmFsc2lnbi5jb20vcmVwb3NpdG9yeS8wDQYJ KoZIhvcNAQELBQADggEBAKidzTLMfGfiC1DXpVxwo2biJe/qtLZTMG6HEjdcM+LCKFbjk71F lfNY2BVxTPPkgokUvv6lzEe96wZUgj7mv7716oj1ecQoIguMevYOC+MqrkmaDpvCJ/JsthtV SgG2GeFoUHRYvBJFGE+u3l4bEzDnVSY0gKL+FIoEqweEYVIRolAAtnLgcvQRZ24TogtgCNfo FJdEO0cV5Q911vjp/kd/mvMhMYuyf0Eimg5WuBLzvw7gmd9RZCLb3IF+fvkdqOJ8W88L66qf 6txWe+ukuCws5gb/riRZf8VEfz6aFV76ZxJffGxSlzEr2r28tNW2uhT/IlSYQLg5wfl1Cxsc GqMwggNfMIICR6ADAgECAgsEAAAAAAEhWFMIojANBgkqhkiG9w0BAQsFADBMMSAwHgYDVQQL ExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMzETMBEGA1UEChMKR2xvYmFsU2lnbjETMBEGA1UE AxMKR2xvYmFsU2lnbjAeFw0wOTAzMTgxMDAwMDBaFw0yOTAzMTgxMDAwMDBaMEwxIDAeBgNV BAsTF0dsb2JhbFNpZ24gUm9vdCBDQSAtIFIzMRMwEQYDVQQKEwpHbG9iYWxTaWduMRMwEQYD VQQDEwpHbG9iYWxTaWduMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzCV2kHkG eCIW9cCDtoTKKJ79BXYRxa2IcvxGAkPHsoqdBF8kyy5L4WCCRuFSqwyBR3Bs3WTR6/Usow+C PQwrrpfXthSGEHm7OxOAd4wI4UnSamIvH176lmjfiSeVOJ8G1z7JyyZZDXPesMjpJg6DFcbv W4vSBGDKSaYo9mk79svIKJHlnYphVzesdBTcdOA67nIvLpz70Lu/9T0A4QYz6IIrrlOmOhZz jN1BDiA6wLSnoemyT5AuMmDpV8u5BJJoaOU4JmB1sp93/5EU764gSfytQBVI0QIxYRleuJfv rXe3ZJp6v1/BE++bYvsNbOBUaRapA9pu6YOTcXbGaYWCFwIDAQABo0IwQDAOBgNVHQ8BAf8E BAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUj/BLf6guRSSuTVD6Y5qL3uLdG7ww DQYJKoZIhvcNAQELBQADggEBAEtA28BQqv7IDO/3llRFSbuWAAlBrLMThoYoBzPKa+Z0uboA La6kCtP18fEPir9zZ0qDx0R7eOCvbmxvAymOMzlFw47kuVdsqvwSluxTxi3kJGy5lGP73FNo Z1Y+g7jPNSHDyWj+ztrCU6rMkIrp8F1GjJXdelgoGi8d3s0AN0GP7URt11Mol37zZwQeFdeK lrTT3kwnpEwbc3N29BeZwh96DuMtCK0KHCz/PKtVDg+Rfjbrw1dJvuEuLXxgi8NBURMjnc73 MmuUAaiZ5ywzHzo7JdKGQM47LIZ4yWEvFLru21Vv34TuBQlNvSjYcs7TYlBlHuuSl4Mx2bO1 ykdYP18xggIDMIIB/wIBATBtMF0xCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWdu IG52LXNhMTMwMQYDVQQDEypHbG9iYWxTaWduIFBlcnNvbmFsU2lnbiAyIENBIC0gU0hBMjU2 IC0gRzMCDEl9lexJcAiKwAWbkDANBglghkgBZQMEAgEFAKBpMC8GCSqGSIb3DQEJBDEiBCB1 hdN+15YgrzdHOYlGIDgH1Z393+n2atHqAdmV2odHvTAYBgkqhkiG9w0BCQMxCwYJKoZIhvcN AQcBMBwGCSqGSIb3DQEJBTEPFw0xODAyMTkwOTU4MjdaMA0GCSqGSIb3DQEBAQUABIIBAIYJ rroL/O9xwtOdTUhkbvXH+oYvFTtPzrlg1QvLkCQn+IC6BUF4840hpt7MrQOeU5vASAlo54t6 Y1GJiOj/kfMU3p+u0GCPf7LsiRWGfFjbpzX2nHaDisT7PGFDp9beMSXQIjZiO9ElElSbTpPF Dr4ofT21umsiuzZevY9GV2+CPEmt9ijuFCcDG1eEBppgEBlwjG+BkoYv2sEu7DiqKGf10znh Vk9aHngQtVuBi67P8LBCEsNKlGKW1Y1yH9hgT8G7CMM1jfqbCC/NkzVwba/F394z8LZOv5eI St5DPdqs5HzDpF3vjXuGz312Nx1IWV5VfAFYWflGgCVIch0H6mg= --B_3601879107_1679118586-- From owner-freebsd-xen@freebsd.org Mon Feb 19 10:07:19 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17CAAF15C53; Mon, 19 Feb 2018 10:07:19 +0000 (UTC) (envelope-from prvs=5814e79b2=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 584346B31C; Mon, 19 Feb 2018 10:07:18 +0000 (UTC) (envelope-from prvs=5814e79b2=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.46,534,1511827200"; d="scan'208";a="68150322" Date: Mon, 19 Feb 2018 10:05:58 +0000 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Laurence Pawling CC: "freebsd-xen@freebsd.org" , "freebsd-virtualization@freebsd.org" , "freebsd-net@freebsd.org" , David King , Vlad Galu Subject: Re: multi-vCPU networking issues as client OS under Xen Message-ID: <20180219100558.adgb6m5ukdfvxehp@MacBook-Pro-de-Roger.local> References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: NeoMutt/20171208 X-ClientProxiedBy: AMSPEX02CAS01.citrite.net (10.69.22.112) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Feb 2018 10:07:19 -0000 On Mon, Feb 19, 2018 at 09:58:30AM +0000, Laurence Pawling via freebsd-xen wrote: > Hi all, > > > > I’m wondering if anyone here has seen this issue before, I’ve spent the last couple of days troubleshooting: > > > > Platform: > > Host: XenServer 7.0 running on 2 x E2660-v4, 256GB RAM > > Server VM: FreeBSD 11 (tested on 11.0-p15 and 11.1-p6), 2GB RAM (also tested with 32GB RAM), 1x50GB HDD, 1 x NIC, 2 or more vCPUs in any combination (2 sockets x 1 core, 1 socket x 2 cores, …) > > Client VM: FreeBSD 11, any configuration of vCPUs, RAM and HDD. > > > > Behaviour: > > Sporadic interruption of TCP sessions when utilising the above machine as a “server” with “clients” connecting. Looking into the communication with pcap/Wireshark, you see a TCP Dup Ack sent from both ends, followed by the client sending an RST packet, terminating the TCP session. We have also seen evidence of the client sending a Keepalive packet, which is ACK’d by the server before the RST is sent from the client end. > > > > To recreate: > > On the above VM, perform a vanilla install of nginx: > > pkg install nginx > > service nginx onestart > > Then on a client VM (currently only tested with FreeBSD), run the following (or similar): > > for i in {1..10000}; do if [ $(curl -s -o /dev/null -w "%{http_code}" http://10.2.122.71) != 200 ] ; then echo "error"; fi; done > > When vCPUs=1 on the server, I get no errors, when vCPUs>1 I get errors reported. The frequency of errors *seems* to be proportional to the number of vCPUs, but they are sporadic with no clear periodicity or pattern, so that is just anecdotal. Also, the problem seems by far the most prevalent when communicating between two VMs on the same host, in the same VLAN. Xen still sends packets via the switch rather than bridging internally between the interfaces. When using >1 vCPUs can you set hw.xn.num_queues=1 on /boot/loader.conf and try to reproduce the issue? I'm afraid this is rather related to multiqueue (which is only used if >1 vCPUs). Thanks, Roger. From owner-freebsd-xen@freebsd.org Mon Feb 19 10:42:18 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7806BF18D1C; Mon, 19 Feb 2018 10:42:18 +0000 (UTC) (envelope-from laurence.pawling@globalsign.com) Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sg2apc01on0091.outbound.protection.outlook.com [104.47.125.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F5B46D962; Mon, 19 Feb 2018 10:42:16 +0000 (UTC) (envelope-from laurence.pawling@globalsign.com) Received: from HK2PR0302MB2545.apcprd03.prod.outlook.com (10.170.152.14) by HK2PR0302MB2579.apcprd03.prod.outlook.com (10.170.145.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.527.6; Mon, 19 Feb 2018 10:42:12 +0000 Received: from HK2PR0302MB2545.apcprd03.prod.outlook.com ([fe80::997e:7c67:c760:d087]) by HK2PR0302MB2545.apcprd03.prod.outlook.com ([fe80::997e:7c67:c760:d087%2]) with mapi id 15.20.0527.012; Mon, 19 Feb 2018 10:42:08 +0000 From: Laurence Pawling To: =?utf-8?B?Um9nZXIgUGF1IE1vbm7DqQ==?= CC: "freebsd-xen@freebsd.org" , "freebsd-virtualization@freebsd.org" , "freebsd-net@freebsd.org" , David King , Vlad Galu Subject: Re: multi-vCPU networking issues as client OS under Xen Thread-Topic: multi-vCPU networking issues as client OS under Xen Thread-Index: AQHTqWgyLMPz8qfMa0GcAhXNzUcYuKOrf6kAgAAKGgA= Date: Mon, 19 Feb 2018 10:42:08 +0000 Message-ID: References: <20180219100558.adgb6m5ukdfvxehp@MacBook-Pro-de-Roger.local> In-Reply-To: <20180219100558.adgb6m5ukdfvxehp@MacBook-Pro-de-Roger.local> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [89.197.152.162] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HK2PR0302MB2579; 7:mOYvIhWLrbCaDEWovAyP5/Xnh43SekpUIWaOUzDvFcFcy3bYI595F/JF2RBByJ1+FNW/jNfcrwpm9+ItxOJAaqgjRj9nctwa04gP6POwLf6CckDH884Ah59VFBkcmwIe4rV+Ir5Wjavk3IkRBYsLE0DIlEDnX0mALxxFnlZ5Q0uXUSGP1XpltMX0X2Clbf4TsbR/j0lm+D/wP/Hd31Uf+MggG/KA2vdp6XUNRJqf+kIMfL2NLTQ/6enz0fQPNfY1 x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10019020)(7966004)(396003)(366004)(346002)(376002)(39380400002)(39850400004)(189003)(199004)(8666007)(4326008)(25786009)(33656002)(97736004)(99936001)(54906003)(83716003)(107886003)(316002)(99286004)(86362001)(6246003)(106356001)(6486002)(105586002)(5250100002)(6436002)(53936002)(6512007)(66066001)(2900100001)(82746002)(14454004)(81166006)(68736007)(2906002)(5660300001)(81156014)(8676002)(3660700001)(26005)(6116002)(3846002)(229853002)(55236004)(305945005)(6506007)(102836004)(6346003)(508600001)(7736002)(2950100002)(36756003)(6916009)(186003)(76176011)(8936002)(3280700002); DIR:OUT; SFP:1102; SCL:1; SRVR:HK2PR0302MB2579; H:HK2PR0302MB2545.apcprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; x-ms-office365-filtering-correlation-id: 22d8cdaa-6b7f-4db5-5d65-08d577856d39 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(2017052603307)(7153060)(49563074)(7193020); SRVR:HK2PR0302MB2579; x-ms-traffictypediagnostic: HK2PR0302MB2579: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(102415395)(6040501)(2401047)(5005006)(8121501046)(3231101)(944501161)(93006095)(93001095)(3002001)(10201501046)(6041288)(20161123562045)(20161123560045)(201703131423095)(201703011903075)(201702281528075)(20161123555045)(201703061421075)(20161123564045)(20161123558120)(6072148)(201708071742011); SRVR:HK2PR0302MB2579; BCL:0; PCL:0; RULEID:; SRVR:HK2PR0302MB2579; x-forefront-prvs: 0588B2BD96 received-spf: None (protection.outlook.com: globalsign.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: EhUssBYBudn1yhuBRkD8kZaLgC2I0vav5iH1734j/l3c1m0F+Yuf5QCpT7xe5ON3hwJBEuvUNCCuTkp7+xSYvxMMLiQbWRHrOLCZGf8/mSLupxZPmsmKrGy/1QJTLjHWj1OIs38WRJI3DXFKpyr2hfz13Vq9eG5b0cWBJMe9pHY= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha256; boundary="B_3601881728_1046608566" MIME-Version: 1.0 X-OriginatorOrg: globalsign.com X-MS-Exchange-CrossTenant-Network-Message-Id: 22d8cdaa-6b7f-4db5-5d65-08d577856d39 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Feb 2018 10:42:08.9156 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8fff67c1-8281-4635-b62f-93106cb7a9a8 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HK2PR0302MB2579 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Feb 2018 10:42:18 -0000 --B_3601881728_1046608566 Content-type: text/plain; charset="UTF-8" Content-transfer-encoding: 7bit > When using >1 vCPUs can you set hw.xn.num_queues=1 on > /boot/loader.conf and try to reproduce the issue? > > I'm afraid this is rather related to multiqueue (which is only used > if >1 vCPUs). > > Thanks, Roger. Roger - thanks for your quick reply, this is confirmed. Setting hw.xn.num_queues=1 on the server VM when vCPUs > 1 prevents the issue. For reference, please can you comment on the performance impact of this? Laurence --B_3601881728_1046608566 Content-type: application/pkcs7-signature; name="smime.p7s" Content-transfer-encoding: base64 Content-disposition: attachment; filename="smime.p7s" MIIP9AYJKoZIhvcNAQcCoIIP5TCCD+ECAQExDzANBglghkgBZQMEAgEFADALBgkqhkiG9w0B BwGggg21MIIFYjCCBEqgAwIBAgIMSX2V7ElwCIrABZuQMA0GCSqGSIb3DQEBCwUAMF0xCzAJ BgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMTMwMQYDVQQDEypHbG9iYWxT aWduIFBlcnNvbmFsU2lnbiAyIENBIC0gU0hBMjU2IC0gRzMwHhcNMTYwODI2MTQzNjA4WhcN MTkwODI3MTQzNjA4WjCBnDELMAkGA1UEBhMCR0IxDTALBgNVBAgTBEtlbnQxEjAQBgNVBAcT CU1haWRzdG9uZTEfMB0GA1UEChMWR01PIEdsb2JhbFNpZ24gTGltaXRlZDEZMBcGA1UEAxMQ TGF1cmVuY2UgUGF3bGluZzEuMCwGCSqGSIb3DQEJARYfbGF1cmVuY2UucGF3bGluZ0BnbG9i YWxzaWduLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALBcjCf6ajlJBRGN ptXjlTf3Wwv5TuzE6iW5j3YOxxGE2uRSoisepRd5pHkQ3ffF1u4Rjfc3aXcHvlmipxf3pA3e TrGJGrlLhvpnY2Zg7ZXDYHEqZIfgASxl/5EXeshGqVi9uIoCjHng+oyfXU0GWuKPgqh88oZN Wm45QoeH4t96TRdlY/dpFjK8ea6vFvZApvbum8bYJZGlJOtgPl0j8/uF1akM8SNxciDQfT39 2quhOd6LxK0LeX9qIe5lXjbhojmyb0IUf4HKZanSuFMer5OY9MnsFgMsUdLs/D5LWw2dsqcF YfQgGs2FEolsnBwmogblGogYxpqy8qIK2JZwJp0CAwEAAaOCAeAwggHcMA4GA1UdDwEB/wQE AwIFoDCBngYIKwYBBQUHAQEEgZEwgY4wTQYIKwYBBQUHMAKGQWh0dHA6Ly9zZWN1cmUuZ2xv YmFsc2lnbi5jb20vY2FjZXJ0L2dzcGVyc29uYWxzaWduMnNoYTJnM29jc3AuY3J0MD0GCCsG AQUFBzABhjFodHRwOi8vb2NzcDIuZ2xvYmFsc2lnbi5jb20vZ3NwZXJzb25hbHNpZ24yc2hh MmczME0GA1UdIARGMEQwQgYKKwYBBAGgMgEoCjA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3 dy5nbG9iYWxzaWduLmNvbS9yZXBvc2l0b3J5LzAJBgNVHRMEAjAAMEQGA1UdHwQ9MDswOaA3 oDWGM2h0dHA6Ly9jcmwuZ2xvYmFsc2lnbi5jb20vZ3NwZXJzb25hbHNpZ24yc2hhMmczLmNy bDAqBgNVHREEIzAhgR9sYXVyZW5jZS5wYXdsaW5nQGdsb2JhbHNpZ24uY29tMB0GA1UdJQQW MBQGCCsGAQUFBwMCBggrBgEFBQcDBDAdBgNVHQ4EFgQU5SIdmwI3kfJf98Y87lFl62amcP0w HwYDVR0jBBgwFoAUaXKCYjFnlUSFd5GAxAQ2SZ17C2EwDQYJKoZIhvcNAQELBQADggEBACMg 3mWAnD3BBOGpqvrPNem5D3G0GMKlVWSsRax251QsJwogpkoFDgbyX4QtHmkooPRGNydJGwhV TjvWu5BMle9UuZbcDw6zno3/hLD7kDIUlNJOuU+IgRhWtZwp6ELxaSvIBkiDKPRmpue3P73X gMTq2VnDwUx5Sxk9FkZcyFNfZ1wvZGWA03sZAZXj1UkmBrsGZ/dOpfsLr1Z0N4AR2ZVx+H4C MyWzVGJzp5Lvqw54WDY4KL4XRmkw8SrdujgIA5SsoAl4wEmSl1GZiXdb88koxbQ7yME+roNa q86YT7L4n4cWSBsf5lQTodjeYLWN1mmlBGzBxcv2ceTcy541lbwwggToMIID0KADAgECAg5I G2oJE72AJMpdIvAU8zANBgkqhkiG9w0BAQsFADBMMSAwHgYDVQQLExdHbG9iYWxTaWduIFJv b3QgQ0EgLSBSMzETMBEGA1UEChMKR2xvYmFsU2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjAe Fw0xNjA2MTUwMDAwMDBaFw0yNDA2MTUwMDAwMDBaMF0xCzAJBgNVBAYTAkJFMRkwFwYDVQQK ExBHbG9iYWxTaWduIG52LXNhMTMwMQYDVQQDEypHbG9iYWxTaWduIFBlcnNvbmFsU2lnbiAy IENBIC0gU0hBMjU2IC0gRzMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC2lmiT Zf0sAexiow1Uv4vLpEORopqvsYPytW1v2fDq3M8We9cZ44QDhfnGH3CPH2xJxWkZHnCRODV+ AkhdOWsYRKZqpch09F31hD5jH3FEggg+7cmn9uluJkEpgMlJuv/ZdqCjuxv2ed9LKNNKe5xR wg9lUKGfqEqd5UYEjYNP3LLIPc+YIQmYTMDxj3qpPzcmZUfYZo9JJsLDQL5mPWz/Oq0pRvAT nOHy65mni8LTX1Btog5vxwaXOC9OoY5HArSDANik47pBB2Dl3Tda8gfBO6ecl2gut++pSDa8 6WmomapH6cf2UdL5sSy2xUm1mJ5TU9r7cvN8D/hxPtcD+mfDAgMBAAGjggG1MIIBsTAOBgNV HQ8BAf8EBAMCAQYwagYDVR0lBGMwYQYIKwYBBQUHAwIGCCsGAQUFBwMEBggrBgEFBQcDCQYK KwYBBAGCNxQCAgYKKwYBBAGCNwoDBAYJKwYBBAGCNxUGBgorBgEEAYI3CgMMBggrBgEFBQcD BwYIKwYBBQUHAxEwEgYDVR0TAQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQUaXKCYjFnlUSFd5GA xAQ2SZ17C2EwHwYDVR0jBBgwFoAUj/BLf6guRSSuTVD6Y5qL3uLdG7wwPgYIKwYBBQUHAQEE MjAwMC4GCCsGAQUFBzABhiJodHRwOi8vb2NzcDIuZ2xvYmFsc2lnbi5jb20vcm9vdHIzMDYG A1UdHwQvMC0wK6ApoCeGJWh0dHA6Ly9jcmwuZ2xvYmFsc2lnbi5jb20vcm9vdC1yMy5jcmww ZwYDVR0gBGAwXjALBgkrBgEEAaAyASgwDAYKKwYBBAGgMgEoCjBBBgkrBgEEAaAyAV8wNDAy BggrBgEFBQcCARYmaHR0cHM6Ly93d3cuZ2xvYmFsc2lnbi5jb20vcmVwb3NpdG9yeS8wDQYJ KoZIhvcNAQELBQADggEBAKidzTLMfGfiC1DXpVxwo2biJe/qtLZTMG6HEjdcM+LCKFbjk71F lfNY2BVxTPPkgokUvv6lzEe96wZUgj7mv7716oj1ecQoIguMevYOC+MqrkmaDpvCJ/JsthtV SgG2GeFoUHRYvBJFGE+u3l4bEzDnVSY0gKL+FIoEqweEYVIRolAAtnLgcvQRZ24TogtgCNfo FJdEO0cV5Q911vjp/kd/mvMhMYuyf0Eimg5WuBLzvw7gmd9RZCLb3IF+fvkdqOJ8W88L66qf 6txWe+ukuCws5gb/riRZf8VEfz6aFV76ZxJffGxSlzEr2r28tNW2uhT/IlSYQLg5wfl1Cxsc GqMwggNfMIICR6ADAgECAgsEAAAAAAEhWFMIojANBgkqhkiG9w0BAQsFADBMMSAwHgYDVQQL ExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMzETMBEGA1UEChMKR2xvYmFsU2lnbjETMBEGA1UE AxMKR2xvYmFsU2lnbjAeFw0wOTAzMTgxMDAwMDBaFw0yOTAzMTgxMDAwMDBaMEwxIDAeBgNV BAsTF0dsb2JhbFNpZ24gUm9vdCBDQSAtIFIzMRMwEQYDVQQKEwpHbG9iYWxTaWduMRMwEQYD VQQDEwpHbG9iYWxTaWduMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzCV2kHkG eCIW9cCDtoTKKJ79BXYRxa2IcvxGAkPHsoqdBF8kyy5L4WCCRuFSqwyBR3Bs3WTR6/Usow+C PQwrrpfXthSGEHm7OxOAd4wI4UnSamIvH176lmjfiSeVOJ8G1z7JyyZZDXPesMjpJg6DFcbv W4vSBGDKSaYo9mk79svIKJHlnYphVzesdBTcdOA67nIvLpz70Lu/9T0A4QYz6IIrrlOmOhZz jN1BDiA6wLSnoemyT5AuMmDpV8u5BJJoaOU4JmB1sp93/5EU764gSfytQBVI0QIxYRleuJfv rXe3ZJp6v1/BE++bYvsNbOBUaRapA9pu6YOTcXbGaYWCFwIDAQABo0IwQDAOBgNVHQ8BAf8E BAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUj/BLf6guRSSuTVD6Y5qL3uLdG7ww DQYJKoZIhvcNAQELBQADggEBAEtA28BQqv7IDO/3llRFSbuWAAlBrLMThoYoBzPKa+Z0uboA La6kCtP18fEPir9zZ0qDx0R7eOCvbmxvAymOMzlFw47kuVdsqvwSluxTxi3kJGy5lGP73FNo Z1Y+g7jPNSHDyWj+ztrCU6rMkIrp8F1GjJXdelgoGi8d3s0AN0GP7URt11Mol37zZwQeFdeK lrTT3kwnpEwbc3N29BeZwh96DuMtCK0KHCz/PKtVDg+Rfjbrw1dJvuEuLXxgi8NBURMjnc73 MmuUAaiZ5ywzHzo7JdKGQM47LIZ4yWEvFLru21Vv34TuBQlNvSjYcs7TYlBlHuuSl4Mx2bO1 ykdYP18xggIDMIIB/wIBATBtMF0xCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWdu IG52LXNhMTMwMQYDVQQDEypHbG9iYWxTaWduIFBlcnNvbmFsU2lnbiAyIENBIC0gU0hBMjU2 IC0gRzMCDEl9lexJcAiKwAWbkDANBglghkgBZQMEAgEFAKBpMC8GCSqGSIb3DQEJBDEiBCB8 aNzfpYozQ/xfkCCmjjSKys2lJJQRltQwtSN3tVOFSTAYBgkqhkiG9w0BCQMxCwYJKoZIhvcN AQcBMBwGCSqGSIb3DQEJBTEPFw0xODAyMTkxMDQyMDhaMA0GCSqGSIb3DQEBAQUABIIBADW/ nOtPmnBaLXY094SU+v12CNLetpAJYGhjPS7ExN8H7goqX8cajt6j+W9gaBH5yIu3eTe/HS89 +YVS4cPYUgZote8QKxRmxd7+3xomnEQKs1iXPp2QW5fyxi/3xlCJxy190rTlSVzJqstYMPT7 YVGoeYv4PirWpmnyTtGfD+nc1RbUH08w527rvfHds4rFICaFDh6pGLDOu++JdsGAc1Pb+DXq F0ENsOeY+eBszBIvj/ihb+iM2Go2px7oD1K78VaX2PjLeIoeMvXb58uihT+/b0iTpzzTMUXz yrvx+W9LTvRZo5T1+F27++FAXwCvif64zW8mUtQ5M8nCRgtjfeE= --B_3601881728_1046608566-- From owner-freebsd-xen@freebsd.org Mon Feb 19 11:02:28 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 093E8F1A800; Mon, 19 Feb 2018 11:02:28 +0000 (UTC) (envelope-from prvs=5814e79b2=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F9C96E80D; Mon, 19 Feb 2018 11:02:27 +0000 (UTC) (envelope-from prvs=5814e79b2=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.46,534,1511827200"; d="scan'208";a="68152684" Date: Mon, 19 Feb 2018 11:02:19 +0000 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Laurence Pawling CC: "freebsd-xen@freebsd.org" , "freebsd-virtualization@freebsd.org" , "freebsd-net@freebsd.org" , David King , Vlad Galu Subject: Re: multi-vCPU networking issues as client OS under Xen Message-ID: <20180219110219.r4yrgbc4yomb3gly@MacBook-Pro-de-Roger.local> References: <20180219100558.adgb6m5ukdfvxehp@MacBook-Pro-de-Roger.local> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20171208 X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Feb 2018 11:02:28 -0000 On Mon, Feb 19, 2018 at 10:42:08AM +0000, Laurence Pawling wrote: > > When using >1 vCPUs can you set hw.xn.num_queues=1 on > > /boot/loader.conf and try to reproduce the issue? > > > > I'm afraid this is rather related to multiqueue (which is only used > > if >1 vCPUs). > > > > Thanks, Roger. > > Roger - thanks for your quick reply, this is confirmed. Setting hw.xn.num_queues=1 on the server VM when vCPUs > 1 prevents the issue. I've also been told that in order to discard this being a XenServer specific issue you should execute the following on Dom0 and reboot the server: # xe-switch-network-backend bridge And then try to reproduce the issue again with >1 vCPUs (and of course removing the queue limit in loader.conf) > For reference, please can you comment on the performance impact of this? I'm afraid I don't have any numbers. Roger. From owner-freebsd-xen@freebsd.org Mon Feb 19 16:23:56 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53CB5F0D32B for ; Mon, 19 Feb 2018 16:23:56 +0000 (UTC) (envelope-from sarah.parker@proofdot.com) Received: from IND01-MA1-obe.outbound.protection.outlook.com (mail-ma1ind01on0130.outbound.protection.outlook.com [104.47.100.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 239197D473 for ; Mon, 19 Feb 2018 16:23:54 +0000 (UTC) (envelope-from sarah.parker@proofdot.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NETORGFT3391454.onmicrosoft.com; s=selector1-proofdot-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=RiuT/25OUuAOwj3VTMWWrUqyVmCIJmelw4Lw/AQp76U=; b=Bsk5H8gYJlpAZhI5qHIFxB52ULWJamVHZbv8PJtuS/060XGmX8Y4u+ZXOXBE0M6hrDLv0M8NFDvgpK+xRtbhEHobT8bNqJHGL5tazIrjMLEgNGZGd9dFf3j9hzDKVf7V4RYsXpvJcnJiswMJUkjVmK9brzRCKoYB/+NsBpDtkcg= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=sarah.parker@proofdot.com; Received: from SR (125.99.255.106) by PN1PR01MB0462.INDPRD01.PROD.OUTLOOK.COM (10.164.141.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.506.18; Mon, 19 Feb 2018 16:23:51 +0000 From: "sarah parker" To: Subject: Machine to Machine Users Contact List Date: Mon, 19 Feb 2018 21:50:40 +0530 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdOpnUXR/o8E9XORTHmzN6rpYQr6kQ== Content-Language: en-us X-Originating-IP: [125.99.255.106] X-ClientProxiedBy: PN1PR0101CA0040.INDPRD01.PROD.OUTLOOK.COM (10.174.150.26) To PN1PR01MB0462.INDPRD01.PROD.OUTLOOK.COM (10.164.141.23) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 180c5a2d-81e9-44c2-1479-08d577b529e4 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(7021125)(5600026)(4604075)(4534165)(7022125)(4603075)(4627221)(201702281549075)(7048125)(7024125)(7027125)(7028125)(7023125)(2017052603307)(7153060)(7193020); SRVR:PN1PR01MB0462; X-Microsoft-Exchange-Diagnostics: 1; PN1PR01MB0462; 3:2iq7eHGmVbWVxOADzzMa6/QaxYkv+YeYaU/pl8W9hmpfV4O8fNwF9uP/3n5f4uSiP3SQYEb0XbPaWF0u9YQdvgoF0m2NmcEwgHuPrdvOINKVJbzWs3X2wN6B79NoKR3r7qGBcaN4FbJ2X6CLn2v/J0qyV/6jE053tSiE4rzwIOxyy3pPz8WlHLLPhwea+Z0iV/FVZB5D01FgkiQADFbMn5esKLbpIYiBOn5i33Km0tFrdr9B/2u1cJ40hQezafos; 25:PtuqX46z8aTcmYTHhL1KjyEPYmum57QaNZhDe7NfKFFlWb3DdyhOnLDhwFDkRQIeEqPMDv3HC7maAWGBLY0kSWFAZlxFDU6Wi6oUEBQBu9lNX3C2EwHpLwMP6HQ0D/onJjznFyDYwq7icoRnhaZ8b/8x24NVzlXpI9AgOADhFroJwJlKnXjBY0O6x7RGspTK1MT5pUnLFF+B3WFCTL5Ahudf5LEZ33B7uwxKqj48YgghMNjr6p5OScj5ovpzBIPX2STCeKaeEmHeM4L12LqkWXIDm9X0421voCFn9p7dJn9pAC9ABq9YflPrlo8Kpcuch74P6EmGetEs8yWuP2hdHA==; 31:XvZCCjMGR8xWZr/Tq3kTGyWFwoSrOT9eDm5cpYT+L1YAllBGVWL+LqiTBxXCQA1M2fz3eMKOf+LRSdEDccczjyUgWciix4XtszZhTcMJGKiOppW+WUN7w3EZRIGHD7JT1nmav8nTzCoE9WH9ydWDkCg96Oyca6/ieIUfqs83q2AJGzpf1kkkM6sFAjk8n8xXatbHwwlL+9T1QW93KvSUSMhXoenM/aioO4C8SNHycRU= X-MS-TrafficTypeDiagnostic: PN1PR01MB0462: X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(28532068793085)(21748063052155); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(8121501046)(5005006)(3231101)(944501161)(93006095)(93001095)(10201501046)(3002001)(6041288)(20161123560045)(20161123564045)(20161123558120)(2016111802025)(20161123562045)(6072148)(6043046)(201708071742011); SRVR:PN1PR01MB0462; BCL:0; PCL:0; RULEID:; SRVR:PN1PR01MB0462; X-Microsoft-Exchange-Diagnostics: 1; PN1PR01MB0462; 4:DZMY9QqPFjpxCQYw03H9EtNnx2Mnp9Ivmvx+hdCvkk/WQYgDSu/IdT/qsd9z3r+jJp8y3qecFeAqWCA806ija3Yvba5e+lQtICsIuTTT/wodrm7kT2XIZ77gVkNOGedJf1Pmz3DLKsw/n9vHDAa/SrfIBnumG5eiqvULfv+1l3K2IwgulFpaI8G5bT44B9AfYny/e4hxAFGCYiEORojfjgCRJrbOTB8TAgrFsOBAqxuoslFclPrCHOXmfn6YaLj/zSS97z72jBM5WjGmnQGqiERjmhfVo08oy708UQkRsoayecdGqrxbLaQ1LkyKZ3XBwBwzKeuWWyMjM2D4xi8XDcwZoy+1RiBHg+whyrFlq38= X-Forefront-PRVS: 0588B2BD96 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(396003)(39840400004)(346002)(366004)(376002)(39380400002)(199004)(189003)(6496006)(52116002)(8936002)(7826002)(7736002)(68736007)(61296003)(316002)(626008)(478600001)(81156014)(81166006)(26005)(16586007)(8676002)(71636004)(16526019)(186003)(86362001)(66066001)(36756003)(7520500002)(223583001)(88246002)(6306002)(54896002)(260700001)(25786009)(270700001)(97736004)(6666003)(6916009)(5660300001)(2906002)(386003)(53936002)(790700001)(6116002)(3846002)(9476002)(2351001)(102836004)(106356001)(50226002)(105586002)(84326002)(6486002)(26710200004); DIR:OUT; SFP:1102; SCL:1; SRVR:PN1PR01MB0462; H:SR; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: proofdot.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; PN1PR01MB0462; 23:YPhnQxKg0WOq97XXG3ENzeY3zw/u2Yu37Buom7RFN?= =?us-ascii?Q?FHJC5t5cdK8sewsujzhMFTpZa9IEsTY4krNqzVvk0Jwtqgd4g84uWCV8V7it?= =?us-ascii?Q?mNoCutUiH42dXAP9iBrmVFImVRe2bTzWSh9dyPzVXLxs/VlnJ8vvyS6YDNpf?= =?us-ascii?Q?3cQ70BWqVHNilPxSZQwKoaow4xLepyOr0rba/HMjw4vAqnv3iJXlUPbtKpSD?= =?us-ascii?Q?SPvTvBAbCNDcvST1O6oGGg28V5rPE07K2SVcwQ+sb0YClu2DW/8p+18aq8gu?= =?us-ascii?Q?jVlPD/yNEBlsyAL3Qk5iDkr+WNoo76WCgfpL5mTygu10yXumXssMT3gQewW6?= =?us-ascii?Q?VNlhh3H7zjkVQB5ffh2l6oXToAg2i5GDxM29Ft5hidmoErV09OXASopNyR7z?= =?us-ascii?Q?/v69Y+WjyRCFQxqev8VTTGCnsrKhgxzj0fYJBcY3Q0CJ2pKhlXg4o2glSmYf?= =?us-ascii?Q?t60kPHkWBZfGo0XWMn24kRQGM3aCF34ie1J47D+SGqgS+wVeMt2DKm30Y58c?= =?us-ascii?Q?YDksbUGj74i8zHK080OC9XaRyogENXVr47Q66VByLGcmYMDsu7H1BJodbLgi?= =?us-ascii?Q?b78iq6/aVLpCJ8zp9D/YoEXbXl8ad29wHnx1hF/t9rD7Jf4pDUHA/nZI9Kho?= =?us-ascii?Q?+bTlPG6o2VSuKUtAOrZmcnCOvrTOLUCCxRWRgXn8DRR/YUZy/6zB/x46BEK0?= =?us-ascii?Q?ySyKqPaKh1S3kR1fUenwgRyMJ+Q+Uuem6t6Ge9z7DDwUQ851z3320cYXdCTg?= =?us-ascii?Q?dZdLhJImzPDecP52Q0B95pz1CX9AAuy/N6bIHQ148Io81kFuKWFrXWGVlNJN?= =?us-ascii?Q?ZK7CTW6mdihZZBE/kIc2qgwh3M9eY6UDB6RyyptOyI97EibjkFuvIdDZk3is?= =?us-ascii?Q?DwxvyMcHXRfvWuu7ET4RXHJKoY0VoYvnJXMIBejlRPkggEie/m4c1iPFCL63?= =?us-ascii?Q?KKPcC1u22fWnLCqpMkd+s7T3Y0oUe5OoKvgcjFOLOcixToEX6TOaNd4fncGU?= =?us-ascii?Q?tOm3P95/RqckeTt6WTSs6WY0UbM2ygAzK9a2HCr/BNHMI65lM1It+wNKj1HA?= =?us-ascii?Q?1IOqNe/BOXKESyKMLRjROkNKXgefUEeY9ChIltvBS3XR6uIvGMz7C9Kli/tV?= =?us-ascii?Q?wtyFQstijW5KV5rSk4l7z79vN2BzdnR9w4KdF5iOHlYc3BrxpaQVQNWPvdLa?= =?us-ascii?Q?qKHqpEp6zPgLAXletz9+1Fnh2gFByfTT4eI8zVIoFo/7i3wNeDMXY8qddt6P?= =?us-ascii?Q?w7tDjmiWxKTo5Hp2MDsfHPjoM2KHfSgGuHFuV3T?= X-Microsoft-Exchange-Diagnostics: 1; PN1PR01MB0462; 6:JMTwtx++9SzuDApvDipGGjk5QPGzxf8yjWvxT2zHHUxREqSOCZUyoWojMcGWbKvTxikvY78C6o07saMM/9f2p/HN8Q4syij6noLYlriKsifbWmshrHAzF9OHHGg2wH74hzVe4bgPIw+izCRLcCIkaMt2p8C2RSFALPnI+9kQvY2h4vbonmpdTgs9XafO8BortJTzgVgKy+9fEVkYX1xKeH4BBNllHmUHPuTncR8+XGL+K+vCZiUw7AVtiZ4uY009DFNW48xdI79gVnXeJybI8i1p6UTMKKwPfTzVGtcyzzRlH5w/+1JEJBo71Q+MuyTJ3Hc3MEiM7kvDmvwLmLfzjJYc7yGeH0hb4xMelQnuK2U=; 5:VOQBEaEtWDd3ZFRWnZGcsfweMCQGyEgKv15uHI+OqQRAPU3iaWmVud/TGERkSrYPkDd5Zop0ZRnT/BMerk+B0QLKZ5cwskvRe38oujq9+LppY/ORN+AFOU90Ue+8g6kdXE9yWdt07hgwZYe/uZsWF8uHIEiX/NPe/afeIjfUU70=; 24:o1McadMSdpuAAA1j+iKs9fHlao7OgitsluB03qamMA5/mZGp0KTx+toCHlQ4yRd/OsG7d943rp+SYqUdaZIEeFpr3J94USvE2d1x2AXRxrw=; 7:2uS0KC46YLiTi/jqg4shOCcyKOS4qFAAQq7ISgT2fwgFyBuhfawXtdoK8LrqLV4Rz8kFlZeLI7M4GKIQ+QpPZrIss4/DmmeLAVc4ib48XgQ/GCqiZn+9c5/s065k3pgLH9pNQQ/TOS149EDrhvHip1Oapuz+cD8Tdq/Span3uSVZBVyZ9U6yG40aoycNSJsErog5KZEdqOVi0oyxAFVjXFTOlrLJj5Ucbgy3uDWoxAC3mb3dfKzvet0yu7MhYZab SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: proofdot.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Feb 2018 16:23:51.7142 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 180c5a2d-81e9-44c2-1479-08d577b529e4 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: e6770ff3-1fee-46be-8f1f-528758499d83 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PN1PR01MB0462 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Feb 2018 16:23:56 -0000 Hello there, Hope you having a great day! Would you be interested in acquiring the list of customers or companies using Machine to Machine (M2M)? We provide the Database across North America, EMEA, APAC and Latin America. We also have other Technology Users like: Internet of Things (IoT), Web of Things (WoT), Big Data, SCADA, Cloud Computing, VoIP, PBX, SIP and many more. Information Fields: Company, First Name, Last Name, Title, Email Address, Address, City, State/Province, ZIP/Postal Code, Country, Phone, Fax, Employees, Revenue, SIC Code, Industry and Web Address. If you're interested please let me know the geography and job title you are looking out to target for so that we can provide you with counts, pricing, sample file and more information in my next email. Appreciate your time and look forward to hear from you. Regards, Sarah Parker Marketing Executive To Opt Out, please reply with Leave Out in the Subject Line.