From owner-freebsd-net@freebsd.org Tue Jul 26 16:06:37 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3DD4BA4445 for ; Tue, 26 Jul 2016 16:06:37 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5F3F318EB for ; Tue, 26 Jul 2016 16:06:36 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 141101A0D71 for ; Tue, 26 Jul 2016 11:06:29 -0500 (CDT) Subject: Re: IPv6 -> IPv4 fallback broken in serf, kernel bug? To: freebsd-net@freebsd.org References: <201607261559.u6QFxF8a081339@gw.catspoiler.org> From: Karl Denninger Message-ID: <4b7e5fc9-7bc6-02e0-f147-3a5cb0e41788@denninger.net> Date: Tue, 26 Jul 2016 11:06:23 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <201607261559.u6QFxF8a081339@gw.catspoiler.org> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms040802090807040902010000" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2016 16:06:37 -0000 This is a cryptographically signed message in MIME format. --------------ms040802090807040902010000 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 7/26/2016 10:59, Don Lewis wrote: > Serf has some code to fall back from IPv4 if an IPv6 and more generally= > try different addresses on multi-homed servers if connection attempts > fail, but it does not work properly on recent versions of FreeBSD. I've= > tested both recent FreeBSD 10.3-STABLE and HEAD. > > The way that it is supposed to work is that serf creates a socket, sets= > it non-blocking, calls connect(), and then passes the fd to poll(). Whe= n > the connection attempt fails, it expects to see a POLLERR event. The > POLLERR event handler will then call getsockopt(fd, SOL_SOCKET, > SO_ERROR, &error, ...). If the returned error is ECONNREFUSED or one o= f > a couple of other errors, then serf will move on to the next address. > > Instead what happens is that serf also(?) sees POLLIN set, which it > processes first by calling read(), which returns an ECONNREFUSED error.= > That not a documented error return from read(). > > An easy way to test this is to truss svn and attempt to do an http > checkout from a host that has both IPv6 and IPv4 addresses, but is not > listening on port 80. The only connection attempt will be to the IPv6 > address. > > socket(PF_INET6,SOCK_STREAM|SOCK_CLOEXEC,6) =3D 4 (0x4) > fcntl(4,F_GETFL,) =3D 2 (0x2) > fcntl(4,F_SETFL,O_NONBLOCK|0x2) =3D 0 (0x0) > setsockopt(0x4,0x6,0x1,0x7fffffffdda4,0x4) =3D 0 (0x0) > gettimeofday({ 1469515046.979461 },0x0) =3D 0 (0x0) > connect(4,{ AF_INET6 [xxxx:xxxx:xxxx:xxxx::xxxx]:80 },28) ERR#36 'Opera= tion now in progress' > gettimeofday({ 1469515046.979614 },0x0) =3D 0 (0x0) > kevent(3,{ 4,EVFILT_READ,EV_ADD,0x0,0x0,0x805491300 },1,0x0,0,0x0) =3D = 0 (0x0) > kevent(3,{ 4,EVFILT_WRITE,EV_ADD,0x0,0x0,0x805491300 },1,0x0,0,0x0) =3D= 0 (0x0) > kevent(3,0x0,0,{ 4,EVFILT_READ,EV_EOF,NOTE_LOWAT|0x3c,0x0,0x805491300 4= ,EVFILT_WRITE,EV_EOF,NOTE_LOWAT|0x3c,0x8000,0x805491300 },32,{ 0.50000000= 0 }) =3D 2 (0x2) > read(4,0x80549c064,8000) ERR#61 'Connection refused' > kevent(3,{ 4,EVFILT_READ,EV_DELETE,0x0,0x0,0x0 },1,0x0,0,0x0) =3D 0 (0x= 0) > kevent(3,{ 4,EVFILT_WRITE,EV_DELETE,0x0,0x0,0x0 },1,0x0,0,0x0) =3D 0 (0= x0) > kevent(3,{ 4,EVFILT_READ,EV_DELETE,0x0,0x0,0x0 },1,0x0,0,0x0) ERR#2 'No= such file or directory' > kevent(3,{ 4,EVFILT_WRITE,EV_DELETE,0x0,0x0,0x0 },1,0x0,0,0x0) ERR#2 'N= o such file or directory' > close(4) =3D 0 (0x0) > close(3) =3D 0 (0x0) > svn: E170013: Unable to connect to a repository at URL ... > > > It looks like it should be possible to patch serf to handle this, but: > * Should POLLIN be set for this event? > =20 > * What errno value should read() return in this case, if it is > ECONNREFUSED, then that should be documented. > > This is kinda serious in that the above manifestation in svn effectively disables it for those of us that are on IPv4 connections and have no provider capability for IPv6 at the present time. When I was running 10.2 this was not a problem but as soon as I rolled forward to 11.x it showed up. Fortunately svnlite does work, but if this same breakage manages to migrate there as well....... --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms040802090807040902010000 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA3MjYxNjA2MjNaME8GCSqGSIb3DQEJBDFCBEC7 clJf7fU6vKtuOJ6TYU3duO0G6VT+xfpEk6I6fi5/hwxebYNlJogdqHr9Y6W3tn/VCFLlEW4x CbAkg7M/ks2nMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAorU7t7pz lIIEWPCFtyveyiqW7Wp4qALm7w01cA9aqhirAQ5A5FH2YhDL3Z2y8eHzQFFcNPgLjdwmd7S0 gt4HX+B8w8FLYoi0EJgowd1rxD696Koep6R9Va9qwTkrknTELUJVWDZ5G6lbEAgrBFseXX2X qELJxX11RLPEwW4h07Lim/imkBqZH89cVqOhcu+Qno/uxjmKLqIST5JgR2+RNVww09YCaswp KwqxPq4WODMOME09hDqYzeaN1MCRU2ZOKjqikxSpzVm/XgqlaFMB3NWN63fMgGlDNBfjea4J YChpsbajO11L+3vBxQYB1pxwgIOfCHpGlrfYA4543KhrGfyNOo1uOM+NfWqJc79lLtfRgqt1 71wKNGBHXq9mCgMI9LBHw6HIjzkD5oKdl3NJYrP3e5xiDF0MlgweU+Y/tNNsJ7IPc7UXBFTM 3e2GFOUuPjTlijPoVolCsVLsqQlJCqYe0I1f4wxsnU2oiN0S19fvMqvWGjtebtKdzpZ/Z2iQ Bx6CEQwl2TeocVkfv3w3bLtQX7ioZ8pZWGxaCgAdxtA82kVYpuoAN/+7K29k8nN1mp76264+ RO6Yog+Rq4GcHhxvfPcfCZOSJ9jDcTKlY+JH6NKjFBFNXZ6om9L8Tt9hc8UvIizJ1kAEcxvy pkAKf5OE8NiXKk/evMvTT2blWIMAAAAAAAA= --------------ms040802090807040902010000--