From owner-freebsd-fs@FreeBSD.ORG Sun May 31 21:00:24 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 258C647F for ; Sun, 31 May 2015 21:00:24 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F1E431326 for ; Sun, 31 May 2015 21:00:23 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t4VL0Nlo033903 for ; Sun, 31 May 2015 21:00:23 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201505312100.t4VL0Nlo033903@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 Date: Sun, 31 May 2015 21:00:23 +0000 Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 May 2015 21:00:24 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f 3 problems total for which you should take action. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 07:07:21 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D971EFD for ; Mon, 1 Jun 2015 07:07:21 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from Exch2-2.slu.se (webmail.slu.se [77.235.224.122]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 226FD15F3 for ; Mon, 1 Jun 2015 07:07:20 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by Exch2-2.slu.se (77.235.224.122) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 1 Jun 2015 08:51:52 +0200 Received: from exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b]) by exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b%22]) with mapi id 15.00.1076.000; Mon, 1 Jun 2015 08:51:51 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: "freebsd-fs@freebsd.org" Subject: Strange networking behaviour in storage server Thread-Topic: Strange networking behaviour in storage server Thread-Index: AQHQnDdwGE76F9tCPEKB5OJa9/tQDQ== Date: Mon, 1 Jun 2015 06:51:51 +0000 Message-ID: <1433141528.14998.165.camel@data-b104.adm.slu.se> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.32] Content-Type: text/plain; charset="utf-8" Content-ID: <1E761E3936F4224295DB02C685D8433C@ad.slu.se> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 07:07:21 -0000 SGV5IQ0KDQpTbyB3ZSBoYXZlIHRoaXMgWkZTIHN0b3JhZ2Ugc2VydmVyIHVwZ3JhZGVkIGZyb20g OS4zLVJFTEVBU0UgdG8NCjEwLjEtU1RBQkxFIHRvIG92ZXJjb21lIG5vdCBiZWluZyBhYmxlIHRv IDEpIHVzZSBTU0QgZHJpdmVzIGFzIEwyQVJDWzFdDQphbmQgMikgbm90IGJlaW5nIGFibGUgdG8g aG90c3dhcCBTQVRBIGRyaXZlc1syXS4NCg0KQWZ0ZXIgdGhlIHVwZ3JhZGUgd2XCtHZlIG5vdGlj ZWQgYSB2ZXJ5IG9kZCBuZXR3b3JraW5nIGJlaGF2aW91ciwgaXQNCnNlbmRzL3JlY2VpdmVzIGZ1 bGwgc3BlZWQgZm9yIGEgd2hpbGUsIHRoZW4gdGhlcmUgaXMgYSBjb3VwbGUgb2YgbWludXRlcw0K b2YgY29tcGxldGUgc2lsZW5jZSB3aGVyZSBldmVuIHRlcm1pbmFsIGNvbW1hbmRzIGxpa2UgYW4g ImxzIiBqdXN0IHdhaXRzDQp1bnRpbCB0aGV5IGFyZSBleGVjdXRlZCBhbmQgdGhlbiBpdCBzdGFy dHMgc2VuZGluZyBmdWxsIHNwZWVkIGFnYWluLiBJDQrCtHZlIGxpbmtlZCB0byBhIHNjcmVlbnNo b3Qgc2hvd2luZyB0aGlzIHNlbmQgYW5kIHBhdXNlIGJlaGF2aW91ci4gVGhlDQpibHVlIGxpbmUg aXMgdGhlIHRvdGFsLCBncmVlbiBpcyBTTUIgYW5kIHR1cnF1b2lzZSBpcyBORlMgb3ZlciBqdW1i bw0KZnJhbWVzLiBJdCBiZWhhdmVzIHRoaXMgd2F5IHJlZ2FyZGxlc3Mgb2YgdGhlIHByb3RvY29s Lg0KDQpodHRwOi8vb2k2Mi50aW55cGljLmNvbS8zM3h2amI2LmpwZw0KDQpUaGUgcHJvYmxlbSBp cyB0aGF0IHRoZXNlIHBhdXNlcyBjYW4gc29tZXRpbWVzIGJlIHNvIGxvbmcgdGhhdA0KY29ubmVj dGlvbnMgZHJvcC4gTGlrZSBzb21lb25lIGlzIGNvcHlpbmcgZmlsZXMgb3ZlciBTTUIgb3IgaVND U0kgYW5kDQpzdWRkZW5seSB0aGV5IGdldCBhbiBlcnJvciBtZXNzYWdlIHNheWluZyB0aGF0IHRo ZSB0cmFuc2ZlciBmYWlsZWQgYW5kDQp0aGV5IGhhdmUgdG8gc3RhcnQgb3ZlciB3aXRoIHRoZSBm aWxlKHMpLiBUaGF0wrRzIGhvcnJpYmxlIQ0KDQpTbyBmYXIgTkZTIGhhcyBwcm92ZW4gdG8gYmUg dGhlIG1vc3QgcmVzaWxsaWVudCwgaXTCtHMgc3R1cGlkIHNpbXBsZQ0KbmF0dXJlIGp1c3Qgd2Fp dHMgYW5kIHJlc3VtZXMgdHJhbnNmZXIgd2hlbiBwYXVzZSBpcyBvdmVyLiBLdWR1cyBmb3INCnRo YXQuDQoNClRoZSBzZXJ2ZXIgaXMgZHJpdmVuIGJ5IGEgU3VwZXJtaWNybyBYOVNSTC1GLCBhIFhl b24gMTYyMHYyIGFuZCA2NEdCIEVDQw0KUkFNLiBUaGUgaGFyZHdhcmUgaGFzIGJlZW4gcnVsZWQg b3V0LCB3ZSBoYXBwZW5lZCB0byBoYXZlIGEgaWRlbnRpY2FsIE1CDQphbmQgQ1BVIGx5aW5nIGFy b3VuZCBhbmQgdGhhdCBkaWRuwrR0IGltcHJvdmUgdGhpbmdzLiBXZSBoYXZlIGFsc28NCmluc3Rh bGxlZCBhIEludGVsIFBSTyAxMDAvMTAwMCBRdWFkLXBvcnQgZXRoZXJuZXQgYWRhcHRlciB0byB0 ZXN0IGlmDQp0aGF0IHdvdWxkIGNoYW5nZSB0aGluZ3MsIGJ1dCBpdCBoYXNuwrR0LCBpdCBzdGls bCBiZWhhdmVzIHRoaXMgd2F5Lg0KDQpUaGUgdHdvIGJ1aWx0LWluIE5JQydzIGFyZSBJbnRlbCA4 MjU3NEwgYW5kIHRoZSBRdWFkLXBvcnQgTklDJ3MgYXJlDQpJbnRlbCA4MjU3MUVCLCBzbyBib3Ro IGVtKDQpIGRyaXZlbi4gSSBoYXBwZW4gdG8ga25vdyB0aGF0IHRoZSBlbSBkcml2ZXINCmhhcyB1 cGRhdGVkIGJldHdlZW4gOS4zIGFuZCAxMC4xLiBQZXJoYXBzIHRoYXQgaXMgdG8gYmxhbWUsIGJ1 dCBJIGhhdmUNCm5vIGlkZWEuDQoNCklzIHRoZXJlIGFueW9uZSB0aGF0IGNhbiBtYWtlIHNlbnNl IG9mIHRoaXM/DQoNClsxXToNCmh0dHBzOi8vYnVncy5mcmVlYnNkLm9yZy9idWd6aWxsYS9zaG93 X2J1Zy5jZ2k/aWQ9MTk3MTY0DQoNClsyXToNCmh0dHBzOi8vYnVncy5mcmVlYnNkLm9yZy9idWd6 aWxsYS9zaG93X2J1Zy5jZ2k/aWQ9MTkxMzQ4DQoNCi9LDQo= From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 08:05:29 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 25491C1A for ; Mon, 1 Jun 2015 08:05:29 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from EXCH2-1.slu.se (webmail.slu.se [77.235.224.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AAF6B15C2 for ; Mon, 1 Jun 2015 08:05:28 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by EXCH2-1.slu.se (77.235.224.121) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 1 Jun 2015 09:49:57 +0200 Received: from exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b]) by exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b%22]) with mapi id 15.00.1076.000; Mon, 1 Jun 2015 09:49:57 +0200 From: =?iso-8859-1?Q?Karli_Sj=F6berg?= To: "freebsd-fs@freebsd.org" Subject: Strange networking behaviour in storage server Thread-Topic: Strange networking behaviour in storage server Thread-Index: AQHQnD9NwzSA7ysFDUSHMhgmIK6yWw== Date: Mon, 1 Jun 2015 07:49:56 +0000 Message-ID: <1433145014066.28625@slu.se> Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.32] MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 08:05:29 -0000 Hey! So we have this ZFS storage server upgraded from 9.3-RELEASE to 10.1-STABLE to overcome not being able to 1) use SSD drives as L2ARC[1] and 2) not being able to hotswap SATA drives[2]. After the upgrade we=B4ve noticed a very odd networking behaviour, it sends/receives full speed for a while, then there is a couple of minutes of complete silence where even terminal commands like an "ls" just waits until they are executed and then it starts sending full speed again. I =B4ve linked to a screenshot showing this send and pause behaviour. The blue line is the total, green is SMB and turquoise is NFS over jumbo frames. It behaves this way regardless of the protocol. http://oi62.tinypic.com/33xvjb6.jpg The problem is that these pauses can sometimes be so long that connections drop. Like someone is copying files over SMB or iSCSI and suddenly they get an error message saying that the transfer failed and they have to start over with the file(s). That=B4s horrible! So far NFS has proven to be the most resillient, it=B4s stupid simple nature just waits and resumes transfer when pause is over. Kudus for that. The server is driven by a Supermicro X9SRL-F, a Xeon 1620v2 and 64GB ECC RAM. The hardware has been ruled out, we happened to have a identical MB and CPU lying around and that didn=B4t improve things. We have also installed a Intel PRO 100/1000 Quad-port ethernet adapter to test if that would change things, but it hasn=B4t, it still behaves this way. The two built-in NIC's are Intel 82574L and the Quad-port NIC's are Intel 82571EB, so both em(4) driven. I happen to know that the em driver has updated between 9.3 and 10.1. Perhaps that is to blame, but I have no idea. Is there anyone that can make sense of this? [1]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D197164 [2]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191348 /K From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 08:15:14 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA66EE36 for ; Mon, 1 Jun 2015 08:15:14 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from EXCH2-1.slu.se (pop.slu.se [77.235.224.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 501F3185E for ; Mon, 1 Jun 2015 08:15:13 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by EXCH2-1.slu.se (77.235.224.121) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 1 Jun 2015 10:14:50 +0200 Received: from exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b]) by exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b%22]) with mapi id 15.00.1076.000; Mon, 1 Jun 2015 10:14:50 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: "freebsd-fs@freebsd.org" Subject: [Fwd: Strange networking behaviour in storage server] Thread-Topic: [Fwd: Strange networking behaviour in storage server] Thread-Index: AQHQnEMHKRZDP6w6/0GyUl+lHxPr0w== Date: Mon, 1 Jun 2015 08:14:49 +0000 Message-ID: <1433146506.14998.177.camel@data-b104.adm.slu.se> Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.32] Content-Type: text/plain; charset="utf-8" Content-ID: <5ECAB7C268A0AA49A92AABDB5EA13CB9@ad.slu.se> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 08:15:14 -0000 LS0tLS0tLS0gVmlkYXJlYmVmb3JkcmF0IG1lZGRlbGFuZGUgLS0tLS0tLS0NCj4gRnLDpW46IEth cmxpIFNqw7ZiZXJnIDxrYXJsaS5zam9iZXJnQHNsdS5zZT4NCj4gVGlsbDogZnJlZWJzZC1mc0Bm cmVlYnNkLm9yZyA8ZnJlZWJzZC1mc0BmcmVlYnNkLm9yZz4NCj4gw4RtbmU6IFN0cmFuZ2UgbmV0 d29ya2luZyBiZWhhdmlvdXIgaW4gc3RvcmFnZSBzZXJ2ZXINCj4gRGF0dW06IE1vbiwgMSBKdW4g MjAxNSAwNzo0OTo1NiArMDAwMA0KPiANCj4gSGV5IQ0KPiANCj4gU28gd2UgaGF2ZSB0aGlzIFpG UyBzdG9yYWdlIHNlcnZlciB1cGdyYWRlZCBmcm9tIDkuMy1SRUxFQVNFIHRvDQo+IDEwLjEtU1RB QkxFIHRvIG92ZXJjb21lIG5vdCBiZWluZyBhYmxlIHRvIDEpIHVzZSBTU0QgZHJpdmVzIGFzDQo+ IEwyQVJDWzFdDQo+IGFuZCAyKSBub3QgYmVpbmcgYWJsZSB0byBob3Rzd2FwIFNBVEEgZHJpdmVz WzJdLg0KPiANCj4gQWZ0ZXIgdGhlIHVwZ3JhZGUgd2XCtHZlIG5vdGljZWQgYSB2ZXJ5IG9kZCBu ZXR3b3JraW5nIGJlaGF2aW91ciwgaXQNCj4gc2VuZHMvcmVjZWl2ZXMgZnVsbCBzcGVlZCBmb3Ig YSB3aGlsZSwgdGhlbiB0aGVyZSBpcyBhIGNvdXBsZSBvZg0KPiBtaW51dGVzDQo+IG9mIGNvbXBs ZXRlIHNpbGVuY2Ugd2hlcmUgZXZlbiB0ZXJtaW5hbCBjb21tYW5kcyBsaWtlIGFuICJscyIganVz dA0KPiB3YWl0cw0KPiB1bnRpbCB0aGV5IGFyZSBleGVjdXRlZCBhbmQgdGhlbiBpdCBzdGFydHMg c2VuZGluZyBmdWxsIHNwZWVkIGFnYWluLiBJDQo+IMK0dmUgbGlua2VkIHRvIGEgc2NyZWVuc2hv dCBzaG93aW5nIHRoaXMgc2VuZCBhbmQgcGF1c2UgYmVoYXZpb3VyLiBUaGUNCj4gYmx1ZSBsaW5l IGlzIHRoZSB0b3RhbCwgZ3JlZW4gaXMgU01CIGFuZCB0dXJxdW9pc2UgaXMgTkZTIG92ZXIganVt Ym8NCj4gZnJhbWVzLiBJdCBiZWhhdmVzIHRoaXMgd2F5IHJlZ2FyZGxlc3Mgb2YgdGhlIHByb3Rv Y29sLg0KPiANCj4gaHR0cDovL29pNjIudGlueXBpYy5jb20vMzN4dmpiNi5qcGcNCj4gDQo+IFRo ZSBwcm9ibGVtIGlzIHRoYXQgdGhlc2UgcGF1c2VzIGNhbiBzb21ldGltZXMgYmUgc28gbG9uZyB0 aGF0DQo+IGNvbm5lY3Rpb25zIGRyb3AuIExpa2Ugc29tZW9uZSBpcyBjb3B5aW5nIGZpbGVzIG92 ZXIgU01CIG9yIGlTQ1NJIGFuZA0KPiBzdWRkZW5seSB0aGV5IGdldCBhbiBlcnJvciBtZXNzYWdl IHNheWluZyB0aGF0IHRoZSB0cmFuc2ZlciBmYWlsZWQgYW5kDQo+IHRoZXkgaGF2ZSB0byBzdGFy dCBvdmVyIHdpdGggdGhlIGZpbGUocykuIFRoYXTCtHMgaG9ycmlibGUhDQo+IA0KPiBTbyBmYXIg TkZTIGhhcyBwcm92ZW4gdG8gYmUgdGhlIG1vc3QgcmVzaWxsaWVudCwgaXTCtHMgc3R1cGlkIHNp bXBsZQ0KPiBuYXR1cmUganVzdCB3YWl0cyBhbmQgcmVzdW1lcyB0cmFuc2ZlciB3aGVuIHBhdXNl IGlzIG92ZXIuIEt1ZHVzIGZvcg0KPiB0aGF0Lg0KPiANCj4gVGhlIHNlcnZlciBpcyBkcml2ZW4g YnkgYSBTdXBlcm1pY3JvIFg5U1JMLUYsIGEgWGVvbiAxNjIwdjIgYW5kIDY0R0INCj4gRUNDDQo+ IFJBTS4gVGhlIGhhcmR3YXJlIGhhcyBiZWVuIHJ1bGVkIG91dCwgd2UgaGFwcGVuZWQgdG8gaGF2 ZSBhIGlkZW50aWNhbA0KPiBNQg0KPiBhbmQgQ1BVIGx5aW5nIGFyb3VuZCBhbmQgdGhhdCBkaWRu wrR0IGltcHJvdmUgdGhpbmdzLiBXZSBoYXZlIGFsc28NCj4gaW5zdGFsbGVkIGEgSW50ZWwgUFJP IDEwMC8xMDAwIFF1YWQtcG9ydCBldGhlcm5ldCBhZGFwdGVyIHRvIHRlc3QgaWYNCj4gdGhhdCB3 b3VsZCBjaGFuZ2UgdGhpbmdzLCBidXQgaXQgaGFzbsK0dCwgaXQgc3RpbGwgYmVoYXZlcyB0aGlz IHdheS4NCj4gDQo+IFRoZSB0d28gYnVpbHQtaW4gTklDJ3MgYXJlIEludGVsIDgyNTc0TCBhbmQg dGhlIFF1YWQtcG9ydCBOSUMncyBhcmUNCj4gSW50ZWwgODI1NzFFQiwgc28gYm90aCBlbSg0KSBk cml2ZW4uIEkgaGFwcGVuIHRvIGtub3cgdGhhdCB0aGUgZW0NCj4gZHJpdmVyDQo+IGhhcyB1cGRh dGVkIGJldHdlZW4gOS4zIGFuZCAxMC4xLiBQZXJoYXBzIHRoYXQgaXMgdG8gYmxhbWUsIGJ1dCBJ IGhhdmUNCj4gbm8gaWRlYS4NCj4gDQo+IElzIHRoZXJlIGFueW9uZSB0aGF0IGNhbiBtYWtlIHNl bnNlIG9mIHRoaXM/DQo+IA0KPiBbMV06DQo+IGh0dHBzOi8vYnVncy5mcmVlYnNkLm9yZy9idWd6 aWxsYS9zaG93X2J1Zy5jZ2k/aWQ9MTk3MTY0DQo+IA0KPiBbMl06DQo+IGh0dHBzOi8vYnVncy5m cmVlYnNkLm9yZy9idWd6aWxsYS9zaG93X2J1Zy5jZ2k/aWQ9MTkxMzQ4DQo+IA0KPiAvSw0KPiAN Cj4gDQoNCkFub3RoZXIgb2JzZXJ2YXRpb24gScK0dmUgbWFkZSBpcyB0aGF0IGR1cmluZyB0aGVz ZSBwYXVzZXMsIHRoZSBlbnRpcmUNCnN5c3RlbSBpcyBwdXQgb24gaG9sZCwgZXZlbiBaRlMgc2Ny dWIgc3RvcHMgYW5kIHRoZW4gcmVzdW1lcyBhZnRlciBhDQp3aGlsZS4gTG9va2luZyBpbiB0b3As IHRoZSBzeXN0ZW0gaXMgY29tcGxldGx5IGlkbGUuDQoNCk5vcm1hbGx5IGR1cmluZyBzY3J1Yiwg dGhlIGtlcm5lbCBlYXRzIDIwLTMwJSBDUFUsIGJ1dCBkdXJpbmcgYSBwYXVzZSwNCmV2ZW4gdGhl IFtrZXJuZWxdIGdvZXMgZG93biB0byAwLjAwJS4gTWFrZXMgbWUgdGhpbmsgdGhlIG5ldHdvcmtp bmcgaGFzDQpub3RoaW5nIHRvIGRvIHdpdGggaXQuDQoNCldoYXTCtHMgdGhlbiB0byBibGFtZT8g WkZTPw0KDQovSw0K From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 08:33:09 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5284D187 for ; Mon, 1 Jun 2015 08:33:09 +0000 (UTC) (envelope-from andrnils@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CEC721CE7 for ; Mon, 1 Jun 2015 08:33:08 +0000 (UTC) (envelope-from andrnils@gmail.com) Received: by wibut5 with SMTP id ut5so31180293wib.1 for ; Mon, 01 Jun 2015 01:33:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=jCP7kQJlkWNZXAq25hJhbugakijqlrlDEllyCcMnOzc=; b=Dgnc0LOJ5fnNFB8tFhGW5GTdjP+4wF8EmIrADQ4JSfDmwVIDiIbbfjxQj8wxwXYU+3 euvct/dayRdrbVPWZ4AGPGnBGIi1yyOE/zHDq/MsnwhtDu1KrQWiJ2k1Fz1zkliaB292 rxpRN+4MgiZnQLII8+fxfFQIVB1KUbB7ZdnhnfJX6acflnfOplePSt3RS/JQs775b/hn sMlmh0zRlw8oGlPE5p9P77SAXoeAcWEGCt2BUWqgWuJWYxbKHVNGxTR5VrzYsyHuiGuW m0iqYpuRb5/1jZaQS6x+HbWwQbrxQU9URJJnnte5qvyQwaKfmKsEjsVAl1cHpcO3L+tp HzJQ== MIME-Version: 1.0 X-Received: by 10.180.91.107 with SMTP id cd11mr17931936wib.51.1433147587138; Mon, 01 Jun 2015 01:33:07 -0700 (PDT) Received: by 10.28.153.10 with HTTP; Mon, 1 Jun 2015 01:33:07 -0700 (PDT) In-Reply-To: <1433146506.14998.177.camel@data-b104.adm.slu.se> References: <1433146506.14998.177.camel@data-b104.adm.slu.se> Date: Mon, 1 Jun 2015 10:33:07 +0200 Message-ID: Subject: Re: [Fwd: Strange networking behaviour in storage server] From: Andreas Nilsson To: =?UTF-8?Q?Karli_Sj=C3=B6berg?= Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 08:33:09 -0000 On Mon, Jun 1, 2015 at 10:14 AM, Karli Sj=C3=B6berg = wrote: > -------- Vidarebefordrat meddelande -------- > > Fr=C3=A5n: Karli Sj=C3=B6berg > > Till: freebsd-fs@freebsd.org > > =C3=84mne: Strange networking behaviour in storage server > > Datum: Mon, 1 Jun 2015 07:49:56 +0000 > > > > Hey! > > > > So we have this ZFS storage server upgraded from 9.3-RELEASE to > > 10.1-STABLE to overcome not being able to 1) use SSD drives as > > L2ARC[1] > > and 2) not being able to hotswap SATA drives[2]. > > > > After the upgrade we=C2=B4ve noticed a very odd networking behaviour, i= t > > sends/receives full speed for a while, then there is a couple of > > minutes > > of complete silence where even terminal commands like an "ls" just > > waits > > until they are executed and then it starts sending full speed again. I > > =C2=B4ve linked to a screenshot showing this send and pause behaviour. = The > > blue line is the total, green is SMB and turquoise is NFS over jumbo > > frames. It behaves this way regardless of the protocol. > > > > http://oi62.tinypic.com/33xvjb6.jpg > > > > The problem is that these pauses can sometimes be so long that > > connections drop. Like someone is copying files over SMB or iSCSI and > > suddenly they get an error message saying that the transfer failed and > > they have to start over with the file(s). That=C2=B4s horrible! > > > > So far NFS has proven to be the most resillient, it=C2=B4s stupid simpl= e > > nature just waits and resumes transfer when pause is over. Kudus for > > that. > > > > The server is driven by a Supermicro X9SRL-F, a Xeon 1620v2 and 64GB > > ECC > > RAM. The hardware has been ruled out, we happened to have a identical > > MB > > and CPU lying around and that didn=C2=B4t improve things. We have also > > installed a Intel PRO 100/1000 Quad-port ethernet adapter to test if > > that would change things, but it hasn=C2=B4t, it still behaves this way= . > > > > The two built-in NIC's are Intel 82574L and the Quad-port NIC's are > > Intel 82571EB, so both em(4) driven. I happen to know that the em > > driver > > has updated between 9.3 and 10.1. Perhaps that is to blame, but I have > > no idea. > > > > Is there anyone that can make sense of this? > > > > [1]: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D197164 > > > > [2]: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191348 > > > > /K > > > > > > Another observation I=C2=B4ve made is that during these pauses, the entir= e > system is put on hold, even ZFS scrub stops and then resumes after a > while. Looking in top, the system is completly idle. > > Normally during scrub, the kernel eats 20-30% CPU, but during a pause, > even the [kernel] goes down to 0.00%. Makes me think the networking has > nothing to do with it. > > What=C2=B4s then to blame? ZFS? > > /K > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > Hello, does this happen when clients are only reading from server? Otherwise I would suspect that it could be caused by ZFS writing out a large chunck of data sitting in its caches, and until that is complete I/O is stalled. Have you tried what is suggested in https://wiki.freebsd.org/ZFSTuningGuide ? In particular setting vfs.zfs.write_limit_override to something appropriate for your site. The timeout seems to be defaulting to 5 now. Best regards Andreas From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 09:17:43 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17604CCA for ; Mon, 1 Jun 2015 09:17:43 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from Exch2-3.slu.se (pop.slu.se [77.235.224.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9ED0D1711 for ; Mon, 1 Jun 2015 09:17:41 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by Exch2-3.slu.se (77.235.224.123) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 1 Jun 2015 11:02:12 +0200 Received: from exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b]) by exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b%22]) with mapi id 15.00.1076.000; Mon, 1 Jun 2015 11:02:12 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Andreas Nilsson CC: "freebsd-fs@freebsd.org" Subject: Re: [Fwd: Strange networking behaviour in storage server] Thread-Topic: [Fwd: Strange networking behaviour in storage server] Thread-Index: AQHQnEMH0FmSWuTxD0+w8f7+o424g52XMR6AgAAINIA= Date: Mon, 1 Jun 2015 09:02:12 +0000 Message-ID: <1433149349.14998.181.camel@data-b104.adm.slu.se> References: <1433146506.14998.177.camel@data-b104.adm.slu.se> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.32] Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 09:17:43 -0000 bcOlbiAyMDE1LTA2LTAxIGtsb2NrYW4gMTA6MzMgKzAyMDAgc2tyZXYgQW5kcmVhcyBOaWxzc29u Og0KPiANCj4gDQo+IE9uIE1vbiwgSnVuIDEsIDIwMTUgYXQgMTA6MTQgQU0sIEthcmxpIFNqw7Zi ZXJnIDxrYXJsaS5zam9iZXJnQHNsdS5zZT4NCj4gd3JvdGU6DQo+ICAgICAgICAgLS0tLS0tLS0g VmlkYXJlYmVmb3JkcmF0IG1lZGRlbGFuZGUgLS0tLS0tLS0NCj4gICAgICAgICA+IEZyw6VuOiBL YXJsaSBTasO2YmVyZyA8a2FybGkuc2pvYmVyZ0BzbHUuc2U+DQo+ICAgICAgICAgPiBUaWxsOiBm cmVlYnNkLWZzQGZyZWVic2Qub3JnIDxmcmVlYnNkLWZzQGZyZWVic2Qub3JnPg0KPiAgICAgICAg ID4gw4RtbmU6IFN0cmFuZ2UgbmV0d29ya2luZyBiZWhhdmlvdXIgaW4gc3RvcmFnZSBzZXJ2ZXIN Cj4gICAgICAgICA+IERhdHVtOiBNb24sIDEgSnVuIDIwMTUgMDc6NDk6NTYgKzAwMDANCj4gICAg ICAgICA+DQo+ICAgICAgICAgPiBIZXkhDQo+ICAgICAgICAgPg0KPiAgICAgICAgID4gU28gd2Ug aGF2ZSB0aGlzIFpGUyBzdG9yYWdlIHNlcnZlciB1cGdyYWRlZCBmcm9tIDkuMy1SRUxFQVNFDQo+ ICAgICAgICAgdG8NCj4gICAgICAgICA+IDEwLjEtU1RBQkxFIHRvIG92ZXJjb21lIG5vdCBiZWlu ZyBhYmxlIHRvIDEpIHVzZSBTU0QgZHJpdmVzDQo+ICAgICAgICAgYXMNCj4gICAgICAgICA+IEwy QVJDWzFdDQo+ICAgICAgICAgPiBhbmQgMikgbm90IGJlaW5nIGFibGUgdG8gaG90c3dhcCBTQVRB IGRyaXZlc1syXS4NCj4gICAgICAgICA+DQo+ICAgICAgICAgPiBBZnRlciB0aGUgdXBncmFkZSB3 ZcK0dmUgbm90aWNlZCBhIHZlcnkgb2RkIG5ldHdvcmtpbmcNCj4gICAgICAgICBiZWhhdmlvdXIs IGl0DQo+ICAgICAgICAgPiBzZW5kcy9yZWNlaXZlcyBmdWxsIHNwZWVkIGZvciBhIHdoaWxlLCB0 aGVuIHRoZXJlIGlzIGENCj4gICAgICAgICBjb3VwbGUgb2YNCj4gICAgICAgICA+IG1pbnV0ZXMN Cj4gICAgICAgICA+IG9mIGNvbXBsZXRlIHNpbGVuY2Ugd2hlcmUgZXZlbiB0ZXJtaW5hbCBjb21t YW5kcyBsaWtlIGFuDQo+ICAgICAgICAgImxzIiBqdXN0DQo+ICAgICAgICAgPiB3YWl0cw0KPiAg ICAgICAgID4gdW50aWwgdGhleSBhcmUgZXhlY3V0ZWQgYW5kIHRoZW4gaXQgc3RhcnRzIHNlbmRp bmcgZnVsbA0KPiAgICAgICAgIHNwZWVkIGFnYWluLiBJDQo+ICAgICAgICAgPiDCtHZlIGxpbmtl ZCB0byBhIHNjcmVlbnNob3Qgc2hvd2luZyB0aGlzIHNlbmQgYW5kIHBhdXNlDQo+ICAgICAgICAg YmVoYXZpb3VyLiBUaGUNCj4gICAgICAgICA+IGJsdWUgbGluZSBpcyB0aGUgdG90YWwsIGdyZWVu IGlzIFNNQiBhbmQgdHVycXVvaXNlIGlzIE5GUw0KPiAgICAgICAgIG92ZXIganVtYm8NCj4gICAg ICAgICA+IGZyYW1lcy4gSXQgYmVoYXZlcyB0aGlzIHdheSByZWdhcmRsZXNzIG9mIHRoZSBwcm90 b2NvbC4NCj4gICAgICAgICA+DQo+ICAgICAgICAgPiBodHRwOi8vb2k2Mi50aW55cGljLmNvbS8z M3h2amI2LmpwZw0KPiAgICAgICAgID4NCj4gICAgICAgICA+IFRoZSBwcm9ibGVtIGlzIHRoYXQg dGhlc2UgcGF1c2VzIGNhbiBzb21ldGltZXMgYmUgc28gbG9uZw0KPiAgICAgICAgIHRoYXQNCj4g ICAgICAgICA+IGNvbm5lY3Rpb25zIGRyb3AuIExpa2Ugc29tZW9uZSBpcyBjb3B5aW5nIGZpbGVz IG92ZXIgU01CIG9yDQo+ICAgICAgICAgaVNDU0kgYW5kDQo+ICAgICAgICAgPiBzdWRkZW5seSB0 aGV5IGdldCBhbiBlcnJvciBtZXNzYWdlIHNheWluZyB0aGF0IHRoZSB0cmFuc2Zlcg0KPiAgICAg ICAgIGZhaWxlZCBhbmQNCj4gICAgICAgICA+IHRoZXkgaGF2ZSB0byBzdGFydCBvdmVyIHdpdGgg dGhlIGZpbGUocykuIFRoYXTCtHMgaG9ycmlibGUhDQo+ICAgICAgICAgPg0KPiAgICAgICAgID4g U28gZmFyIE5GUyBoYXMgcHJvdmVuIHRvIGJlIHRoZSBtb3N0IHJlc2lsbGllbnQsIGl0wrRzIHN0 dXBpZA0KPiAgICAgICAgIHNpbXBsZQ0KPiAgICAgICAgID4gbmF0dXJlIGp1c3Qgd2FpdHMgYW5k IHJlc3VtZXMgdHJhbnNmZXIgd2hlbiBwYXVzZSBpcyBvdmVyLg0KPiAgICAgICAgIEt1ZHVzIGZv cg0KPiAgICAgICAgID4gdGhhdC4NCj4gICAgICAgICA+DQo+ICAgICAgICAgPiBUaGUgc2VydmVy IGlzIGRyaXZlbiBieSBhIFN1cGVybWljcm8gWDlTUkwtRiwgYSBYZW9uIDE2MjB2Mg0KPiAgICAg ICAgIGFuZCA2NEdCDQo+ICAgICAgICAgPiBFQ0MNCj4gICAgICAgICA+IFJBTS4gVGhlIGhhcmR3 YXJlIGhhcyBiZWVuIHJ1bGVkIG91dCwgd2UgaGFwcGVuZWQgdG8gaGF2ZSBhDQo+ICAgICAgICAg aWRlbnRpY2FsDQo+ICAgICAgICAgPiBNQg0KPiAgICAgICAgID4gYW5kIENQVSBseWluZyBhcm91 bmQgYW5kIHRoYXQgZGlkbsK0dCBpbXByb3ZlIHRoaW5ncy4gV2UgaGF2ZQ0KPiAgICAgICAgIGFs c28NCj4gICAgICAgICA+IGluc3RhbGxlZCBhIEludGVsIFBSTyAxMDAvMTAwMCBRdWFkLXBvcnQg ZXRoZXJuZXQgYWRhcHRlciB0bw0KPiAgICAgICAgIHRlc3QgaWYNCj4gICAgICAgICA+IHRoYXQg d291bGQgY2hhbmdlIHRoaW5ncywgYnV0IGl0IGhhc27CtHQsIGl0IHN0aWxsIGJlaGF2ZXMNCj4g ICAgICAgICB0aGlzIHdheS4NCj4gICAgICAgICA+DQo+ICAgICAgICAgPiBUaGUgdHdvIGJ1aWx0 LWluIE5JQydzIGFyZSBJbnRlbCA4MjU3NEwgYW5kIHRoZSBRdWFkLXBvcnQNCj4gICAgICAgICBO SUMncyBhcmUNCj4gICAgICAgICA+IEludGVsIDgyNTcxRUIsIHNvIGJvdGggZW0oNCkgZHJpdmVu LiBJIGhhcHBlbiB0byBrbm93IHRoYXQNCj4gICAgICAgICB0aGUgZW0NCj4gICAgICAgICA+IGRy aXZlcg0KPiAgICAgICAgID4gaGFzIHVwZGF0ZWQgYmV0d2VlbiA5LjMgYW5kIDEwLjEuIFBlcmhh cHMgdGhhdCBpcyB0byBibGFtZSwNCj4gICAgICAgICBidXQgSSBoYXZlDQo+ICAgICAgICAgPiBu byBpZGVhLg0KPiAgICAgICAgID4NCj4gICAgICAgICA+IElzIHRoZXJlIGFueW9uZSB0aGF0IGNh biBtYWtlIHNlbnNlIG9mIHRoaXM/DQo+ICAgICAgICAgPg0KPiAgICAgICAgID4gWzFdOg0KPiAg ICAgICAgID4gaHR0cHM6Ly9idWdzLmZyZWVic2Qub3JnL2J1Z3ppbGxhL3Nob3dfYnVnLmNnaT9p ZD0xOTcxNjQNCj4gICAgICAgICA+DQo+ICAgICAgICAgPiBbMl06DQo+ICAgICAgICAgPiBodHRw czovL2J1Z3MuZnJlZWJzZC5vcmcvYnVnemlsbGEvc2hvd19idWcuY2dpP2lkPTE5MTM0OA0KPiAg ICAgICAgID4NCj4gICAgICAgICA+IC9LDQo+ICAgICAgICAgPg0KPiAgICAgICAgID4NCj4gICAg ICAgICANCj4gICAgICAgICANCj4gICAgICAgICBBbm90aGVyIG9ic2VydmF0aW9uIEnCtHZlIG1h ZGUgaXMgdGhhdCBkdXJpbmcgdGhlc2UgcGF1c2VzLCB0aGUNCj4gICAgICAgICBlbnRpcmUNCj4g ICAgICAgICBzeXN0ZW0gaXMgcHV0IG9uIGhvbGQsIGV2ZW4gWkZTIHNjcnViIHN0b3BzIGFuZCB0 aGVuIHJlc3VtZXMNCj4gICAgICAgICBhZnRlciBhDQo+ICAgICAgICAgd2hpbGUuIExvb2tpbmcg aW4gdG9wLCB0aGUgc3lzdGVtIGlzIGNvbXBsZXRseSBpZGxlLg0KPiAgICAgICAgIA0KPiAgICAg ICAgIE5vcm1hbGx5IGR1cmluZyBzY3J1YiwgdGhlIGtlcm5lbCBlYXRzIDIwLTMwJSBDUFUsIGJ1 dCBkdXJpbmcNCj4gICAgICAgICBhIHBhdXNlLA0KPiAgICAgICAgIGV2ZW4gdGhlIFtrZXJuZWxd IGdvZXMgZG93biB0byAwLjAwJS4gTWFrZXMgbWUgdGhpbmsgdGhlDQo+ICAgICAgICAgbmV0d29y a2luZyBoYXMNCj4gICAgICAgICBub3RoaW5nIHRvIGRvIHdpdGggaXQuDQo+ICAgICAgICAgDQo+ ICAgICAgICAgV2hhdMK0cyB0aGVuIHRvIGJsYW1lPyBaRlM/DQo+ICAgICAgICAgDQo+ICAgICAg ICAgL0sNCj4gICAgICAgICBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fXw0KPiAgICAgICAgIGZyZWVic2QtZnNAZnJlZWJzZC5vcmcgbWFpbGluZyBsaXN0DQo+ ICAgICAgICAgaHR0cDovL2xpc3RzLmZyZWVic2Qub3JnL21haWxtYW4vbGlzdGluZm8vZnJlZWJz ZC1mcw0KPiAgICAgICAgIFRvIHVuc3Vic2NyaWJlLCBzZW5kIGFueSBtYWlsIHRvDQo+ICAgICAg ICAgImZyZWVic2QtZnMtdW5zdWJzY3JpYmVAZnJlZWJzZC5vcmciDQo+IA0KPiANCj4gSGVsbG8s DQo+IA0KPiANCj4gZG9lcyB0aGlzIGhhcHBlbiB3aGVuIGNsaWVudHMgYXJlIG9ubHkgcmVhZGlu ZyBmcm9tIHNlcnZlcj8gDQoNClllcyBpdCBoYXBwZW5zIHdoZW4gY2xpZW50cyBhcmUgb25seSBy ZWFkaW5nIGZyb20gdGhlIHNlcnZlci4NCg0KPiBPdGhlcndpc2UgSSB3b3VsZCBzdXNwZWN0IHRo YXQgaXQgY291bGQgYmUgY2F1c2VkIGJ5IFpGUyB3cml0aW5nIG91dCBhDQo+IGxhcmdlIGNodW5j ayBvZiBkYXRhIHNpdHRpbmcgaW4gaXRzIGNhY2hlcywgYW5kIHVudGlsIHRoYXQgaXMgY29tcGxl dGUNCj4gSS9PIGlzIHN0YWxsZWQuDQoNClRoYXTCtHMgd2hhdCBzbyBzdHJhbmdlLCB3ZSBoYXZl IHRocmVlIG1vcmUgc3lzdGVtcyBzZXQgdXAgYWJvdXQgdGhlIHNhbWUNCnNpemUgYW5kIG5vbmUg b2Ygb3RoZXJzIGFyZSBhY3RpbmcgdGhpcyB3YXkuDQoNClRoZSBvbmx5IHRoaW5nIEkgY2FuIHRo aW5rIG9mIHRoYXQgZGlmZmVycyB0aGF0IHdlIGhhdmVuwrR0IHRlc3RlZCBydWxpbmcNCm91dCB5 ZXQgaXMgY3RsZCwgdGhlIG90aGVyIHN5c3RlbXMgYXJlIHN0aWxsIHJ1bm5pbmcgaXN0Z3QgYXMg dGhlaXINCmlTQ1NJIGRhZW1vbi4NCg0KL0sNCg0KPiANCj4gDQo+IEhhdmUgeW91IHRyaWVkIHdo YXQgaXMgc3VnZ2VzdGVkIGluDQo+IGh0dHBzOi8vd2lraS5mcmVlYnNkLm9yZy9aRlNUdW5pbmdH dWlkZSA/IEluIHBhcnRpY3VsYXIgc2V0dGluZw0KPiB2ZnMuemZzLndyaXRlX2xpbWl0X292ZXJy aWRlIHRvIHNvbWV0aGluZyBhcHByb3ByaWF0ZSBmb3IgeW91ciBzaXRlLg0KPiBUaGUgdGltZW91 dCBzZWVtcyB0byBiZSBkZWZhdWx0aW5nIHRvIDUgbm93Lg0KPiANCj4gDQo+IEJlc3QgcmVnYXJk cw0KPiANCj4gQW5kcmVhcw0KPiANCj4gDQo+IA0KDQo= From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 09:56:06 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A94B67EF for ; Mon, 1 Jun 2015 09:56:06 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-ig0-x22c.google.com (mail-ig0-x22c.google.com [IPv6:2607:f8b0:4001:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E907106E for ; Mon, 1 Jun 2015 09:56:06 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: by igbjd9 with SMTP id jd9so57012697igb.1 for ; Mon, 01 Jun 2015 02:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=/POQGUQdVnvh5dAvnaB39ykhX14DNEA1GbeFEXH0nT0=; b=BqUQSs3zdHHZC5Ntx5FhouEaT8NYcD/SM4erQQOtxauW2PEsQVdV9BuCZpLV7w1hFn Bk6D8PfDmNnuQyURgxPLOVqdUQMcFhaBjjI37DnzGmK+PXe0CvA8tczje72CZe3/eQVl 6ElbPLXthawuBno7RCynXOgEAAXxyVB8UUBRrLSuivBzKXe262ahxGCWpz+ZOaXK7+RG gI4Z6a8SlpZnaNaKTch7Nr0/EK/2n//OEcQcNikoja98JXrN/SoyPmNj1DUndlCOkgt5 j0wAyEjzhjG6SwSXRizEQ2hiAWkJHxNZNyJhrPYVPtibFaen55XrxY0Hm0btITo82MX7 tZUA== MIME-Version: 1.0 X-Received: by 10.50.178.230 with SMTP id db6mr12264249igc.26.1433152565804; Mon, 01 Jun 2015 02:56:05 -0700 (PDT) Received: by 10.65.15.33 with HTTP; Mon, 1 Jun 2015 02:56:05 -0700 (PDT) In-Reply-To: <1433149349.14998.181.camel@data-b104.adm.slu.se> References: <1433146506.14998.177.camel@data-b104.adm.slu.se> <1433149349.14998.181.camel@data-b104.adm.slu.se> Date: Mon, 1 Jun 2015 02:56:05 -0700 Message-ID: Subject: Re: [Fwd: Strange networking behaviour in storage server] From: Mehmet Erol Sanliturk To: =?UTF-8?Q?Karli_Sj=C3=B6berg?= Cc: Andreas Nilsson , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 09:56:06 -0000 On Mon, Jun 1, 2015 at 2:02 AM, Karli Sj=C3=B6berg w= rote: > m=C3=A5n 2015-06-01 klockan 10:33 +0200 skrev Andreas Nilsson: > > > > > > On Mon, Jun 1, 2015 at 10:14 AM, Karli Sj=C3=B6berg > > wrote: > > -------- Vidarebefordrat meddelande -------- > > > Fr=C3=A5n: Karli Sj=C3=B6berg > > > Till: freebsd-fs@freebsd.org > > > =C3=84mne: Strange networking behaviour in storage server > > > Datum: Mon, 1 Jun 2015 07:49:56 +0000 > > > > > > Hey! > > > > > > So we have this ZFS storage server upgraded from 9.3-RELEASE > > to > > > 10.1-STABLE to overcome not being able to 1) use SSD drives > > as > > > L2ARC[1] > > > and 2) not being able to hotswap SATA drives[2]. > > > > > > After the upgrade we=C2=B4ve noticed a very odd networking > > behaviour, it > > > sends/receives full speed for a while, then there is a > > couple of > > > minutes > > > of complete silence where even terminal commands like an > > "ls" just > > > waits > > > until they are executed and then it starts sending full > > speed again. I > > > =C2=B4ve linked to a screenshot showing this send and pause > > behaviour. The > > > blue line is the total, green is SMB and turquoise is NFS > > over jumbo > > > frames. It behaves this way regardless of the protocol. > > > > > > http://oi62.tinypic.com/33xvjb6.jpg > > > > > > The problem is that these pauses can sometimes be so long > > that > > > connections drop. Like someone is copying files over SMB or > > iSCSI and > > > suddenly they get an error message saying that the transfer > > failed and > > > they have to start over with the file(s). That=C2=B4s horribl= e! > > > > > > So far NFS has proven to be the most resillient, it=C2=B4s st= upid > > simple > > > nature just waits and resumes transfer when pause is over. > > Kudus for > > > that. > > > > > > The server is driven by a Supermicro X9SRL-F, a Xeon 1620v2 > > and 64GB > > > ECC > > > RAM. The hardware has been ruled out, we happened to have a > > identical > > > MB > > > and CPU lying around and that didn=C2=B4t improve things. We = have > > also > > > installed a Intel PRO 100/1000 Quad-port ethernet adapter to > > test if > > > that would change things, but it hasn=C2=B4t, it still behave= s > > this way. > > > > > > The two built-in NIC's are Intel 82574L and the Quad-port > > NIC's are > > > Intel 82571EB, so both em(4) driven. I happen to know that > > the em > > > driver > > > has updated between 9.3 and 10.1. Perhaps that is to blame, > > but I have > > > no idea. > > > > > > Is there anyone that can make sense of this? > > > > > > [1]: > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D197164 > > > > > > [2]: > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191348 > > > > > > /K > > > > > > > > > > > > Another observation I=C2=B4ve made is that during these pauses,= the > > entire > > system is put on hold, even ZFS scrub stops and then resumes > > after a > > while. Looking in top, the system is completly idle. > > > > Normally during scrub, the kernel eats 20-30% CPU, but during > > a pause, > > even the [kernel] goes down to 0.00%. Makes me think the > > networking has > > nothing to do with it. > > > > What=C2=B4s then to blame? ZFS? > > > > /K > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to > > "freebsd-fs-unsubscribe@freebsd.org" > > > > > > Hello, > > > > > > does this happen when clients are only reading from server? > > Yes it happens when clients are only reading from the server. > > > Otherwise I would suspect that it could be caused by ZFS writing out a > > large chunck of data sitting in its caches, and until that is complete > > I/O is stalled. > > That=C2=B4s what so strange, we have three more systems set up about the = same > size and none of others are acting this way. > > The only thing I can think of that differs that we haven=C2=B4t tested ru= ling > out yet is ctld, the other systems are still running istgt as their > iSCSI daemon. > > /K > > If there are other three similar systems and they are exactly installed with the same structure , my first possibility to consider would be to suspect a slowly progressing hardware failure : >From a circuit , it is not possible to get a response in expected time , but , it is responding after a time which is not normal . Such an action may be caused by a faulty soldered or cracked line point in the circuit : When it is hot , it is disconnecting , when it is cold it is connecting . Thank you very much . Mehmet Erol Sanliturk > > > > > > Have you tried what is suggested in > > https://wiki.freebsd.org/ZFSTuningGuide ? In particular setting > > vfs.zfs.write_limit_override to something appropriate for your site. > > The timeout seems to be defaulting to 5 now. > > > > > > Best regards > > > > Andreas > > > > > > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 10:11:18 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04C78C45 for ; Mon, 1 Jun 2015 10:11:18 +0000 (UTC) (envelope-from andrnils@gmail.com) Received: from mail-wg0-x230.google.com (mail-wg0-x230.google.com [IPv6:2a00:1450:400c:c00::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81B3E1512 for ; Mon, 1 Jun 2015 10:11:17 +0000 (UTC) (envelope-from andrnils@gmail.com) Received: by wgez8 with SMTP id z8so109791134wge.0 for ; Mon, 01 Jun 2015 03:11:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ut+T9SCbGxyhS8mKn4XKR+0VHX3bW6Cwa1c3B0RuMPs=; b=o8tPbSvNiQLfMar00PuC7oapfOW8INaUwLvy4CQIE5o2otkXBfEElEjisj6rubxcRr nvz+kTvYNGApozCqNS/MxWd0w1LoW1O6m4MSlzSruvrnoiFpnJFm9SFhbmx/rjz9Gi0M hWc31SlrBRlTzdfQypX6Rz7P6uH1RvvY/IFGPgPxQbtTiMrTpi4WdFCMHtHnCRzgf1cE VDP0obbGCQx+xclq88x3xGXluI/aSL20AJggc2BXFEkYL1enTFpF0ta3EIO7zXICuxld MrJcKxVJwUlJ3Kd4SD2ZBXtK+y8PHB3RYE8VxWpJiQtG9Cu0KQ9z/iywR8n/xGvIyJgh fi7w== MIME-Version: 1.0 X-Received: by 10.194.104.164 with SMTP id gf4mr39867357wjb.28.1433153475902; Mon, 01 Jun 2015 03:11:15 -0700 (PDT) Received: by 10.28.153.10 with HTTP; Mon, 1 Jun 2015 03:11:15 -0700 (PDT) In-Reply-To: References: <1433146506.14998.177.camel@data-b104.adm.slu.se> <1433149349.14998.181.camel@data-b104.adm.slu.se> Date: Mon, 1 Jun 2015 12:11:15 +0200 Message-ID: Subject: Re: [Fwd: Strange networking behaviour in storage server] From: Andreas Nilsson To: Mehmet Erol Sanliturk Cc: =?UTF-8?Q?Karli_Sj=C3=B6berg?= , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 10:11:18 -0000 On Mon, Jun 1, 2015 at 11:56 AM, Mehmet Erol Sanliturk < m.e.sanliturk@gmail.com> wrote: > > > On Mon, Jun 1, 2015 at 2:02 AM, Karli Sj=C3=B6berg > wrote: > >> m=C3=A5n 2015-06-01 klockan 10:33 +0200 skrev Andreas Nilsson: >> > >> > >> > On Mon, Jun 1, 2015 at 10:14 AM, Karli Sj=C3=B6berg >> > wrote: >> > -------- Vidarebefordrat meddelande -------- >> > > Fr=C3=A5n: Karli Sj=C3=B6berg >> > > Till: freebsd-fs@freebsd.org >> > > =C3=84mne: Strange networking behaviour in storage server >> > > Datum: Mon, 1 Jun 2015 07:49:56 +0000 >> > > >> > > Hey! >> > > >> > > So we have this ZFS storage server upgraded from 9.3-RELEASE >> > to >> > > 10.1-STABLE to overcome not being able to 1) use SSD drives >> > as >> > > L2ARC[1] >> > > and 2) not being able to hotswap SATA drives[2]. >> > > >> > > After the upgrade we=C2=B4ve noticed a very odd networking >> > behaviour, it >> > > sends/receives full speed for a while, then there is a >> > couple of >> > > minutes >> > > of complete silence where even terminal commands like an >> > "ls" just >> > > waits >> > > until they are executed and then it starts sending full >> > speed again. I >> > > =C2=B4ve linked to a screenshot showing this send and pause >> > behaviour. The >> > > blue line is the total, green is SMB and turquoise is NFS >> > over jumbo >> > > frames. It behaves this way regardless of the protocol. >> > > >> > > http://oi62.tinypic.com/33xvjb6.jpg >> > > >> > > The problem is that these pauses can sometimes be so long >> > that >> > > connections drop. Like someone is copying files over SMB or >> > iSCSI and >> > > suddenly they get an error message saying that the transfer >> > failed and >> > > they have to start over with the file(s). That=C2=B4s horrib= le! >> > > >> > > So far NFS has proven to be the most resillient, it=C2=B4s s= tupid >> > simple >> > > nature just waits and resumes transfer when pause is over. >> > Kudus for >> > > that. >> > > >> > > The server is driven by a Supermicro X9SRL-F, a Xeon 1620v2 >> > and 64GB >> > > ECC >> > > RAM. The hardware has been ruled out, we happened to have a >> > identical >> > > MB >> > > and CPU lying around and that didn=C2=B4t improve things. We= have >> > also >> > > installed a Intel PRO 100/1000 Quad-port ethernet adapter to >> > test if >> > > that would change things, but it hasn=C2=B4t, it still behav= es >> > this way. >> > > >> > > The two built-in NIC's are Intel 82574L and the Quad-port >> > NIC's are >> > > Intel 82571EB, so both em(4) driven. I happen to know that >> > the em >> > > driver >> > > has updated between 9.3 and 10.1. Perhaps that is to blame, >> > but I have >> > > no idea. >> > > >> > > Is there anyone that can make sense of this? >> > > >> > > [1]: >> > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D197164 >> > > >> > > [2]: >> > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191348 >> > > >> > > /K >> > > >> > > >> > >> > >> > Another observation I=C2=B4ve made is that during these pauses= , the >> > entire >> > system is put on hold, even ZFS scrub stops and then resumes >> > after a >> > while. Looking in top, the system is completly idle. >> > >> > Normally during scrub, the kernel eats 20-30% CPU, but during >> > a pause, >> > even the [kernel] goes down to 0.00%. Makes me think the >> > networking has >> > nothing to do with it. >> > >> > What=C2=B4s then to blame? ZFS? >> > >> > /K >> > _______________________________________________ >> > freebsd-fs@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> > To unsubscribe, send any mail to >> > "freebsd-fs-unsubscribe@freebsd.org" >> > >> > >> > Hello, >> > >> > >> > does this happen when clients are only reading from server? >> >> Yes it happens when clients are only reading from the server. >> >> > Otherwise I would suspect that it could be caused by ZFS writing out a >> > large chunck of data sitting in its caches, and until that is complete >> > I/O is stalled. >> >> That=C2=B4s what so strange, we have three more systems set up about the= same >> size and none of others are acting this way. >> >> The only thing I can think of that differs that we haven=C2=B4t tested r= uling >> out yet is ctld, the other systems are still running istgt as their >> iSCSI daemon. >> >> /K >> >> What does a zpool status say? Could very well be disks starting to fail. Anything in dmesg concerning cam timeouts? Best regards Andreas From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 10:18:42 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B45C9F4A for ; Mon, 1 Jun 2015 10:18:42 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from Exch2-2.slu.se (pop.slu.se [77.235.224.122]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17F981602 for ; Mon, 1 Jun 2015 10:18:41 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by Exch2-2.slu.se (77.235.224.122) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 1 Jun 2015 12:18:21 +0200 Received: from exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b]) by exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b%22]) with mapi id 15.00.1076.000; Mon, 1 Jun 2015 12:18:21 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Andreas Nilsson CC: Mehmet Erol Sanliturk , "freebsd-fs@freebsd.org" Subject: Re: [Fwd: Strange networking behaviour in storage server] Thread-Topic: [Fwd: Strange networking behaviour in storage server] Thread-Index: AQHQnEMH0FmSWuTxD0+w8f7+o424g52XMR6AgAAINICAAA76gIAABD2AgAACDoA= Date: Mon, 1 Jun 2015 10:18:20 +0000 Message-ID: <1433153917.14998.185.camel@data-b104.adm.slu.se> References: <1433146506.14998.177.camel@data-b104.adm.slu.se> <1433149349.14998.181.camel@data-b104.adm.slu.se> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.32] Content-Type: text/plain; charset="utf-8" Content-ID: <377550D1C845E644A5ACD9E3A902E93B@ad.slu.se> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 10:18:42 -0000 bcOlbiAyMDE1LTA2LTAxIGtsb2NrYW4gMTI6MTEgKzAyMDAgc2tyZXYgQW5kcmVhcyBOaWxzc29u Og0KPiANCj4gDQo+IE9uIE1vbiwgSnVuIDEsIDIwMTUgYXQgMTE6NTYgQU0sIE1laG1ldCBFcm9s IFNhbmxpdHVyaw0KPiA8bS5lLnNhbmxpdHVya0BnbWFpbC5jb20+IHdyb3RlOg0KPiAgICAgICAg IA0KPiAgICAgICAgIA0KPiAgICAgICAgIE9uIE1vbiwgSnVuIDEsIDIwMTUgYXQgMjowMiBBTSwg S2FybGkgU2rDtmJlcmcNCj4gICAgICAgICA8a2FybGkuc2pvYmVyZ0BzbHUuc2U+IHdyb3RlOg0K PiAgICAgICAgICAgICAgICAgbcOlbiAyMDE1LTA2LTAxIGtsb2NrYW4gMTA6MzMgKzAyMDAgc2ty ZXYgQW5kcmVhcw0KPiAgICAgICAgICAgICAgICAgTmlsc3NvbjoNCj4gICAgICAgICAgICAgICAg ID4NCj4gICAgICAgICAgICAgICAgID4NCj4gICAgICAgICAgICAgICAgID4gT24gTW9uLCBKdW4g MSwgMjAxNSBhdCAxMDoxNCBBTSwgS2FybGkgU2rDtmJlcmcNCj4gICAgICAgICAgICAgICAgIDxr YXJsaS5zam9iZXJnQHNsdS5zZT4NCj4gICAgICAgICAgICAgICAgID4gd3JvdGU6DQo+ICAgICAg ICAgICAgICAgICA+ICAgICAgICAgLS0tLS0tLS0gVmlkYXJlYmVmb3JkcmF0IG1lZGRlbGFuZGUg LS0tLS0tLS0NCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IEZyw6VuOiBLYXJsaSBTasO2 YmVyZyA8a2FybGkuc2pvYmVyZ0BzbHUuc2U+DQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAg PiBUaWxsOiBmcmVlYnNkLWZzQGZyZWVic2Qub3JnDQo+ICAgICAgICAgICAgICAgICA8ZnJlZWJz ZC1mc0BmcmVlYnNkLm9yZz4NCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IMOEbW5lOiBT dHJhbmdlIG5ldHdvcmtpbmcgYmVoYXZpb3VyIGluDQo+ICAgICAgICAgICAgICAgICBzdG9yYWdl IHNlcnZlcg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgID4gRGF0dW06IE1vbiwgMSBKdW4g MjAxNSAwNzo0OTo1NiArMDAwMA0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgID4NCj4gICAg ICAgICAgICAgICAgID4gICAgICAgICA+IEhleSENCj4gICAgICAgICAgICAgICAgID4gICAgICAg ICA+DQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgPiBTbyB3ZSBoYXZlIHRoaXMgWkZTIHN0 b3JhZ2Ugc2VydmVyDQo+ICAgICAgICAgICAgICAgICB1cGdyYWRlZCBmcm9tIDkuMy1SRUxFQVNF DQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgdG8NCj4gICAgICAgICAgICAgICAgID4gICAg ICAgICA+IDEwLjEtU1RBQkxFIHRvIG92ZXJjb21lIG5vdCBiZWluZyBhYmxlIHRvDQo+ICAgICAg ICAgICAgICAgICAxKSB1c2UgU1NEIGRyaXZlcw0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAg IGFzDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgPiBMMkFSQ1sxXQ0KPiAgICAgICAgICAg ICAgICAgPiAgICAgICAgID4gYW5kIDIpIG5vdCBiZWluZyBhYmxlIHRvIGhvdHN3YXAgU0FUQQ0K PiAgICAgICAgICAgICAgICAgZHJpdmVzWzJdLg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAg ID4NCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IEFmdGVyIHRoZSB1cGdyYWRlIHdlwrR2 ZSBub3RpY2VkIGEgdmVyeSBvZGQNCj4gICAgICAgICAgICAgICAgIG5ldHdvcmtpbmcNCj4gICAg ICAgICAgICAgICAgID4gICAgICAgICBiZWhhdmlvdXIsIGl0DQo+ICAgICAgICAgICAgICAgICA+ ICAgICAgICAgPiBzZW5kcy9yZWNlaXZlcyBmdWxsIHNwZWVkIGZvciBhIHdoaWxlLA0KPiAgICAg ICAgICAgICAgICAgdGhlbiB0aGVyZSBpcyBhDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAg Y291cGxlIG9mDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgPiBtaW51dGVzDQo+ICAgICAg ICAgICAgICAgICA+ICAgICAgICAgPiBvZiBjb21wbGV0ZSBzaWxlbmNlIHdoZXJlIGV2ZW4gdGVy bWluYWwNCj4gICAgICAgICAgICAgICAgIGNvbW1hbmRzIGxpa2UgYW4NCj4gICAgICAgICAgICAg ICAgID4gICAgICAgICAibHMiIGp1c3QNCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IHdh aXRzDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgPiB1bnRpbCB0aGV5IGFyZSBleGVjdXRl ZCBhbmQgdGhlbiBpdCBzdGFydHMNCj4gICAgICAgICAgICAgICAgIHNlbmRpbmcgZnVsbA0KPiAg ICAgICAgICAgICAgICAgPiAgICAgICAgIHNwZWVkIGFnYWluLiBJDQo+ICAgICAgICAgICAgICAg ICA+ICAgICAgICAgPiDCtHZlIGxpbmtlZCB0byBhIHNjcmVlbnNob3Qgc2hvd2luZyB0aGlzDQo+ ICAgICAgICAgICAgICAgICBzZW5kIGFuZCBwYXVzZQ0KPiAgICAgICAgICAgICAgICAgPiAgICAg ICAgIGJlaGF2aW91ci4gVGhlDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgPiBibHVlIGxp bmUgaXMgdGhlIHRvdGFsLCBncmVlbiBpcyBTTUIgYW5kDQo+ICAgICAgICAgICAgICAgICB0dXJx dW9pc2UgaXMgTkZTDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgb3ZlciBqdW1ibw0KPiAg ICAgICAgICAgICAgICAgPiAgICAgICAgID4gZnJhbWVzLiBJdCBiZWhhdmVzIHRoaXMgd2F5IHJl Z2FyZGxlc3Mgb2YNCj4gICAgICAgICAgICAgICAgIHRoZSBwcm90b2NvbC4NCj4gICAgICAgICAg ICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgPiBodHRwOi8v b2k2Mi50aW55cGljLmNvbS8zM3h2amI2LmpwZw0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAg ID4NCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IFRoZSBwcm9ibGVtIGlzIHRoYXQgdGhl c2UgcGF1c2VzIGNhbg0KPiAgICAgICAgICAgICAgICAgc29tZXRpbWVzIGJlIHNvIGxvbmcNCj4g ICAgICAgICAgICAgICAgID4gICAgICAgICB0aGF0DQo+ICAgICAgICAgICAgICAgICA+ICAgICAg ICAgPiBjb25uZWN0aW9ucyBkcm9wLiBMaWtlIHNvbWVvbmUgaXMgY29weWluZw0KPiAgICAgICAg ICAgICAgICAgZmlsZXMgb3ZlciBTTUIgb3INCj4gICAgICAgICAgICAgICAgID4gICAgICAgICBp U0NTSSBhbmQNCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IHN1ZGRlbmx5IHRoZXkgZ2V0 IGFuIGVycm9yIG1lc3NhZ2Ugc2F5aW5nDQo+ICAgICAgICAgICAgICAgICB0aGF0IHRoZSB0cmFu c2Zlcg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgIGZhaWxlZCBhbmQNCj4gICAgICAgICAg ICAgICAgID4gICAgICAgICA+IHRoZXkgaGF2ZSB0byBzdGFydCBvdmVyIHdpdGggdGhlIGZpbGUo cykuDQo+ICAgICAgICAgICAgICAgICBUaGF0wrRzIGhvcnJpYmxlIQ0KPiAgICAgICAgICAgICAg ICAgPiAgICAgICAgID4NCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IFNvIGZhciBORlMg aGFzIHByb3ZlbiB0byBiZSB0aGUgbW9zdA0KPiAgICAgICAgICAgICAgICAgcmVzaWxsaWVudCwg aXTCtHMgc3R1cGlkDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgc2ltcGxlDQo+ICAgICAg ICAgICAgICAgICA+ICAgICAgICAgPiBuYXR1cmUganVzdCB3YWl0cyBhbmQgcmVzdW1lcyB0cmFu c2Zlcg0KPiAgICAgICAgICAgICAgICAgd2hlbiBwYXVzZSBpcyBvdmVyLg0KPiAgICAgICAgICAg ICAgICAgPiAgICAgICAgIEt1ZHVzIGZvcg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgID4g dGhhdC4NCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgICAgICAgICA+ ICAgICAgICAgPiBUaGUgc2VydmVyIGlzIGRyaXZlbiBieSBhIFN1cGVybWljcm8NCj4gICAgICAg ICAgICAgICAgIFg5U1JMLUYsIGEgWGVvbiAxNjIwdjINCj4gICAgICAgICAgICAgICAgID4gICAg ICAgICBhbmQgNjRHQg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgID4gRUNDDQo+ICAgICAg ICAgICAgICAgICA+ICAgICAgICAgPiBSQU0uIFRoZSBoYXJkd2FyZSBoYXMgYmVlbiBydWxlZCBv dXQsIHdlDQo+ICAgICAgICAgICAgICAgICBoYXBwZW5lZCB0byBoYXZlIGENCj4gICAgICAgICAg ICAgICAgID4gICAgICAgICBpZGVudGljYWwNCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+ IE1CDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgPiBhbmQgQ1BVIGx5aW5nIGFyb3VuZCBh bmQgdGhhdCBkaWRuwrR0DQo+ICAgICAgICAgICAgICAgICBpbXByb3ZlIHRoaW5ncy4gV2UgaGF2 ZQ0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgIGFsc28NCj4gICAgICAgICAgICAgICAgID4g ICAgICAgICA+IGluc3RhbGxlZCBhIEludGVsIFBSTyAxMDAvMTAwMCBRdWFkLXBvcnQNCj4gICAg ICAgICAgICAgICAgIGV0aGVybmV0IGFkYXB0ZXIgdG8NCj4gICAgICAgICAgICAgICAgID4gICAg ICAgICB0ZXN0IGlmDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgPiB0aGF0IHdvdWxkIGNo YW5nZSB0aGluZ3MsIGJ1dCBpdCBoYXNuwrR0LA0KPiAgICAgICAgICAgICAgICAgaXQgc3RpbGwg YmVoYXZlcw0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgIHRoaXMgd2F5Lg0KPiAgICAgICAg ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IFRoZSB0 d28gYnVpbHQtaW4gTklDJ3MgYXJlIEludGVsIDgyNTc0TA0KPiAgICAgICAgICAgICAgICAgYW5k IHRoZSBRdWFkLXBvcnQNCj4gICAgICAgICAgICAgICAgID4gICAgICAgICBOSUMncyBhcmUNCj4g ICAgICAgICAgICAgICAgID4gICAgICAgICA+IEludGVsIDgyNTcxRUIsIHNvIGJvdGggZW0oNCkg ZHJpdmVuLiBJDQo+ICAgICAgICAgICAgICAgICBoYXBwZW4gdG8ga25vdyB0aGF0DQo+ICAgICAg ICAgICAgICAgICA+ICAgICAgICAgdGhlIGVtDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAg PiBkcml2ZXINCj4gICAgICAgICAgICAgICAgID4gICAgICAgICA+IGhhcyB1cGRhdGVkIGJldHdl ZW4gOS4zIGFuZCAxMC4xLiBQZXJoYXBzDQo+ICAgICAgICAgICAgICAgICB0aGF0IGlzIHRvIGJs YW1lLA0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgIGJ1dCBJIGhhdmUNCj4gICAgICAgICAg ICAgICAgID4gICAgICAgICA+IG5vIGlkZWEuDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAg Pg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgID4gSXMgdGhlcmUgYW55b25lIHRoYXQgY2Fu IG1ha2Ugc2Vuc2Ugb2YNCj4gICAgICAgICAgICAgICAgIHRoaXM/DQo+ICAgICAgICAgICAgICAg ICA+ICAgICAgICAgPg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgID4gWzFdOg0KPiAgICAg ICAgICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICAgICAgICAgIGh0dHBzOi8vYnVncy5m cmVlYnNkLm9yZy9idWd6aWxsYS9zaG93X2J1Zy5jZ2k/aWQ9MTk3MTY0DQo+ICAgICAgICAgICAg ICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgID4gWzJdOg0KPiAg ICAgICAgICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICAgICAgICAgIGh0dHBzOi8vYnVn cy5mcmVlYnNkLm9yZy9idWd6aWxsYS9zaG93X2J1Zy5jZ2k/aWQ9MTkxMzQ4DQo+ICAgICAgICAg ICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAgID4gL0sNCj4g ICAgICAgICAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAg Pg0KPiAgICAgICAgICAgICAgICAgPg0KPiAgICAgICAgICAgICAgICAgPg0KPiAgICAgICAgICAg ICAgICAgPiAgICAgICAgIEFub3RoZXIgb2JzZXJ2YXRpb24gScK0dmUgbWFkZSBpcyB0aGF0IGR1 cmluZw0KPiAgICAgICAgICAgICAgICAgdGhlc2UgcGF1c2VzLCB0aGUNCj4gICAgICAgICAgICAg ICAgID4gICAgICAgICBlbnRpcmUNCj4gICAgICAgICAgICAgICAgID4gICAgICAgICBzeXN0ZW0g aXMgcHV0IG9uIGhvbGQsIGV2ZW4gWkZTIHNjcnViIHN0b3BzDQo+ICAgICAgICAgICAgICAgICBh bmQgdGhlbiByZXN1bWVzDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgYWZ0ZXIgYQ0KPiAg ICAgICAgICAgICAgICAgPiAgICAgICAgIHdoaWxlLiBMb29raW5nIGluIHRvcCwgdGhlIHN5c3Rl bSBpcw0KPiAgICAgICAgICAgICAgICAgY29tcGxldGx5IGlkbGUuDQo+ICAgICAgICAgICAgICAg ICA+DQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgTm9ybWFsbHkgZHVyaW5nIHNjcnViLCB0 aGUga2VybmVsIGVhdHMNCj4gICAgICAgICAgICAgICAgIDIwLTMwJSBDUFUsIGJ1dCBkdXJpbmcN Cj4gICAgICAgICAgICAgICAgID4gICAgICAgICBhIHBhdXNlLA0KPiAgICAgICAgICAgICAgICAg PiAgICAgICAgIGV2ZW4gdGhlIFtrZXJuZWxdIGdvZXMgZG93biB0byAwLjAwJS4gTWFrZXMNCj4g ICAgICAgICAgICAgICAgIG1lIHRoaW5rIHRoZQ0KPiAgICAgICAgICAgICAgICAgPiAgICAgICAg IG5ldHdvcmtpbmcgaGFzDQo+ICAgICAgICAgICAgICAgICA+ICAgICAgICAgbm90aGluZyB0byBk byB3aXRoIGl0Lg0KPiAgICAgICAgICAgICAgICAgPg0KPiAgICAgICAgICAgICAgICAgPiAgICAg ICAgIFdoYXTCtHMgdGhlbiB0byBibGFtZT8gWkZTPw0KPiAgICAgICAgICAgICAgICAgPg0KPiAg ICAgICAgICAgICAgICAgPiAgICAgICAgIC9LDQo+ICAgICAgICAgICAgICAgICA+DQo+ICAgICAg ICAgICAgICAgICAgX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X18NCj4gICAgICAgICAgICAgICAgID4gICAgICAgICBmcmVlYnNkLWZzQGZyZWVic2Qub3JnIG1h aWxpbmcgbGlzdA0KPiAgICAgICAgICAgICAgICAgPg0KPiAgICAgICAgICAgICAgICAgIGh0dHA6 Ly9saXN0cy5mcmVlYnNkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ZyZWVic2QtZnMNCj4gICAgICAg ICAgICAgICAgID4gICAgICAgICBUbyB1bnN1YnNjcmliZSwgc2VuZCBhbnkgbWFpbCB0bw0KPiAg ICAgICAgICAgICAgICAgPiAgICAgICAgICJmcmVlYnNkLWZzLXVuc3Vic2NyaWJlQGZyZWVic2Qu b3JnIg0KPiAgICAgICAgICAgICAgICAgPg0KPiAgICAgICAgICAgICAgICAgPg0KPiAgICAgICAg ICAgICAgICAgPiBIZWxsbywNCj4gICAgICAgICAgICAgICAgID4NCj4gICAgICAgICAgICAgICAg ID4NCj4gICAgICAgICAgICAgICAgID4gZG9lcyB0aGlzIGhhcHBlbiB3aGVuIGNsaWVudHMgYXJl IG9ubHkgcmVhZGluZyBmcm9tDQo+ICAgICAgICAgICAgICAgICBzZXJ2ZXI/DQo+ICAgICAgICAg ICAgICAgICANCj4gICAgICAgICAgICAgICAgIFllcyBpdCBoYXBwZW5zIHdoZW4gY2xpZW50cyBh cmUgb25seSByZWFkaW5nIGZyb20gdGhlDQo+ICAgICAgICAgICAgICAgICBzZXJ2ZXIuDQo+ICAg ICAgICAgICAgICAgICANCj4gICAgICAgICAgICAgICAgID4gT3RoZXJ3aXNlIEkgd291bGQgc3Vz cGVjdCB0aGF0IGl0IGNvdWxkIGJlIGNhdXNlZCBieQ0KPiAgICAgICAgICAgICAgICAgWkZTIHdy aXRpbmcgb3V0IGENCj4gICAgICAgICAgICAgICAgID4gbGFyZ2UgY2h1bmNrIG9mIGRhdGEgc2l0 dGluZyBpbiBpdHMgY2FjaGVzLCBhbmQNCj4gICAgICAgICAgICAgICAgIHVudGlsIHRoYXQgaXMg Y29tcGxldGUNCj4gICAgICAgICAgICAgICAgID4gSS9PIGlzIHN0YWxsZWQuDQo+ICAgICAgICAg ICAgICAgICANCj4gICAgICAgICAgICAgICAgIFRoYXTCtHMgd2hhdCBzbyBzdHJhbmdlLCB3ZSBo YXZlIHRocmVlIG1vcmUgc3lzdGVtcyBzZXQNCj4gICAgICAgICAgICAgICAgIHVwIGFib3V0IHRo ZSBzYW1lDQo+ICAgICAgICAgICAgICAgICBzaXplIGFuZCBub25lIG9mIG90aGVycyBhcmUgYWN0 aW5nIHRoaXMgd2F5Lg0KPiAgICAgICAgICAgICAgICAgDQo+ICAgICAgICAgICAgICAgICBUaGUg b25seSB0aGluZyBJIGNhbiB0aGluayBvZiB0aGF0IGRpZmZlcnMgdGhhdCB3ZQ0KPiAgICAgICAg ICAgICAgICAgaGF2ZW7CtHQgdGVzdGVkIHJ1bGluZw0KPiAgICAgICAgICAgICAgICAgb3V0IHll dCBpcyBjdGxkLCB0aGUgb3RoZXIgc3lzdGVtcyBhcmUgc3RpbGwgcnVubmluZw0KPiAgICAgICAg ICAgICAgICAgaXN0Z3QgYXMgdGhlaXINCj4gICAgICAgICAgICAgICAgIGlTQ1NJIGRhZW1vbi4N Cj4gICAgICAgICAgICAgICAgIA0KPiAgICAgICAgICAgICAgICAgL0sNCj4gICAgICAgICAgICAg ICAgIA0KPiBXaGF0IGRvZXMgYSB6cG9vbCBzdGF0dXMgc2F5PyBDb3VsZCB2ZXJ5IHdlbGwgYmUg ZGlza3Mgc3RhcnRpbmcgdG8NCj4gZmFpbC4NCj4gDQo+IA0KPiBBbnl0aGluZyBpbiBkbWVzZyBj b25jZXJuaW5nIGNhbSB0aW1lb3V0cz8NCj4gDQo+IA0KPiBCZXN0IHJlZ2FyZHMNCj4gDQo+IEFu ZHJlYXMNCj4gDQoNClBvb2wgc3RhdHVzIGlzIGZpbmUsIHNjcnViYmVkIG11bHRpcGxlIHRpbWVz IHdpdGhvdXQgZXJyb3JzLiBObyBzdG9yYWdlDQpyZWxhdGVkIGVycm9ycywgd2XCtHJlIHVzaW5n IExTSSBIQkEncyBzbyBubyBjYW0sIGJ1dCBub3RoaW5nIG1wcy1yZWxhdGVkDQplaXRoZXIuDQoN Ci9LDQo= From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 10:28:31 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 804B91D0 for ; Mon, 1 Jun 2015 10:28:31 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from EXCH2-1.slu.se (imap.slu.se [77.235.224.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC2E11832 for ; Mon, 1 Jun 2015 10:28:30 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by EXCH2-1.slu.se (77.235.224.121) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Mon, 1 Jun 2015 12:28:10 +0200 Received: from exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b]) by exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b%22]) with mapi id 15.00.1076.000; Mon, 1 Jun 2015 12:28:10 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Mehmet Erol Sanliturk CC: Andreas Nilsson , "freebsd-fs@freebsd.org" Subject: Re: [Fwd: Strange networking behaviour in storage server] Thread-Topic: [Fwd: Strange networking behaviour in storage server] Thread-Index: AQHQnEMH0FmSWuTxD0+w8f7+o424g52XMR6AgAAINICAAA76gIAACQoA Date: Mon, 1 Jun 2015 10:28:10 +0000 Message-ID: <1433154506.14998.192.camel@data-b104.adm.slu.se> References: <1433146506.14998.177.camel@data-b104.adm.slu.se> <1433149349.14998.181.camel@data-b104.adm.slu.se> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.32] Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 10:28:31 -0000 bcOlbiAyMDE1LTA2LTAxIGtsb2NrYW4gMDI6NTYgLTA3MDAgc2tyZXYgTWVobWV0IEVyb2wgU2Fu bGl0dXJrOg0KPiANCj4gDQo+IE9uIE1vbiwgSnVuIDEsIDIwMTUgYXQgMjowMiBBTSwgS2FybGkg U2rDtmJlcmcgPGthcmxpLnNqb2JlcmdAc2x1LnNlPg0KPiB3cm90ZToNCj4gICAgICAgICBtw6Vu IDIwMTUtMDYtMDEga2xvY2thbiAxMDozMyArMDIwMCBza3JldiBBbmRyZWFzIE5pbHNzb246DQo+ ICAgICAgICAgPg0KPiAgICAgICAgID4NCj4gICAgICAgICA+IE9uIE1vbiwgSnVuIDEsIDIwMTUg YXQgMTA6MTQgQU0sIEthcmxpIFNqw7ZiZXJnDQo+ICAgICAgICAgPGthcmxpLnNqb2JlcmdAc2x1 LnNlPg0KPiAgICAgICAgID4gd3JvdGU6DQo+ICAgICAgICAgPiAgICAgICAgIC0tLS0tLS0tIFZp ZGFyZWJlZm9yZHJhdCBtZWRkZWxhbmRlIC0tLS0tLS0tDQo+ICAgICAgICAgPiAgICAgICAgID4g RnLDpW46IEthcmxpIFNqw7ZiZXJnIDxrYXJsaS5zam9iZXJnQHNsdS5zZT4NCj4gICAgICAgICA+ ICAgICAgICAgPiBUaWxsOiBmcmVlYnNkLWZzQGZyZWVic2Qub3JnDQo+ICAgICAgICAgPGZyZWVi c2QtZnNAZnJlZWJzZC5vcmc+DQo+ICAgICAgICAgPiAgICAgICAgID4gw4RtbmU6IFN0cmFuZ2Ug bmV0d29ya2luZyBiZWhhdmlvdXIgaW4gc3RvcmFnZQ0KPiAgICAgICAgIHNlcnZlcg0KPiAgICAg ICAgID4gICAgICAgICA+IERhdHVtOiBNb24sIDEgSnVuIDIwMTUgMDc6NDk6NTYgKzAwMDANCj4g ICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+IEhleSENCj4gICAgICAg ICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+IFNvIHdlIGhhdmUgdGhpcyBaRlMg c3RvcmFnZSBzZXJ2ZXIgdXBncmFkZWQgZnJvbQ0KPiAgICAgICAgIDkuMy1SRUxFQVNFDQo+ICAg ICAgICAgPiAgICAgICAgIHRvDQo+ICAgICAgICAgPiAgICAgICAgID4gMTAuMS1TVEFCTEUgdG8g b3ZlcmNvbWUgbm90IGJlaW5nIGFibGUgdG8gMSkgdXNlDQo+ICAgICAgICAgU1NEIGRyaXZlcw0K PiAgICAgICAgID4gICAgICAgICBhcw0KPiAgICAgICAgID4gICAgICAgICA+IEwyQVJDWzFdDQo+ ICAgICAgICAgPiAgICAgICAgID4gYW5kIDIpIG5vdCBiZWluZyBhYmxlIHRvIGhvdHN3YXAgU0FU QSBkcml2ZXNbMl0uDQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAg PiBBZnRlciB0aGUgdXBncmFkZSB3ZcK0dmUgbm90aWNlZCBhIHZlcnkgb2RkDQo+ICAgICAgICAg bmV0d29ya2luZw0KPiAgICAgICAgID4gICAgICAgICBiZWhhdmlvdXIsIGl0DQo+ICAgICAgICAg PiAgICAgICAgID4gc2VuZHMvcmVjZWl2ZXMgZnVsbCBzcGVlZCBmb3IgYSB3aGlsZSwgdGhlbiB0 aGVyZQ0KPiAgICAgICAgIGlzIGENCj4gICAgICAgICA+ICAgICAgICAgY291cGxlIG9mDQo+ICAg ICAgICAgPiAgICAgICAgID4gbWludXRlcw0KPiAgICAgICAgID4gICAgICAgICA+IG9mIGNvbXBs ZXRlIHNpbGVuY2Ugd2hlcmUgZXZlbiB0ZXJtaW5hbCBjb21tYW5kcw0KPiAgICAgICAgIGxpa2Ug YW4NCj4gICAgICAgICA+ICAgICAgICAgImxzIiBqdXN0DQo+ICAgICAgICAgPiAgICAgICAgID4g d2FpdHMNCj4gICAgICAgICA+ICAgICAgICAgPiB1bnRpbCB0aGV5IGFyZSBleGVjdXRlZCBhbmQg dGhlbiBpdCBzdGFydHMgc2VuZGluZw0KPiAgICAgICAgIGZ1bGwNCj4gICAgICAgICA+ICAgICAg ICAgc3BlZWQgYWdhaW4uIEkNCj4gICAgICAgICA+ICAgICAgICAgPiDCtHZlIGxpbmtlZCB0byBh IHNjcmVlbnNob3Qgc2hvd2luZyB0aGlzIHNlbmQgYW5kDQo+ICAgICAgICAgcGF1c2UNCj4gICAg ICAgICA+ICAgICAgICAgYmVoYXZpb3VyLiBUaGUNCj4gICAgICAgICA+ICAgICAgICAgPiBibHVl IGxpbmUgaXMgdGhlIHRvdGFsLCBncmVlbiBpcyBTTUIgYW5kIHR1cnF1b2lzZQ0KPiAgICAgICAg IGlzIE5GUw0KPiAgICAgICAgID4gICAgICAgICBvdmVyIGp1bWJvDQo+ICAgICAgICAgPiAgICAg ICAgID4gZnJhbWVzLiBJdCBiZWhhdmVzIHRoaXMgd2F5IHJlZ2FyZGxlc3Mgb2YgdGhlDQo+ICAg ICAgICAgcHJvdG9jb2wuDQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAg ICAgPiBodHRwOi8vb2k2Mi50aW55cGljLmNvbS8zM3h2amI2LmpwZw0KPiAgICAgICAgID4gICAg ICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4gVGhlIHByb2JsZW0gaXMgdGhhdCB0aGVzZSBw YXVzZXMgY2FuIHNvbWV0aW1lcyBiZQ0KPiAgICAgICAgIHNvIGxvbmcNCj4gICAgICAgICA+ICAg ICAgICAgdGhhdA0KPiAgICAgICAgID4gICAgICAgICA+IGNvbm5lY3Rpb25zIGRyb3AuIExpa2Ug c29tZW9uZSBpcyBjb3B5aW5nIGZpbGVzDQo+ICAgICAgICAgb3ZlciBTTUIgb3INCj4gICAgICAg ICA+ICAgICAgICAgaVNDU0kgYW5kDQo+ICAgICAgICAgPiAgICAgICAgID4gc3VkZGVubHkgdGhl eSBnZXQgYW4gZXJyb3IgbWVzc2FnZSBzYXlpbmcgdGhhdCB0aGUNCj4gICAgICAgICB0cmFuc2Zl cg0KPiAgICAgICAgID4gICAgICAgICBmYWlsZWQgYW5kDQo+ICAgICAgICAgPiAgICAgICAgID4g dGhleSBoYXZlIHRvIHN0YXJ0IG92ZXIgd2l0aCB0aGUgZmlsZShzKS4gVGhhdMK0cw0KPiAgICAg ICAgIGhvcnJpYmxlIQ0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAg ID4gU28gZmFyIE5GUyBoYXMgcHJvdmVuIHRvIGJlIHRoZSBtb3N0IHJlc2lsbGllbnQsDQo+ICAg ICAgICAgaXTCtHMgc3R1cGlkDQo+ICAgICAgICAgPiAgICAgICAgIHNpbXBsZQ0KPiAgICAgICAg ID4gICAgICAgICA+IG5hdHVyZSBqdXN0IHdhaXRzIGFuZCByZXN1bWVzIHRyYW5zZmVyIHdoZW4g cGF1c2UNCj4gICAgICAgICBpcyBvdmVyLg0KPiAgICAgICAgID4gICAgICAgICBLdWR1cyBmb3IN Cj4gICAgICAgICA+ICAgICAgICAgPiB0aGF0Lg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAg ICAgICAgPiAgICAgICAgID4gVGhlIHNlcnZlciBpcyBkcml2ZW4gYnkgYSBTdXBlcm1pY3JvIFg5 U1JMLUYsIGENCj4gICAgICAgICBYZW9uIDE2MjB2Mg0KPiAgICAgICAgID4gICAgICAgICBhbmQg NjRHQg0KPiAgICAgICAgID4gICAgICAgICA+IEVDQw0KPiAgICAgICAgID4gICAgICAgICA+IFJB TS4gVGhlIGhhcmR3YXJlIGhhcyBiZWVuIHJ1bGVkIG91dCwgd2UgaGFwcGVuZWQNCj4gICAgICAg ICB0byBoYXZlIGENCj4gICAgICAgICA+ICAgICAgICAgaWRlbnRpY2FsDQo+ICAgICAgICAgPiAg ICAgICAgID4gTUINCj4gICAgICAgICA+ICAgICAgICAgPiBhbmQgQ1BVIGx5aW5nIGFyb3VuZCBh bmQgdGhhdCBkaWRuwrR0IGltcHJvdmUNCj4gICAgICAgICB0aGluZ3MuIFdlIGhhdmUNCj4gICAg ICAgICA+ICAgICAgICAgYWxzbw0KPiAgICAgICAgID4gICAgICAgICA+IGluc3RhbGxlZCBhIElu dGVsIFBSTyAxMDAvMTAwMCBRdWFkLXBvcnQgZXRoZXJuZXQNCj4gICAgICAgICBhZGFwdGVyIHRv DQo+ICAgICAgICAgPiAgICAgICAgIHRlc3QgaWYNCj4gICAgICAgICA+ICAgICAgICAgPiB0aGF0 IHdvdWxkIGNoYW5nZSB0aGluZ3MsIGJ1dCBpdCBoYXNuwrR0LCBpdCBzdGlsbA0KPiAgICAgICAg IGJlaGF2ZXMNCj4gICAgICAgICA+ICAgICAgICAgdGhpcyB3YXkuDQo+ICAgICAgICAgPiAgICAg ICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiBUaGUgdHdvIGJ1aWx0LWluIE5JQydzIGFyZSBJ bnRlbCA4MjU3NEwgYW5kIHRoZQ0KPiAgICAgICAgIFF1YWQtcG9ydA0KPiAgICAgICAgID4gICAg ICAgICBOSUMncyBhcmUNCj4gICAgICAgICA+ICAgICAgICAgPiBJbnRlbCA4MjU3MUVCLCBzbyBi b3RoIGVtKDQpIGRyaXZlbi4gSSBoYXBwZW4gdG8NCj4gICAgICAgICBrbm93IHRoYXQNCj4gICAg ICAgICA+ICAgICAgICAgdGhlIGVtDQo+ICAgICAgICAgPiAgICAgICAgID4gZHJpdmVyDQo+ICAg ICAgICAgPiAgICAgICAgID4gaGFzIHVwZGF0ZWQgYmV0d2VlbiA5LjMgYW5kIDEwLjEuIFBlcmhh cHMgdGhhdCBpcw0KPiAgICAgICAgIHRvIGJsYW1lLA0KPiAgICAgICAgID4gICAgICAgICBidXQg SSBoYXZlDQo+ICAgICAgICAgPiAgICAgICAgID4gbm8gaWRlYS4NCj4gICAgICAgICA+ICAgICAg ICAgPg0KPiAgICAgICAgID4gICAgICAgICA+IElzIHRoZXJlIGFueW9uZSB0aGF0IGNhbiBtYWtl IHNlbnNlIG9mIHRoaXM/DQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAg ICAgPiBbMV06DQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICBodHRwczovL2J1Z3Mu ZnJlZWJzZC5vcmcvYnVnemlsbGEvc2hvd19idWcuY2dpP2lkPTE5NzE2NA0KPiAgICAgICAgID4g ICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4gWzJdOg0KPiAgICAgICAgID4gICAgICAg ICA+DQo+ICAgICAgICAgaHR0cHM6Ly9idWdzLmZyZWVic2Qub3JnL2J1Z3ppbGxhL3Nob3dfYnVn LmNnaT9pZD0xOTEzNDgNCj4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAg ICA+IC9LDQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPg0KPiAg ICAgICAgID4NCj4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgIEFub3RoZXIgb2JzZXJ2 YXRpb24gScK0dmUgbWFkZSBpcyB0aGF0IGR1cmluZyB0aGVzZQ0KPiAgICAgICAgIHBhdXNlcywg dGhlDQo+ICAgICAgICAgPiAgICAgICAgIGVudGlyZQ0KPiAgICAgICAgID4gICAgICAgICBzeXN0 ZW0gaXMgcHV0IG9uIGhvbGQsIGV2ZW4gWkZTIHNjcnViIHN0b3BzIGFuZCB0aGVuDQo+ICAgICAg ICAgcmVzdW1lcw0KPiAgICAgICAgID4gICAgICAgICBhZnRlciBhDQo+ICAgICAgICAgPiAgICAg ICAgIHdoaWxlLiBMb29raW5nIGluIHRvcCwgdGhlIHN5c3RlbSBpcyBjb21wbGV0bHkgaWRsZS4N Cj4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgIE5vcm1hbGx5IGR1cmluZyBzY3J1Yiwg dGhlIGtlcm5lbCBlYXRzIDIwLTMwJSBDUFUsDQo+ICAgICAgICAgYnV0IGR1cmluZw0KPiAgICAg ICAgID4gICAgICAgICBhIHBhdXNlLA0KPiAgICAgICAgID4gICAgICAgICBldmVuIHRoZSBba2Vy bmVsXSBnb2VzIGRvd24gdG8gMC4wMCUuIE1ha2VzIG1lIHRoaW5rDQo+ICAgICAgICAgdGhlDQo+ ICAgICAgICAgPiAgICAgICAgIG5ldHdvcmtpbmcgaGFzDQo+ICAgICAgICAgPiAgICAgICAgIG5v dGhpbmcgdG8gZG8gd2l0aCBpdC4NCj4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgIFdo YXTCtHMgdGhlbiB0byBibGFtZT8gWkZTPw0KPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAg ICAgL0sNCj4gICAgICAgICA+ICAgICAgICAgX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18NCj4gICAgICAgICA+ICAgICAgICAgZnJlZWJzZC1mc0BmcmVlYnNk Lm9yZyBtYWlsaW5nIGxpc3QNCj4gICAgICAgICA+ICAgICAgICAgaHR0cDovL2xpc3RzLmZyZWVi c2Qub3JnL21haWxtYW4vbGlzdGluZm8vZnJlZWJzZC1mcw0KPiAgICAgICAgID4gICAgICAgICBU byB1bnN1YnNjcmliZSwgc2VuZCBhbnkgbWFpbCB0bw0KPiAgICAgICAgID4gICAgICAgICAiZnJl ZWJzZC1mcy11bnN1YnNjcmliZUBmcmVlYnNkLm9yZyINCj4gICAgICAgICA+DQo+ICAgICAgICAg Pg0KPiAgICAgICAgID4gSGVsbG8sDQo+ICAgICAgICAgPg0KPiAgICAgICAgID4NCj4gICAgICAg ICA+IGRvZXMgdGhpcyBoYXBwZW4gd2hlbiBjbGllbnRzIGFyZSBvbmx5IHJlYWRpbmcgZnJvbSBz ZXJ2ZXI/DQo+ICAgICAgICAgDQo+ICAgICAgICAgWWVzIGl0IGhhcHBlbnMgd2hlbiBjbGllbnRz IGFyZSBvbmx5IHJlYWRpbmcgZnJvbSB0aGUgc2VydmVyLg0KPiAgICAgICAgIA0KPiAgICAgICAg ID4gT3RoZXJ3aXNlIEkgd291bGQgc3VzcGVjdCB0aGF0IGl0IGNvdWxkIGJlIGNhdXNlZCBieSBa RlMNCj4gICAgICAgICB3cml0aW5nIG91dCBhDQo+ICAgICAgICAgPiBsYXJnZSBjaHVuY2sgb2Yg ZGF0YSBzaXR0aW5nIGluIGl0cyBjYWNoZXMsIGFuZCB1bnRpbCB0aGF0DQo+ICAgICAgICAgaXMg Y29tcGxldGUNCj4gICAgICAgICA+IEkvTyBpcyBzdGFsbGVkLg0KPiAgICAgICAgIA0KPiAgICAg ICAgIFRoYXTCtHMgd2hhdCBzbyBzdHJhbmdlLCB3ZSBoYXZlIHRocmVlIG1vcmUgc3lzdGVtcyBz ZXQgdXANCj4gICAgICAgICBhYm91dCB0aGUgc2FtZQ0KPiAgICAgICAgIHNpemUgYW5kIG5vbmUg b2Ygb3RoZXJzIGFyZSBhY3RpbmcgdGhpcyB3YXkuDQo+ICAgICAgICAgDQo+ICAgICAgICAgVGhl IG9ubHkgdGhpbmcgSSBjYW4gdGhpbmsgb2YgdGhhdCBkaWZmZXJzIHRoYXQgd2UgaGF2ZW7CtHQN Cj4gICAgICAgICB0ZXN0ZWQgcnVsaW5nDQo+ICAgICAgICAgb3V0IHlldCBpcyBjdGxkLCB0aGUg b3RoZXIgc3lzdGVtcyBhcmUgc3RpbGwgcnVubmluZyBpc3RndCBhcw0KPiAgICAgICAgIHRoZWly DQo+ICAgICAgICAgaVNDU0kgZGFlbW9uLg0KPiAgICAgICAgIA0KPiAgICAgICAgIC9LDQo+ICAg ICAgICAgDQo+IA0KPiANCj4gDQo+IElmIHRoZXJlIGFyZSBvdGhlciB0aHJlZSBzaW1pbGFyIHN5 c3RlbXMgYW5kIHRoZXkgYXJlIGV4YWN0bHkNCj4gaW5zdGFsbGVkIHdpdGggdGhlIHNhbWUgc3Ry dWN0dXJlICwgbXkgZmlyc3QgcG9zc2liaWxpdHkgdG8gY29uc2lkZXINCj4gd291bGQgYmUgdG8g c3VzcGVjdCBhIHNsb3dseSBwcm9ncmVzc2luZyBoYXJkd2FyZSBmYWlsdXJlIDoNCj4gDQo+IA0K PiBGcm9tIGEgY2lyY3VpdCAsIGl0IGlzIG5vdCBwb3NzaWJsZSB0byBnZXQgYSByZXNwb25zZSBp biBleHBlY3RlZA0KPiB0aW1lICwgYnV0ICwgaXQgaXMgcmVzcG9uZGluZyBhZnRlciBhIHRpbWUg d2hpY2ggaXMgbm90IG5vcm1hbCAuIFN1Y2gNCj4gYW4gYWN0aW9uIG1heSBiZSBjYXVzZWQgYnkg YSBmYXVsdHkgc29sZGVyZWQgb3IgY3JhY2tlZCBsaW5lIHBvaW50IGluDQo+IHRoZSBjaXJjdWl0 IDogV2hlbiBpdCBpcyBob3QgLCBpdCBpcyBkaXNjb25uZWN0aW5nICwgd2hlbiBpdCBpcyBjb2xk DQo+IGl0IGlzIGNvbm5lY3RpbmcgLg0KDQpBcyBpbml0aWFsbHkgc3RhdGVkLCBib3RoIG1vdGhl cmJvYXJkIGFuZCBwcm9jZXNzb3IgaGFzIGJlZW4gcmVwbGFjZWQNCndpdGggaWRlbnRpY2FsIGhh cmR3YXJlIHRoYXQgd2VudCB0aHJvdWdoIGEgZGF5IG9mIG1lbXRlc3QgYmVmb3JlIGJlaW5nDQpp bnN0YWxsZWQuIFRoZW4gdGhlcmXCtHMgYW4gZXh0ZXJuYWwgU3VwZXJtaWNybyBKQk9EWypdIGJ1 dCBJIGhhdmVuwrR0DQpzZWVuIGFueSBkaXNrIHRpbWVvdXRzIG9yIFNFUyBlcnJvcnMgbG9nZ2Vk LiBBdCBsZWFzdCBhdCBhIGRyaXZlciBsZXZlbA0KdGhlcmUgc2hvdWxkIGhhdmUgYmVlbiB0aW1l b3V0cyBhdCBzdWNoIGEgbG9uZyBkZWxheSBhcyBmaXZlIG1pbnV0ZXMuDQoNCi9LDQoNClsqXToN Cmh0dHA6Ly93d3cuc3VwZXJtaWNyby5ubC9wcm9kdWN0cy9jaGFzc2lzLzNVLzgzNy9TQzgzN0Uy Ni1SSkJPRDEuY2ZtDQoNCj4gDQo+IA0KPiANCj4gDQo+IA0KPiBUaGFuayB5b3UgdmVyeSBtdWNo IC4NCj4gDQo+IA0KPiANCj4gTWVobWV0IEVyb2wgU2FubGl0dXJrDQo+IA0KPiANCj4gDQo+IA0K PiAgDQo+ICAgICAgICAgPg0KPiAgICAgICAgID4NCj4gICAgICAgICA+IEhhdmUgeW91IHRyaWVk IHdoYXQgaXMgc3VnZ2VzdGVkIGluDQo+ICAgICAgICAgPiBodHRwczovL3dpa2kuZnJlZWJzZC5v cmcvWkZTVHVuaW5nR3VpZGUgPyBJbiBwYXJ0aWN1bGFyDQo+ICAgICAgICAgc2V0dGluZw0KPiAg ICAgICAgID4gdmZzLnpmcy53cml0ZV9saW1pdF9vdmVycmlkZSB0byBzb21ldGhpbmcgYXBwcm9w cmlhdGUgZm9yDQo+ICAgICAgICAgeW91ciBzaXRlLg0KPiAgICAgICAgID4gVGhlIHRpbWVvdXQg c2VlbXMgdG8gYmUgZGVmYXVsdGluZyB0byA1IG5vdy4NCj4gICAgICAgICA+DQo+ICAgICAgICAg Pg0KPiAgICAgICAgID4gQmVzdCByZWdhcmRzDQo+ICAgICAgICAgPg0KPiAgICAgICAgID4gQW5k cmVhcw0KPiAgICAgICAgID4NCj4gICAgICAgICA+DQo+ICAgICAgICAgPg0KPiAgICAgICAgIA0K PiAgICAgICAgIF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f DQo+ICAgICAgICAgZnJlZWJzZC1mc0BmcmVlYnNkLm9yZyBtYWlsaW5nIGxpc3QNCj4gICAgICAg ICBodHRwOi8vbGlzdHMuZnJlZWJzZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLWZzDQo+ ICAgICAgICAgVG8gdW5zdWJzY3JpYmUsIHNlbmQgYW55IG1haWwgdG8NCj4gICAgICAgICAiZnJl ZWJzZC1mcy11bnN1YnNjcmliZUBmcmVlYnNkLm9yZyINCj4gDQo+IA0KDQo= From owner-freebsd-fs@FreeBSD.ORG Mon Jun 1 10:48:42 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 38E39952 for ; Mon, 1 Jun 2015 10:48:42 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DACF61CE4 for ; Mon, 1 Jun 2015 10:48:40 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id 24BA9200057; Mon, 1 Jun 2015 12:41:34 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 16ADD405889; Mon, 1 Jun 2015 12:41:34 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id EFFA3406AF1; Mon, 1 Jun 2015 12:41:33 +0200 (CEST) Received: from arc.aei.uni-hannover.de ([10.117.15.110]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6HF1691) with ESMTP id 2015060112412329-19165 ; Mon, 1 Jun 2015 12:41:23 +0200 Date: Mon, 1 Jun 2015 12:41:23 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Karli =?ISO-8859-1?Q?Sj=F6berg?= Cc: "freebsd-fs@freebsd.org" Subject: Re: [Fwd: Strange networking behaviour in storage server] Message-Id: <20150601124123.d384c1b0028f412fe6fde966@aei.mpg.de> In-Reply-To: <1433153917.14998.185.camel@data-b104.adm.slu.se> References: <1433146506.14998.177.camel@data-b104.adm.slu.se> <1433149349.14998.181.camel@data-b104.adm.slu.se> <1433153917.14998.185.camel@data-b104.adm.slu.se> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6HF1691 | May 7, 2015) at 01.06.2015 12:41:23, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6HF1691 | May 7, 2015) at 01.06.2015 12:41:33, Serialize complete at 01.06.2015 12:41:33 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1 X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2015.6.1.103315 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1000_LESS 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_300_399 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, REFERENCES 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __REFERENCES 0, __SANE_MSGID 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 10:48:42 -0000 On Mon, 1 Jun 2015 10:18:20 +0000 Karli Sj=F6berg wrote about Re: [Fwd: Strange networking behaviour in storage server]: KS> Pool status is fine, scrubbed multiple times without errors. No storage KS> related errors, we=B4re using LSI HBA's so no cam, but nothing KS> mps-related either. Same firmware (which?) on all controllers? cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Tue Jun 2 04:32:24 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF04C9D2 for ; Tue, 2 Jun 2015 04:32:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B82F514CE for ; Tue, 2 Jun 2015 04:32:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t524WOcK088893 for ; Tue, 2 Jun 2015 04:32:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Tue, 02 Jun 2015 04:32:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to bug_severity Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2015 04:32:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 Garrett Cooper,425-314-3911 changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org Severity|Affects Only Me |Affects Some People -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 2 04:56:23 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F1ABD5C for ; Tue, 2 Jun 2015 04:56:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 88E0C195D for ; Tue, 2 Jun 2015 04:56:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t524uN4g015694 for ; Tue, 2 Jun 2015 04:56:23 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Tue, 02 Jun 2015 04:56:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2015 04:56:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 --- Comment #1 from Garrett Cooper,425-314-3911 --- Created attachment 157364 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=157364&action=edit This LOR might be interesting... -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 2 05:56:58 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A6F1474 for ; Tue, 2 Jun 2015 05:56:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 22EE1170F for ; Tue, 2 Jun 2015 05:56:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t525uwwh013172 for ; Tue, 2 Jun 2015 05:56:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Tue, 02 Jun 2015 05:56:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: delphij@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2015 05:56:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 Xin LI changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |delphij@FreeBSD.org --- Comment #2 from Xin LI --- I think the LOR may be unrelated. This happens because umount -f (nfs_unmount in sys/fs/nfsclient/nfs_clvfsops.c:1522) free's the mountpoint structure (1579) but nlm have kept a reference to it. In order to fix this I think we would need a way similar to newnfs_nmcancelreqs() that cancells all lockd RPCs that are associated with the mountpoint (i.e. abort all lock requests that are related to the unmounting mountpoint) which is probably non-trivial with the current structure. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 2 08:48:38 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E541B986 for ; Tue, 2 Jun 2015 08:48:37 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (pop.slu.se [77.235.224.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47FD61010 for ; Tue, 2 Jun 2015 08:48:36 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by exch2-4.slu.se (77.235.224.124) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Tue, 2 Jun 2015 10:33:01 +0200 Received: from exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b]) by exch2-4.slu.se ([fe80::3117:818f:aa48:9d9b%22]) with mapi id 15.00.1076.000; Tue, 2 Jun 2015 10:33:01 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Andreas Nilsson CC: "freebsd-fs@freebsd.org" Subject: Re: [Fwd: Strange networking behaviour in storage server] Thread-Topic: [Fwd: Strange networking behaviour in storage server] Thread-Index: AQHQnEMH0FmSWuTxD0+w8f7+o424g52XMR6AgAAINICAAA76gIAACQoAgAAHGoCAAAJBgIABYl0AgAAGcQA= Date: Tue, 2 Jun 2015 08:33:00 +0000 Message-ID: <1433233998.27174.4.camel@data-b104.adm.slu.se> References: <1433146506.14998.177.camel@data-b104.adm.slu.se> <1433149349.14998.181.camel@data-b104.adm.slu.se> <1433154506.14998.192.camel@data-b104.adm.slu.se> <1433156515.14998.194.camel@data-b104.adm.slu.se> In-Reply-To: Accept-Language: sv-SE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [77.235.228.32] Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2015 08:48:38 -0000 VmV0IGludGUgdmFyZsO2ciBkdSBza3JpdmVyIGVuZ2Vsc2thIG7DpHIgZGV0IGJhcmEgw6RyIG1l bGxhbiBvc3MuLi4gS2Fuc2tlDQpnbMO2bWRlIHN2YXJhIGFsbGE/DQoNCnRpcyAyMDE1LTA2LTAy IGtsb2NrYW4gMTA6MTAgKzAyMDAgc2tyZXYgQW5kcmVhcyBOaWxzc29uOg0KPiBObywgbWJ1ZnMg c2hvdWxkIG5vdCBlZmZlY3QgYSBzY3J1Yi4NCj4gDQo+IA0KPiBZb3UgY2FuIGdldCBzb21lIHN0 YXRzIGZyb20gdm1zdGF0IC16DQoNCkhtbSwgd2hhdCB3b3VsZCBpIGJlIGxvb2tpbmcgZm9yIGV4 YWN0bHk/DQoNCj4gDQo+IA0KPiBIYXZlIGhhZCBhIHN5c3RhdCBydW5uaW5nIHdoaWxlIElPIHN0 YWxscz8NCg0KU2FtZSBhcyBhYm92ZS4NCg0KPiANCj4gDQo+IEFsc28sIHpwb29sIGhhcyB0dW5h YmxlIGZvciBmYWlsbW9kZSwgd2hpY2ggZGVmYXVsdHMgdG8gd2FpdCwgYnV0IGFzDQo+IHlvdSBz YXkgc2NydWIvenBvb2wgc3RhdHVzIGluZGljYXRlcyBubyBlcnJvcnMgdGhpcyBpcyB1bmxpa2Vs eSB0aGUNCj4gY2F1c2UuDQo+IA0KPiANCj4gDQo+IE90aGVyIHRoYW4gdGhhdCBJJ20gb3V0IG9m IGlkZWFzIDooDQoNCldlIGhhdmUgdGhhdCBpbiBjb21tb246KQ0KDQovSw0KDQo+IA0KPiANCj4g QmVzdCByZWdhcmRzDQo+IA0KPiBBbmRyZWFzDQo+IA0KPiANCj4gT24gTW9uLCBKdW4gMSwgMjAx NSBhdCAxOjAxIFBNLCBLYXJsaSBTasO2YmVyZyA8a2FybGkuc2pvYmVyZ0BzbHUuc2U+DQo+IHdy b3RlOg0KPiAgICAgICAgIG3DpW4gMjAxNS0wNi0wMSBrbG9ja2FuIDEyOjUzICswMjAwIHNrcmV2 IEFuZHJlYXMgTmlsc3NvbjoNCj4gICAgICAgICA+IEludGVyZXN0aW5nLg0KPiAgICAgICAgID4N Cj4gICAgICAgICA+DQo+ICAgICAgICAgPiBPdXQgb2YgbWJ1ZnMgcGVyaGFwcz8NCj4gICAgICAg ICANCj4gICAgICAgICBIbW0sIHdoeSB3b3VsZCBkZXBsZXRlZCBtYnVmcyBzdGFsbCBldmVuIGEg c2NydWI/DQo+ICAgICAgICAgDQo+ICAgICAgICAgSG93IHdvdWxkIEkgdmVyaWZ5IHRoYXQ/DQo+ ICAgICAgICAgDQo+ICAgICAgICAgL0sNCj4gICAgICAgICANCj4gICAgICAgICA+DQo+ICAgICAg ICAgPg0KPiAgICAgICAgID4gL0ENCj4gICAgICAgICA+DQo+ICAgICAgICAgPg0KPiAgICAgICAg ID4gT24gTW9uLCBKdW4gMSwgMjAxNSBhdCAxMjoyOCBQTSwgS2FybGkgU2rDtmJlcmcNCj4gICAg ICAgICA8a2FybGkuc2pvYmVyZ0BzbHUuc2U+DQo+ICAgICAgICAgPiB3cm90ZToNCj4gICAgICAg ICA+ICAgICAgICAgbcOlbiAyMDE1LTA2LTAxIGtsb2NrYW4gMDI6NTYgLTA3MDAgc2tyZXYgTWVo bWV0IEVyb2wNCj4gICAgICAgICA+ICAgICAgICAgU2FubGl0dXJrOg0KPiAgICAgICAgID4gICAg ICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiBPbiBN b24sIEp1biAxLCAyMDE1IGF0IDI6MDIgQU0sIEthcmxpIFNqw7ZiZXJnDQo+ICAgICAgICAgPiAg ICAgICAgIDxrYXJsaS5zam9iZXJnQHNsdS5zZT4NCj4gICAgICAgICA+ICAgICAgICAgPiB3cm90 ZToNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIG3DpW4gMjAxNS0wNi0wMSBrbG9ja2Fu IDEwOjMzICswMjAwIHNrcmV2DQo+ICAgICAgICAgQW5kcmVhcw0KPiAgICAgICAgID4gICAgICAg ICBOaWxzc29uOg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4g ICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiBPbiBN b24sIEp1biAxLCAyMDE1IGF0IDEwOjE0IEFNLCBLYXJsaQ0KPiAgICAgICAgIFNqw7ZiZXJnDQo+ ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA8a2FybGkuc2pvYmVyZ0BzbHUuc2U+DQo+ICAg ICAgICAgPiAgICAgICAgID4gICAgICAgICA+IHdyb3RlOg0KPiAgICAgICAgID4gICAgICAgICA+ ICAgICAgICAgPiAgICAgICAgIC0tLS0tLS0tIFZpZGFyZWJlZm9yZHJhdA0KPiAgICAgICAgIG1l ZGRlbGFuZGUNCj4gICAgICAgICA+ICAgICAgICAgLS0tLS0tLS0NCj4gICAgICAgICA+ICAgICAg ICAgPiAgICAgICAgID4gICAgICAgICA+IEZyw6VuOiBLYXJsaSBTasO2YmVyZw0KPiAgICAgICAg ID4gICAgICAgICA8a2FybGkuc2pvYmVyZ0BzbHUuc2U+DQo+ICAgICAgICAgPiAgICAgICAgID4g ICAgICAgICA+ICAgICAgICAgPiBUaWxsOiBmcmVlYnNkLWZzQGZyZWVic2Qub3JnDQo+ICAgICAg ICAgPiAgICAgICAgID4gICAgICAgICA8ZnJlZWJzZC1mc0BmcmVlYnNkLm9yZz4NCj4gICAgICAg ICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+IMOEbW5lOiBTdHJhbmdlIG5ldHdvcmtp bmcNCj4gICAgICAgICBiZWhhdmlvdXIgaW4NCj4gICAgICAgICA+ICAgICAgICAgc3RvcmFnZQ0K PiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgc2VydmVyDQo+ICAgICAgICAgPiAgICAgICAg ID4gICAgICAgICA+ICAgICAgICAgPiBEYXR1bTogTW9uLCAxIEp1biAyMDE1DQo+ICAgICAgICAg MDc6NDk6NTYgKzAwMDANCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiBIZXkhDQo+ICAgICAg ICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+ ICAgICAgICAgPiAgICAgICAgID4gU28gd2UgaGF2ZSB0aGlzIFpGUyBzdG9yYWdlDQo+ICAgICAg ICAgc2VydmVyDQo+ICAgICAgICAgPiAgICAgICAgIHVwZ3JhZGVkIGZyb20NCj4gICAgICAgICA+ ICAgICAgICAgPiAgICAgICAgIDkuMy1SRUxFQVNFDQo+ICAgICAgICAgPiAgICAgICAgID4gICAg ICAgICA+ICAgICAgICAgdG8NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAg ICA+IDEwLjEtU1RBQkxFIHRvIG92ZXJjb21lIG5vdA0KPiAgICAgICAgIGJlaW5nIGFibGUNCj4g ICAgICAgICA+ICAgICAgICAgdG8gMSkgdXNlDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAg ICBTU0QgZHJpdmVzDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgYXMN Cj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+IEwyQVJDWzFdDQo+ICAg ICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiBhbmQgMikgbm90IGJlaW5nIGFi bGUgdG8NCj4gICAgICAgICBob3Rzd2FwIFNBVEENCj4gICAgICAgICA+ICAgICAgICAgZHJpdmVz WzJdLg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAg ICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+IEFmdGVyIHRoZSB1cGdyYWRlIHdlwrR2 ZQ0KPiAgICAgICAgIG5vdGljZWQgYSB2ZXJ5DQo+ICAgICAgICAgPiAgICAgICAgIG9kZA0KPiAg ICAgICAgID4gICAgICAgICA+ICAgICAgICAgbmV0d29ya2luZw0KPiAgICAgICAgID4gICAgICAg ICA+ICAgICAgICAgPiAgICAgICAgIGJlaGF2aW91ciwgaXQNCj4gICAgICAgICA+ICAgICAgICAg PiAgICAgICAgID4gICAgICAgICA+IHNlbmRzL3JlY2VpdmVzIGZ1bGwgc3BlZWQgZm9yDQo+ICAg ICAgICAgYSB3aGlsZSwNCj4gICAgICAgICA+ICAgICAgICAgdGhlbiB0aGVyZQ0KPiAgICAgICAg ID4gICAgICAgICA+ICAgICAgICAgaXMgYQ0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAg PiAgICAgICAgIGNvdXBsZSBvZg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAg ICAgID4gbWludXRlcw0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4g b2YgY29tcGxldGUgc2lsZW5jZSB3aGVyZSBldmVuDQo+ICAgICAgICAgdGVybWluYWwNCj4gICAg ICAgICA+ICAgICAgICAgY29tbWFuZHMNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIGxp a2UgYW4NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICAibHMiIGp1c3QN Cj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+IHdhaXRzDQo+ICAgICAg ICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiB1bnRpbCB0aGV5IGFyZSBleGVjdXRl ZCBhbmQNCj4gICAgICAgICB0aGVuIGl0DQo+ICAgICAgICAgPiAgICAgICAgIHN0YXJ0cyBzZW5k aW5nDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICBmdWxsDQo+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICA+ICAgICAgICAgc3BlZWQgYWdhaW4uIEkNCj4gICAgICAgICA+ICAgICAg ICAgPiAgICAgICAgID4gICAgICAgICA+IMK0dmUgbGlua2VkIHRvIGEgc2NyZWVuc2hvdA0KPiAg ICAgICAgIHNob3dpbmcgdGhpcw0KPiAgICAgICAgID4gICAgICAgICBzZW5kIGFuZA0KPiAgICAg ICAgID4gICAgICAgICA+ICAgICAgICAgcGF1c2UNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICBiZWhhdmlvdXIuIFRoZQ0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAg ICAgPiAgICAgICAgID4gYmx1ZSBsaW5lIGlzIHRoZSB0b3RhbCwgZ3JlZW4NCj4gICAgICAgICBp cyBTTUIgYW5kDQo+ICAgICAgICAgPiAgICAgICAgIHR1cnF1b2lzZQ0KPiAgICAgICAgID4gICAg ICAgICA+ICAgICAgICAgaXMgTkZTDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAg ICAgICAgb3ZlciBqdW1ibw0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAg ID4gZnJhbWVzLiBJdCBiZWhhdmVzIHRoaXMgd2F5DQo+ICAgICAgICAgcmVnYXJkbGVzcw0KPiAg ICAgICAgID4gICAgICAgICBvZiB0aGUNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIHBy b3RvY29sLg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAg ICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgaHR0cDovL29p NjIudGlueXBpYy5jb20vMzN4dmpiNi5qcGcNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAg ID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiBU aGUgcHJvYmxlbSBpcyB0aGF0IHRoZXNlDQo+ICAgICAgICAgcGF1c2VzIGNhbg0KPiAgICAgICAg ID4gICAgICAgICBzb21ldGltZXMgYmUNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIHNv IGxvbmcNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICB0aGF0DQo+ICAg ICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiBjb25uZWN0aW9ucyBkcm9wLiBM aWtlIHNvbWVvbmUNCj4gICAgICAgICBpcw0KPiAgICAgICAgID4gICAgICAgICBjb3B5aW5nIGZp bGVzDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICBvdmVyIFNNQiBvcg0KPiAgICAgICAg ID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIGlTQ1NJIGFuZA0KPiAgICAgICAgID4gICAg ICAgICA+ICAgICAgICAgPiAgICAgICAgID4gc3VkZGVubHkgdGhleSBnZXQgYW4gZXJyb3INCj4g ICAgICAgICBtZXNzYWdlDQo+ICAgICAgICAgPiAgICAgICAgIHNheWluZyB0aGF0IHRoZQ0KPiAg ICAgICAgID4gICAgICAgICA+ICAgICAgICAgdHJhbnNmZXINCj4gICAgICAgICA+ICAgICAgICAg PiAgICAgICAgID4gICAgICAgICBmYWlsZWQgYW5kDQo+ICAgICAgICAgPiAgICAgICAgID4gICAg ICAgICA+ICAgICAgICAgPiB0aGV5IGhhdmUgdG8gc3RhcnQgb3ZlciB3aXRoDQo+ICAgICAgICAg dGhlDQo+ICAgICAgICAgPiAgICAgICAgIGZpbGUocykuIFRoYXTCtHMNCj4gICAgICAgICA+ICAg ICAgICAgPiAgICAgICAgIGhvcnJpYmxlIQ0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAg PiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+IFNv IGZhciBORlMgaGFzIHByb3ZlbiB0byBiZQ0KPiAgICAgICAgIHRoZSBtb3N0DQo+ICAgICAgICAg PiAgICAgICAgIHJlc2lsbGllbnQsDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICBpdMK0 cyBzdHVwaWQNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICBzaW1wbGUN Cj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+IG5hdHVyZSBqdXN0IHdh aXRzIGFuZCByZXN1bWVzDQo+ICAgICAgICAgdHJhbnNmZXINCj4gICAgICAgICA+ICAgICAgICAg d2hlbiBwYXVzZQ0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgaXMgb3Zlci4NCj4gICAg ICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICBLdWR1cyBmb3INCj4gICAgICAgICA+ ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+IHRoYXQuDQo+ICAgICAgICAgPiAgICAgICAg ID4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAg ICAgICAgID4gVGhlIHNlcnZlciBpcyBkcml2ZW4gYnkgYQ0KPiAgICAgICAgIFN1cGVybWljcm8N Cj4gICAgICAgICA+ICAgICAgICAgWDlTUkwtRiwgYQ0KPiAgICAgICAgID4gICAgICAgICA+ICAg ICAgICAgWGVvbiAxNjIwdjINCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAg ICBhbmQgNjRHQg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gRUND DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiBSQU0uIFRoZSBoYXJk d2FyZSBoYXMgYmVlbg0KPiAgICAgICAgIHJ1bGVkIG91dCwgd2UNCj4gICAgICAgICA+ICAgICAg ICAgaGFwcGVuZWQNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIHRvIGhhdmUgYQ0KPiAg ICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIGlkZW50aWNhbA0KPiAgICAgICAg ID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gTUINCj4gICAgICAgICA+ICAgICAgICAg PiAgICAgICAgID4gICAgICAgICA+IGFuZCBDUFUgbHlpbmcgYXJvdW5kIGFuZCB0aGF0DQo+ICAg ICAgICAgZGlkbsK0dA0KPiAgICAgICAgID4gICAgICAgICBpbXByb3ZlDQo+ICAgICAgICAgPiAg ICAgICAgID4gICAgICAgICB0aGluZ3MuIFdlIGhhdmUNCj4gICAgICAgICA+ICAgICAgICAgPiAg ICAgICAgID4gICAgICAgICBhbHNvDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAg ICAgICAgPiBpbnN0YWxsZWQgYSBJbnRlbCBQUk8gMTAwLzEwMDANCj4gICAgICAgICBRdWFkLXBv cnQNCj4gICAgICAgICA+ICAgICAgICAgZXRoZXJuZXQNCj4gICAgICAgICA+ICAgICAgICAgPiAg ICAgICAgIGFkYXB0ZXIgdG8NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAg ICB0ZXN0IGlmDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiB0aGF0 IHdvdWxkIGNoYW5nZSB0aGluZ3MsIGJ1dA0KPiAgICAgICAgIGl0IGhhc27CtHQsDQo+ICAgICAg ICAgPiAgICAgICAgIGl0IHN0aWxsDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICBiZWhh dmVzDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgdGhpcyB3YXkuDQo+ ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAg ICAgICA+ICAgICAgICAgPiAgICAgICAgID4gVGhlIHR3byBidWlsdC1pbiBOSUMncyBhcmUNCj4g ICAgICAgICBJbnRlbCA4MjU3NEwNCj4gICAgICAgICA+ICAgICAgICAgYW5kIHRoZQ0KPiAgICAg ICAgID4gICAgICAgICA+ICAgICAgICAgUXVhZC1wb3J0DQo+ICAgICAgICAgPiAgICAgICAgID4g ICAgICAgICA+ICAgICAgICAgTklDJ3MgYXJlDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAg ICA+ICAgICAgICAgPiBJbnRlbCA4MjU3MUVCLCBzbyBib3RoIGVtKDQpDQo+ICAgICAgICAgZHJp dmVuLiBJDQo+ICAgICAgICAgPiAgICAgICAgIGhhcHBlbiB0bw0KPiAgICAgICAgID4gICAgICAg ICA+ICAgICAgICAga25vdyB0aGF0DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAg ICAgICAgdGhlIGVtDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiBk cml2ZXINCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+IGhhcyB1cGRh dGVkIGJldHdlZW4gOS4zIGFuZA0KPiAgICAgICAgIDEwLjEuDQo+ICAgICAgICAgPiAgICAgICAg IFBlcmhhcHMgdGhhdCBpcw0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgdG8gYmxhbWUs DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgYnV0IEkgaGF2ZQ0KPiAg ICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gbm8gaWRlYS4NCj4gICAgICAg ICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4g ICAgICAgICA+ICAgICAgICAgPiBJcyB0aGVyZSBhbnlvbmUgdGhhdCBjYW4gbWFrZQ0KPiAgICAg ICAgIHNlbnNlIG9mDQo+ICAgICAgICAgPiAgICAgICAgIHRoaXM/DQo+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAg PiAgICAgICAgID4gWzFdOg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAg ID4NCj4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4NCj4gICAgICAgICBodHRwczov L2J1Z3MuZnJlZWJzZC5vcmcvYnVnemlsbGEvc2hvd19idWcuY2dpP2lkPTE5NzE2NA0KPiAgICAg ICAgID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAg PiAgICAgICAgID4gICAgICAgICA+IFsyXToNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAg ID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+DQo+ICAgICAg ICAgaHR0cHM6Ly9idWdzLmZyZWVic2Qub3JnL2J1Z3ppbGxhL3Nob3dfYnVnLmNnaT9pZD0xOTEz NDgNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAg PiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAvSw0KPiAgICAgICAgID4gICAgICAgICA+ ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAg ICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAg QW5vdGhlciBvYnNlcnZhdGlvbiBJwrR2ZSBtYWRlIGlzDQo+ICAgICAgICAgdGhhdA0KPiAgICAg ICAgID4gICAgICAgICBkdXJpbmcgdGhlc2UNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAg IHBhdXNlcywgdGhlDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgZW50 aXJlDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgc3lzdGVtIGlzIHB1 dCBvbiBob2xkLCBldmVuIFpGUw0KPiAgICAgICAgIHNjcnViDQo+ICAgICAgICAgPiAgICAgICAg IHN0b3BzIGFuZCB0aGVuDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICByZXN1bWVzDQo+ ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgYWZ0ZXIgYQ0KPiAgICAgICAg ID4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIHdoaWxlLiBMb29raW5nIGluIHRvcCwgdGhl DQo+ICAgICAgICAgc3lzdGVtIGlzDQo+ICAgICAgICAgPiAgICAgICAgIGNvbXBsZXRseSBpZGxl Lg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+ ICAgICAgICAgPiAgICAgICAgIE5vcm1hbGx5IGR1cmluZyBzY3J1YiwgdGhlDQo+ICAgICAgICAg a2VybmVsIGVhdHMNCj4gICAgICAgICA+ICAgICAgICAgMjAtMzAlIENQVSwNCj4gICAgICAgICA+ ICAgICAgICAgPiAgICAgICAgIGJ1dCBkdXJpbmcNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICBhIHBhdXNlLA0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPiAg ICAgICAgIGV2ZW4gdGhlIFtrZXJuZWxdIGdvZXMgZG93biB0bw0KPiAgICAgICAgIDAuMDAlLg0K PiAgICAgICAgID4gICAgICAgICBNYWtlcyBtZSB0aGluaw0KPiAgICAgICAgID4gICAgICAgICA+ ICAgICAgICAgdGhlDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgbmV0 d29ya2luZyBoYXMNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICBub3Ro aW5nIHRvIGRvIHdpdGggaXQuDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+DQo+ICAg ICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgV2hhdMK0cyB0aGVuIHRvIGJsYW1l PyBaRlM/DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICA+ICAgICAgICAgL0sNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAg ID4NCj4gICAgICAgICA+ICAgICAgICAgIF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAg ZnJlZWJzZC1mc0BmcmVlYnNkLm9yZyBtYWlsaW5nDQo+ICAgICAgICAgbGlzdA0KPiAgICAgICAg ID4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4NCj4gICAgICAgICBodHRwOi8vbGlz dHMuZnJlZWJzZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLWZzDQo+ICAgICAgICAgPiAg ICAgICAgID4gICAgICAgICA+ICAgICAgICAgVG8gdW5zdWJzY3JpYmUsIHNlbmQgYW55IG1haWwg dG8NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICAgImZyZWVic2Qt ZnMtdW5zdWJzY3JpYmVAZnJlZWJzZC5vcmciDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAg ICA+DQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAg ID4gICAgICAgICA+IEhlbGxvLA0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAgICAgPg0KPiAg ICAgICAgID4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+ICAgICAg ICAgPiBkb2VzIHRoaXMgaGFwcGVuIHdoZW4gY2xpZW50cyBhcmUgb25seQ0KPiAgICAgICAgIHJl YWRpbmcNCj4gICAgICAgICA+ICAgICAgICAgZnJvbSBzZXJ2ZXI/DQo+ICAgICAgICAgPiAgICAg ICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgIFllcyBpdCBoYXBwZW5zIHdoZW4g Y2xpZW50cyBhcmUgb25seQ0KPiAgICAgICAgIHJlYWRpbmcgZnJvbQ0KPiAgICAgICAgID4gICAg ICAgICB0aGUgc2VydmVyLg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICA+IE90aGVyd2lzZSBJIHdvdWxkIHN1c3BlY3QgdGhhdCBpdCBjb3VsZA0K PiAgICAgICAgIGJlIGNhdXNlZA0KPiAgICAgICAgID4gICAgICAgICBieSBaRlMNCj4gICAgICAg ICA+ICAgICAgICAgPiAgICAgICAgIHdyaXRpbmcgb3V0IGENCj4gICAgICAgICA+ICAgICAgICAg PiAgICAgICAgID4gbGFyZ2UgY2h1bmNrIG9mIGRhdGEgc2l0dGluZyBpbiBpdHMNCj4gICAgICAg ICBjYWNoZXMsIGFuZA0KPiAgICAgICAgID4gICAgICAgICB1bnRpbCB0aGF0DQo+ICAgICAgICAg PiAgICAgICAgID4gICAgICAgICBpcyBjb21wbGV0ZQ0KPiAgICAgICAgID4gICAgICAgICA+ICAg ICAgICAgPiBJL08gaXMgc3RhbGxlZC4NCj4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAg ID4gICAgICAgICA+ICAgICAgICAgVGhhdMK0cyB3aGF0IHNvIHN0cmFuZ2UsIHdlIGhhdmUgdGhy ZWUgbW9yZQ0KPiAgICAgICAgIHN5c3RlbXMNCj4gICAgICAgICA+ICAgICAgICAgc2V0IHVwDQo+ ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICBhYm91dCB0aGUgc2FtZQ0KPiAgICAgICAgID4g ICAgICAgICA+ICAgICAgICAgc2l6ZSBhbmQgbm9uZSBvZiBvdGhlcnMgYXJlIGFjdGluZyB0aGlz DQo+ICAgICAgICAgd2F5Lg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICBUaGUgb25seSB0aGluZyBJIGNhbiB0aGluayBvZiB0aGF0IGRpZmZlcnMN Cj4gICAgICAgICB0aGF0IHdlDQo+ICAgICAgICAgPiAgICAgICAgIGhhdmVuwrR0DQo+ICAgICAg ICAgPiAgICAgICAgID4gICAgICAgICB0ZXN0ZWQgcnVsaW5nDQo+ICAgICAgICAgPiAgICAgICAg ID4gICAgICAgICBvdXQgeWV0IGlzIGN0bGQsIHRoZSBvdGhlciBzeXN0ZW1zIGFyZQ0KPiAgICAg ICAgIHN0aWxsIHJ1bm5pbmcNCj4gICAgICAgICA+ICAgICAgICAgaXN0Z3QgYXMNCj4gICAgICAg ICA+ICAgICAgICAgPiAgICAgICAgIHRoZWlyDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAg ICBpU0NTSSBkYWVtb24uDQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAg ICAgPiAgICAgICAgIC9LDQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAg ICAgPg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAg ICAgICA+ICAgICAgICAgPiBJZiB0aGVyZSBhcmUgb3RoZXIgdGhyZWUgc2ltaWxhciBzeXN0ZW1z IGFuZCB0aGV5DQo+ICAgICAgICAgYXJlDQo+ICAgICAgICAgPiAgICAgICAgIGV4YWN0bHkNCj4g ICAgICAgICA+ICAgICAgICAgPiBpbnN0YWxsZWQgd2l0aCB0aGUgc2FtZSBzdHJ1Y3R1cmUgLCBt eSBmaXJzdA0KPiAgICAgICAgIHBvc3NpYmlsaXR5IHRvDQo+ICAgICAgICAgPiAgICAgICAgIGNv bnNpZGVyDQo+ICAgICAgICAgPiAgICAgICAgID4gd291bGQgYmUgdG8gc3VzcGVjdCBhIHNsb3ds eSBwcm9ncmVzc2luZyBoYXJkd2FyZQ0KPiAgICAgICAgIGZhaWx1cmUgOg0KPiAgICAgICAgID4g ICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiBG cm9tIGEgY2lyY3VpdCAsIGl0IGlzIG5vdCBwb3NzaWJsZSB0byBnZXQgYQ0KPiAgICAgICAgIHJl c3BvbnNlIGluDQo+ICAgICAgICAgPiAgICAgICAgIGV4cGVjdGVkDQo+ICAgICAgICAgPiAgICAg ICAgID4gdGltZSAsIGJ1dCAsIGl0IGlzIHJlc3BvbmRpbmcgYWZ0ZXIgYSB0aW1lIHdoaWNoDQo+ ICAgICAgICAgaXMgbm90DQo+ICAgICAgICAgPiAgICAgICAgIG5vcm1hbCAuIFN1Y2gNCj4gICAg ICAgICA+ICAgICAgICAgPiBhbiBhY3Rpb24gbWF5IGJlIGNhdXNlZCBieSBhIGZhdWx0eSBzb2xk ZXJlZCBvcg0KPiAgICAgICAgIGNyYWNrZWQgbGluZQ0KPiAgICAgICAgID4gICAgICAgICBwb2lu dCBpbg0KPiAgICAgICAgID4gICAgICAgICA+IHRoZSBjaXJjdWl0IDogV2hlbiBpdCBpcyBob3Qg LCBpdCBpcw0KPiAgICAgICAgIGRpc2Nvbm5lY3RpbmcgLCB3aGVuIGl0DQo+ICAgICAgICAgPiAg ICAgICAgIGlzIGNvbGQNCj4gICAgICAgICA+ICAgICAgICAgPiBpdCBpcyBjb25uZWN0aW5nIC4N Cj4gICAgICAgICA+DQo+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICBBcyBpbml0aWFs bHkgc3RhdGVkLCBib3RoIG1vdGhlcmJvYXJkIGFuZCBwcm9jZXNzb3INCj4gICAgICAgICBoYXMg YmVlbg0KPiAgICAgICAgID4gICAgICAgICByZXBsYWNlZA0KPiAgICAgICAgID4gICAgICAgICB3 aXRoIGlkZW50aWNhbCBoYXJkd2FyZSB0aGF0IHdlbnQgdGhyb3VnaCBhIGRheSBvZg0KPiAgICAg ICAgIG1lbXRlc3QNCj4gICAgICAgICA+ICAgICAgICAgYmVmb3JlIGJlaW5nDQo+ICAgICAgICAg PiAgICAgICAgIGluc3RhbGxlZC4gVGhlbiB0aGVyZcK0cyBhbiBleHRlcm5hbCBTdXBlcm1pY3Jv DQo+ICAgICAgICAgSkJPRFsqXSBidXQgSQ0KPiAgICAgICAgID4gICAgICAgICBoYXZlbsK0dA0K PiAgICAgICAgID4gICAgICAgICBzZWVuIGFueSBkaXNrIHRpbWVvdXRzIG9yIFNFUyBlcnJvcnMg bG9nZ2VkLiBBdA0KPiAgICAgICAgIGxlYXN0IGF0IGENCj4gICAgICAgICA+ICAgICAgICAgZHJp dmVyIGxldmVsDQo+ICAgICAgICAgPiAgICAgICAgIHRoZXJlIHNob3VsZCBoYXZlIGJlZW4gdGlt ZW91dHMgYXQgc3VjaCBhIGxvbmcgZGVsYXkNCj4gICAgICAgICBhcyBmaXZlDQo+ICAgICAgICAg PiAgICAgICAgIG1pbnV0ZXMuDQo+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICAvSw0K PiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgWypdOg0KPiAgICAgICAgID4NCj4gICAg ICAgICAgaHR0cDovL3d3dy5zdXBlcm1pY3JvLm5sL3Byb2R1Y3RzL2NoYXNzaXMvM1UvODM3L1ND ODM3RTI2LVJKQk9EMS5jZm0NCj4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4NCj4g ICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAg ICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+IFRo YW5rIHlvdSB2ZXJ5IG11Y2ggLg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAg ICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+IE1l aG1ldCBFcm9sIFNhbmxpdHVyaw0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAg ICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4NCj4g ICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAg ICAgID4gSGF2ZSB5b3UgdHJpZWQgd2hhdCBpcyBzdWdnZXN0ZWQgaW4NCj4gICAgICAgICA+ICAg ICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICBodHRwczovL3dpa2kuZnJlZWJzZC5vcmcvWkZT VHVuaW5nR3VpZGUgPyBJbg0KPiAgICAgICAgID4gICAgICAgICBwYXJ0aWN1bGFyDQo+ICAgICAg ICAgPiAgICAgICAgID4gICAgICAgICBzZXR0aW5nDQo+ICAgICAgICAgPiAgICAgICAgID4gICAg ICAgICA+IHZmcy56ZnMud3JpdGVfbGltaXRfb3ZlcnJpZGUgdG8NCj4gICAgICAgICBzb21ldGhp bmcNCj4gICAgICAgICA+ICAgICAgICAgYXBwcm9wcmlhdGUgZm9yDQo+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICB5b3VyIHNpdGUuDQo+ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+ IFRoZSB0aW1lb3V0IHNlZW1zIHRvIGJlIGRlZmF1bHRpbmcgdG8gNQ0KPiAgICAgICAgIG5vdy4N Cj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiAg ICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4gQmVzdCByZWdhcmRzDQo+ ICAgICAgICAgPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPiAgICAgICAgID4gICAg ICAgICA+IEFuZHJlYXMNCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAg ICA+ICAgICAgICAgPiAgICAgICAgID4NCj4gICAgICAgICA+ICAgICAgICAgPiAgICAgICAgID4N Cj4gICAgICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAg IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+ICAgICAg ICAgPiAgICAgICAgID4gICAgICAgICBmcmVlYnNkLWZzQGZyZWVic2Qub3JnIG1haWxpbmcgbGlz dA0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgIGh0dHA6Ly9saXN0cy5mcmVlYnNk Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ZyZWVic2QtZnMNCj4gICAgICAgICA+ICAgICAgICAgPiAg ICAgICAgIFRvIHVuc3Vic2NyaWJlLCBzZW5kIGFueSBtYWlsIHRvDQo+ICAgICAgICAgPiAgICAg ICAgID4gICAgICAgICAiZnJlZWJzZC1mcy11bnN1YnNjcmliZUBmcmVlYnNkLm9yZyINCj4gICAg ICAgICA+ICAgICAgICAgPg0KPiAgICAgICAgID4gICAgICAgICA+DQo+ICAgICAgICAgPg0KPiAg ICAgICAgID4NCj4gICAgICAgICA+DQo+ICAgICAgICAgPg0KPiAgICAgICAgIA0KPiAgICAgICAg IA0KPiANCj4gDQoNCg== From owner-freebsd-fs@FreeBSD.ORG Tue Jun 2 15:17:23 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DB5BA61 for ; Tue, 2 Jun 2015 15:17:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0769612E2 for ; Tue, 2 Jun 2015 15:17:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t52FHMj1061925 for ; Tue, 2 Jun 2015 15:17:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199804] ZFS: i/o error - all block copies unavailable Date: Tue, 02 Jun 2015 15:17:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: tsoome@me.com X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jun 2015 15:17:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199804 --- Comment #2 from Toomas Soome --- actually the fix itself is buggy:) the basic idea is correct, but the size of the zero filled buffer is wrong, the correct code is: if (BP_IS_HOLE(&bp)) { memset(dnode_cache_buf, 0, dnode->dn_datablkszsec << SPA_MINBLOCKSHIFT); rc = 0; } else rc = zio_read(spa, &bp, dnode_cache_buf); as the BP_PSIZE() or BP_LSIZE() is 512B at that point (for 512B sector size at least), and indicates the size of BP itself, not the size of the hole - so the original (bad) fix did zero out only first 512B and rest of the buffer did return garbage. This fix is verified by hashing the whole file and checking against file on disk and file read to memory by this reader code. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 05:27:11 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01DBEBA4 for ; Wed, 3 Jun 2015 05:27:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E05D5121D for ; Wed, 3 Jun 2015 05:27:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t535RAAd065230 for ; Wed, 3 Jun 2015 05:27:10 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Wed, 03 Jun 2015 05:27:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pho@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 05:27:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 --- Comment #3 from Peter Holm --- Looks like td_ucred problem: https://people.freebsd.org/~pho/stress/log/svn.txt -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 07:08:12 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9153171A for ; Wed, 3 Jun 2015 07:08:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B0411949 for ; Wed, 3 Jun 2015 07:08:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t5378Cnt004108 for ; Wed, 3 Jun 2015 07:08:12 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Wed, 03 Jun 2015 07:08:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 07:08:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 Konstantin Belousov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kib@FreeBSD.org --- Comment #4 from Konstantin Belousov --- (In reply to Peter Holm from comment #3) Indeed, the unmount which drops the last reference on the nfs mount credentials is the culprit. This sole issue can be easily handled by taking additional reference on the mnt_cred, before the vnode is unlocked. If there are further problems from the mnt disappearing, we could try to busy the filesystem after the vnode is unlocked. But this would have bad consequences for the unmount -f. Lets try the minimal patch first. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 07:10:19 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1CC4B7AE for ; Wed, 3 Jun 2015 07:10:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0462E1983 for ; Wed, 3 Jun 2015 07:10:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t537AID5006281 for ; Wed, 3 Jun 2015 07:10:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Wed, 03 Jun 2015 07:10:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 07:10:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 --- Comment #5 from Konstantin Belousov --- Created attachment 157390 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=157390&action=edit Hold the mnt credentials before vnode unlock. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 08:33:13 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC6AC604 for ; Wed, 3 Jun 2015 08:33:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A52DF1E59 for ; Wed, 3 Jun 2015 08:33:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t538XD5N029011 for ; Wed, 3 Jun 2015 08:33:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Wed, 03 Jun 2015 08:33:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: pho@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 08:33:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 --- Comment #6 from Peter Holm --- I see spinning threads with this patch: https://people.freebsd.org/~pho/stress/log/kostik813.txt -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 11:21:45 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3BF46382 for ; Wed, 3 Jun 2015 11:21:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 25FA71700 for ; Wed, 3 Jun 2015 11:21:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t53BLjuc059949 for ; Wed, 3 Jun 2015 11:21:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199804] ZFS: i/o error - all block copies unavailable Date: Wed, 03 Jun 2015 11:21:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 11:21:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199804 --- Comment #3 from Andriy Gapon --- (In reply to Toomas Soome from comment #2) I think this fix is not completely correct. Now you use the data block size as the hole size. I think that the hole size should be determined based on the level of the hole. Essentially, if the hole is at the zero level, then it covers dn_datablkszsec * 512 bytes of data (one data block). If the hole is one level above, then it covers as many data blocks as a number of block pointers that fit into one indirect block. And so on. Please note that (nlevels - i - 1) is the level number in the loop. E.g. search for BP_SPAN() that is used elsewhere for the same purpose. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 12:20:41 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C736621 for ; Wed, 3 Jun 2015 12:20:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 65C9914E4 for ; Wed, 3 Jun 2015 12:20:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t53CKfuv097077 for ; Wed, 3 Jun 2015 12:20:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Wed, 03 Jun 2015 12:20:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rmacklem@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 12:20:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 --- Comment #7 from Rick Macklem --- It's been a while since I looked at it, but I remember that there is a known issue with the client side NLM for "umount -f". Basically it unlocks and later relocks the vnode without holding a refcnt on the mount point. At the time, I didn't know enough about the mount point refcnt stuff to fix it. I'll try and take a look later to-day. I don't know if this is the cause of the crash (I can't remember what credentials are used), but it does need to be fixed for "umount -f" to be safe when running rpc.lockd. (I keep hoping people will stop using rpc.lockd and rpc.statd, but it's just a dream;-) -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 12:29:04 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7B9C867 for ; Wed, 3 Jun 2015 12:29:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D20A81701 for ; Wed, 3 Jun 2015 12:29:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t53CT4Js005676 for ; Wed, 3 Jun 2015 12:29:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199804] ZFS: i/o error - all block copies unavailable Date: Wed, 03 Jun 2015 12:29:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 12:29:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199804 --- Comment #4 from Andriy Gapon --- Actually please scratch comment #3. We read data blocks one by one, so regardless of the actual hole size we need to care only about the current data block. Still, we have to keep in mind that the hole can occur at a level above zero and in that case we do not need to descend further. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 12:31:29 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 288B9907 for ; Wed, 3 Jun 2015 12:31:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 12C3E18B3 for ; Wed, 3 Jun 2015 12:31:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t53CVS5L087527 for ; Wed, 3 Jun 2015 12:31:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199804] ZFS: i/o error - all block copies unavailable Date: Wed, 03 Jun 2015 12:31:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: avg@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete assigned_to bug_status attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 12:31:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199804 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #156141|0 |1 is obsolete| | Assignee|freebsd-fs@FreeBSD.org |avg@FreeBSD.org Status|Open |In Progress --- Comment #5 from Andriy Gapon --- Created attachment 157394 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=157394&action=edit another take at the fix -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 3 23:19:24 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 87AC946D for ; Wed, 3 Jun 2015 23:19:24 +0000 (UTC) (envelope-from javocado@gmail.com) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 10BB819E1 for ; Wed, 3 Jun 2015 23:19:24 +0000 (UTC) (envelope-from javocado@gmail.com) Received: by labpy14 with SMTP id py14so19726365lab.0 for ; Wed, 03 Jun 2015 16:19:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=RaUER9rbRWL5y9MqajcF90375N1wnqbDGehxNRj8f1Y=; b=pBUI+UrEo5S4eJHAdhT4NI73JXpdTk4G0HE5FHyHx7ZyL6U12jfI8SaOREDJZu26O2 rFd0liw47VxAb6Z4trpfNE0m7EI0apdHDOsBn0AOSqSlafmouB9LPgAj3ZI2OhNzfuPh jJAHThlL53kXBnYb5meOljpUoKyxy+VRSiS9/KtTUkzW39CTjp1gFmSxy+QPOTAJP5oY U4jYQcKso+gyuFC4Zk1GqHvfzTKe30J5VhfPS9FfGX+q0IBnrsHXfj36dfTEc4cu35mM oq4vi5x92Jwz89muokHo3LEHfFYiT0Smk5ZnH/Kz4xgfOnven+AbMcyM2bzxJpwJDCXi PhQA== MIME-Version: 1.0 X-Received: by 10.152.43.37 with SMTP id t5mr33626192lal.96.1433373562127; Wed, 03 Jun 2015 16:19:22 -0700 (PDT) Received: by 10.114.96.8 with HTTP; Wed, 3 Jun 2015 16:19:22 -0700 (PDT) Date: Wed, 3 Jun 2015 16:19:22 -0700 Message-ID: Subject: growfs failure From: javocado To: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jun 2015 23:19:24 -0000 While trying to growfs a filesystem, I receive the following error: growfs: rdfs: read error: 5812093147771869908: Input/output error Here were the steps taken leading up to this point: (original file is 300 GB, growing to 500 GB) (the filesystem is clean with fsck_ufs /dev/md1) geli detach /dev/md1.eli mdconfig -d -u 1 truncate -s +200G geli.img mdconfig -f geli.img -u 1 geli resize -s 300G /dev/md1 geli attach /dev/md1 growfs /dev/md1.eli new file systemsize is: 262143999 frags Warning: 326780 sector(s) cannot be allocated. growfs: 511840.4MB (1048249216 sectors) block size 16384, fragment size 2048 using 2786 cylinder groups of 183.72MB, 11758 blks, 23552 inodes. super-block backups (for fsck -b #) at: 629476448, 629852704, 630228960, 630605216, 630981472, 631357728, 631733984, 632110240, .... growfs: rdfs: read error: 5812093147771869908: Input/output error From owner-freebsd-fs@FreeBSD.ORG Thu Jun 4 09:24:25 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F0F990 for ; Thu, 4 Jun 2015 09:24:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1741A1832 for ; Thu, 4 Jun 2015 09:24:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t549OOjc046303 for ; Thu, 4 Jun 2015 09:24:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200288] Modify after Free: ZFS Date: Thu, 04 Jun 2015 09:24:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jun 2015 09:24:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200288 Andriy Gapon changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Jun 5 12:17:37 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95A7A1B0 for ; Fri, 5 Jun 2015 12:17:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7EF0F1D08 for ; Fri, 5 Jun 2015 12:17:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t55CHbd7035308 for ; Fri, 5 Jun 2015 12:17:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Fri, 05 Jun 2015 12:17:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rmacklem@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2015 12:17:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 --- Comment #8 from Rick Macklem --- Although adding a ref count to the cred struct may help avoid some crashes, the code that prints out the message uses both the mount structure and the NFS specific one hanging off of mnt_data. Unfortunately adding a ref count to the mount structure via vfs_ref(mp) only delays freeing of *mp and not *mnt_data. Also, vfs_destroy_mount() waits for the ref count on the mount structure to go to 0 and, as such, "umount -f" won't complete until the lock RPC completes. If the ref count on the mount structure only free'd the structure and substructure hanging off of mnt_data, then adding a ref count to the mount structure before the VOP_UNLOCK() could avoid the crashes. Maybe this VFS change should be considered? Also, doing "umount -f" on a file system using locking could leave "dangling locks" on files. In theory, rpc.statd should eventually notice that the host isn't responding and get rid of these file locks, but I wouldn't bet on it. I suppose the dangers of using "umount -f" while running rpc.lockd should be documented. I'll come up with a man page patch for that. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Jun 5 23:59:49 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5F8278C for ; Fri, 5 Jun 2015 23:59:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BFA2B187E for ; Fri, 5 Jun 2015 23:59:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t55Nxnhe032201 for ; Fri, 5 Jun 2015 23:59:49 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200663] zfs allow/unallow doesn't show numeric UID when the ID no longer exists in the password file Date: Fri, 05 Jun 2015 23:59:50 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jun 2015 23:59:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200663 Larry Rosenman changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |freebsd-fs@FreeBSD.org -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat Jun 6 00:00:06 2015 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B0147F7 for ; Sat, 6 Jun 2015 00:00:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 24EAE188D for ; Sat, 6 Jun 2015 00:00:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t56006tD034297 for ; Sat, 6 Jun 2015 00:00:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200663] zfs allow/unallow doesn't show numeric UID when the ID no longer exists in the password file Date: Sat, 06 Jun 2015 00:00:06 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ler@lerctr.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jun 2015 00:00:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200663 --- Comment #1 from Larry Rosenman --- Adding freebsd-fs -- You are receiving this mail because: You are on the CC list for the bug.