From owner-freebsd-fs@freebsd.org Sat Aug 13 04:41:10 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6710BB5F7C for ; Sat, 13 Aug 2016 04:41:10 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0073.outbound.protection.outlook.com [65.55.169.73]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3AE1E1726 for ; Sat, 13 Aug 2016 04:41:09 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR01MB0401.CANPRD01.PROD.OUTLOOK.COM (10.169.142.147) by YQBPR01MB0404.CANPRD01.PROD.OUTLOOK.COM (10.169.142.150) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.549.15; Sat, 13 Aug 2016 00:07:12 +0000 Received: from YQBPR01MB0401.CANPRD01.PROD.OUTLOOK.COM ([10.169.142.147]) by YQBPR01MB0401.CANPRD01.PROD.OUTLOOK.COM ([10.169.142.147]) with mapi id 15.01.0557.021; Sat, 13 Aug 2016 00:07:12 +0000 From: Rick Macklem To: Mahmoud Al-Qudsi , "freebsd-fs@freebsd.org" Subject: Re: PR-211674, fuse_vnode and fuse_msgbuf leak in fusefs-ntfs Thread-Topic: PR-211674, fuse_vnode and fuse_msgbuf leak in fusefs-ntfs Thread-Index: AdHxnNoX8eB2gk++Rnigc2FfbUZx+wDVXGlo Date: Sat, 13 Aug 2016 00:07:12 +0000 Message-ID: References: <012c01d1f19d$0aae9c70$200bd550$@neosmart.net> In-Reply-To: <012c01d1f19d$0aae9c70$200bd550$@neosmart.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-originating-ip: [24.57.164.61] x-ms-office365-filtering-correlation-id: bff18731-864d-453c-d501-08d3c30dc690 x-microsoft-exchange-diagnostics: 1; YQBPR01MB0404; 6:vGnFX/sGqAp+oXC4issoTgTH1qcBR7QUJ5WpbLpCoBGLv0Cbl0IE1aGA64GQqXySqzF8r0Va2cuQMIgSNhzb273GBXH2wVDMaIsbYBsQelZuAwCexuedWJ7PPI+xY8Jq1sShFdQl1n8i9mz+YqL9Ig6ZLy0iHs4+IMrEqK1SfWt+LHyfG43jKVkE1pMqSxjkh4Neifva7SgHkn420d3kzD6CmRJK5yRp6euTOzGF6NkiKDP+mUJFwt60bMI+wNHnRKEr5LJ3Tz/PHIfPhqCXCccHtzYK/h/UleD1AsDDvk7EALouDKlAjsADzNl5smxr; 5:PeHmyySERCC/QUVFOzVFDe474xDfubKRTQFUcf/2bY4K/aHAAG5TG1GWzPnmkAr04oKDgT/S8L04iRiK86tQedmX8efLnyiMWKczBgr4X5iqjtaf0wCaK5/ndGeVPH7jEAsWs9vh+Q88RFNVwoeoTQ==; 24:NsKBruuxUmJF8O6QGiijwLYipv2qlZNp7vQJEWttySzjKv9nkH4CuEe/pDh8rWW9VUx43mpWqJiq5gp79DJkDVOyjmPQ/ypftK7mUTqU1WI=; 7:jSDbqcc72wxw53CvXSezLeJQnZ/JwwY46bZjLNKcAzxgEQZ3e2x7tZH4WfpjjZZ/OESbalxqaPltlkdiOERHFUnobp60zg0QeuleOO90thdKrxSyGET8ldivWPg/46XzPQtuE3iqHvhjYIq/8/94z47t9B/Z76IadyUn5EdSdMmI8Vgdaanjrjo25GIWnckOL4tqJ+iSjwgs7k04/7J+2z3HnO35THUo2kVZcHU9lE2dPTtznXtmub124WENZRD4 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:YQBPR01MB0404; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(75325880899374); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040176)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6043046)(6042046); SRVR:YQBPR01MB0404; BCL:0; PCL:0; RULEID:; SRVR:YQBPR01MB0404; x-forefront-prvs: 0033AAD26D x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(7916002)(189002)(199003)(24454002)(16236675004)(74482002)(33656002)(10400500002)(7696003)(7736002)(66066001)(2906002)(2900100001)(2950100001)(50986999)(15975445007)(8676002)(81166006)(8936002)(76176999)(81156014)(54356999)(101416001)(74316002)(19625215002)(2501003)(5002640100001)(92566002)(19617315012)(586003)(9686002)(19580395003)(105586002)(3846002)(7906003)(87936001)(19580405001)(106356001)(11100500001)(189998001)(5001770100001)(6116002)(107886002)(19627405001)(3280700002)(68736007)(77096005)(7846002)(3660700001)(102836003)(122556002)(97736004)(86362001)(21314002); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR01MB0404; H:YQBPR01MB0401.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Aug 2016 00:07:12.0051 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR01MB0404 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Aug 2016 04:41:10 -0000 Mahmoud Al-Qudsi wrote: >Hello, > >Please forgive me if it is not correct form to discuss fusefs-ntfs on the = FreeBSD fs mailing list. > >SUMMARY > >Running on FreeBSD 10.3-RELEASE-p6/i386 with fuse compiled into kernel and= with fusefs-ntfs >2016.2.22 installed, there is a fuse_vnode leak (though = it seems it may be more of a complete failure >to reclaim vnodes) resulting= in quick resource exhaustion. > >REPRODUCTION > >This is easily reproduced with the following: > >ntfs-3g /dev/xxx /mnt/yyyy >cd /mnt/yyyy >find . -exec touch {} \; > >In another virtual terminal: > >vmstat | head -n1; vmstat -m | sed 1d | sort -hk 3,3 > >ACTUAL RESULTS > >fuse_vnode will continuously balloon, and will not be reclaimed until the = filesystem is unmounted. > >(likewise, fuse_msgbuff also balloons but unlike fuse_vnode, it is never r= eclaimed. Separate PR?) > >EXPECTED RESULTS > >fuse_vnode entries should be reclaimed > >ADDITIONAL INFORMATION > >Here's a snapshot of the fuse-related vmstat entries after this process: > >fuse_vnode 36020 9005K - 502349 256 These should be free'd when the vnode is recycled. This should start happen= ing when the system has reached kern.maxvnodes (I think?). Check the sysctl: vfs.fuse.node_count - if this value is much smaller than the # malloc'd, something is broken. I= f it is the same, it may just be that the system hasn't been recycling the vnodes= yet. You could try setting kern.maxvnodes smaller to see if recycling starts hap= pening, but be warned...setting this too small can break your system badly. Since unmounting will cause all vnodes to be recycled, seeing it go down wh= en you unmount is normal. >fuse_msgbuf 58141 14895K - 311095 256,512,1024,2048,4096,8192 At a glance, these are allocated each time the fuse device is opened and are never free'd. I don't know why fuse does this. My guess is that the ntfs-3g file system is opening the device repeatedly. If that isn't happeni= ng, I have no idea why this is occurring. rick >Thank you, >Mahmoud Al-Qudsi >NeoSmart Technologies > > >_______________________________________________ >freebsd-fs@freebsd.org mailing list >https://lists.freebsd.org/mailman/listinfo/freebsd-fs >To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"