From owner-freebsd-fs@freebsd.org Sun Jun 30 00:46:05 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D92E15C5891 for ; Sun, 30 Jun 2019 00:46:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660056.outbound.protection.outlook.com [40.107.66.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F185869E3A; Sun, 30 Jun 2019 00:46:01 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=testarcselector01; d=microsoft.com; cv=none; b=Gccu+dd7SylRUtpLdgkL9rys5lFG93bCo4P/ggabYq8MrQ8FYfMSr1RlQYLm0RT78AS3zo2vItOsV/9Afc5Ps2EJEwar3zCZXOiwFRP6bepbc+K5LUZhmGOdWSCNYAEnBHREYnQCspjrghqs+FytZhORNUOoazM+FwWK74pf4is= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=testarcselector01; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WVl8zO8MhPeiLPY7Xktnn6EGnPFw6Nk3VyAz1VIJwps=; b=T4PlMbcIJSHS3lOHqoMaqbIflDQaI0RT4q4wP6quJcSpB+h5NvG7Y/lx15HLGDT3eEvhOVxwnpEESLkcva2xxMSzpcomUPxMNCbe3HKM0asCRfm86KPoWCSwWbfAib3KF6NCqwezwjufM8o3KkE8C6ViOMwvbDmHRM6bsmAp7D8= ARC-Authentication-Results: i=1; test.office365.com 1;spf=none;dmarc=none;dkim=none;arc=none Received: from YTXPR01MB0285.CANPRD01.PROD.OUTLOOK.COM (10.165.219.7) by YTXPR01MB0285.CANPRD01.PROD.OUTLOOK.COM (10.165.219.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2008.16; Sun, 30 Jun 2019 00:45:59 +0000 Received: from YTXPR01MB0285.CANPRD01.PROD.OUTLOOK.COM ([fe80::7db7:14c1:c4d0:5ecc]) by YTXPR01MB0285.CANPRD01.PROD.OUTLOOK.COM ([fe80::7db7:14c1:c4d0:5ecc%4]) with mapi id 15.20.2008.014; Sun, 30 Jun 2019 00:45:59 +0000 From: Rick Macklem To: Alan Somers CC: "freebsd-fs@freebsd.org" , Sean Fagan Subject: Re: RFC: What should a copy_file_range(2) syscall do by default? Thread-Topic: RFC: What should a copy_file_range(2) syscall do by default? Thread-Index: AQHVKREqKzvk0WaXu025/Chow1UOHaan7Y6AgAt533g= Date: Sun, 30 Jun 2019 00:45:58 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e24f78d5-e2f7-49dd-34c7-08d6fcf4519c x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:YTXPR01MB0285; x-ms-traffictypediagnostic: YTXPR01MB0285: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 008421A8FF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(346002)(366004)(376002)(39850400004)(396003)(199004)(189003)(6246003)(486006)(102836004)(11346002)(5660300002)(446003)(305945005)(53936002)(68736007)(66446008)(64756008)(66556008)(4326008)(74482002)(76116006)(76176011)(91956017)(229853002)(66476007)(14454004)(66946007)(86362001)(73956011)(6916009)(186003)(9686003)(6436002)(8936002)(52536014)(54906003)(476003)(33656002)(256004)(8676002)(81156014)(81166006)(6506007)(53546011)(55016002)(7696005)(478600001)(71200400001)(71190400001)(74316002)(46003)(14444005)(786003)(316002)(25786009)(99286004)(2906002); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0285; H:YTXPR01MB0285.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: xktRgXAbwUwqC1BTwmGtrhG54wUtd4VMEgrqOspJUdKUVPs9gXWjG+LjFYutjDM321K2cdTyu0gHwiAbRb2XXznENhFY79KepFlC6zXrrl8J0NLqPfJzCJKpZakSqjS5q5gfejI4RGCyWiReD4FCow9fN2V5DRXsclgxoCiU6ZgfttbudQps5DH+QgDMc3c6f1r7pGkJjdsA/31nVlKIQ1ktZaiHMOYpY3JUWUyxI+4K84rG/Uc7qk+YxuSG6K25cYtbvUdO+9dUY8Kk9+xs5H9gl+fQWiwGhFKEZ6YkjnRjfcKDpe/nLhvRzvFoh3z11xaofmtAbN9HL8/5eYnxn4fZLFp7WhDLQgvQ/zkjEWClFVlOpRJ1WCwh+LAXcWs4oJYsVY2JZDhS8cyhSdVGmpzeLteN1EfikDoj+F1TM64= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: e24f78d5-e2f7-49dd-34c7-08d6fcf4519c X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Jun 2019 00:45:58.8363 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rmacklem@uoguelph.ca X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0285 X-Rspamd-Queue-Id: F185869E3A X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.66.56 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; NEURAL_HAM_LONG(-0.27)[-0.265,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[uoguelph.ca]; TO_DN_SOME(0.00)[]; ARC_REJECT(2.00)[signature check failed: fail, {[1] = sig:microsoft.com:reject}]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.21)[0.205,0]; MX_GOOD(-0.01)[mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com]; RCVD_IN_DNSWL_NONE(0.00)[56.66.107.40.list.dnswl.org : 127.0.3.0]; NEURAL_SPAM_MEDIUM(0.37)[0.371,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jun 2019 00:46:05 -0000 Well, asomers@ prefers the current patch, despite its complexity. sef@ expressed concerns w.r.t. the complexity and it being done in the kernel. However he did not state a preference for any other specific variant, such as the ones I suggested. As such, I am sticking with the current patch unless I hear otherwise from others. Thanks for your comments, rick ps: I felt a top post was reasonable here, as it summarized separate posts in the thread. ________________________________________ From: Alan Somers Sent: Saturday, June 22, 2019 1:28:03 PM To: Rick Macklem Cc: freebsd-fs@freebsd.org; Sean Fagan Subject: Re: RFC: What should a copy_file_range(2) syscall do by default? On Sat, Jun 22, 2019 at 10:02 AM Rick Macklem wrote: > > Hi, > > sef@ made this comment on phabricator. I don't believe phabricator is the= correct > place for "big picture" discussions, so I'm posting it here (I'm assuming= sef@ doesn't > mind, since the phabricator comments are public). > sef@ wrote: > >This much work in the kernel for what //should// be user-space makes me = twitchy... >but there is lots of precedent for it, so I obviously have to g= et with the times. > > > > I've done a quick review of the code; it seems most of the complexity = is in the hole->detection. I'm also annoyed that linux used size_t for the= amount to copy, when >off_t would have been more appropriate. But not muc= h to do about that now. > > > > Having a default implementation means that user-space can't fall back = if it's not >supported, and do it better (e.g., parallel I/O). Should we a= lso have a pathconf for >the feature? > > > > WRT your question on -fs, I have no objections to this working cross-f= ilesystem, >although I think I might ask to have a flag to fail in that cas= e. > > Well, all I am interested in is a system call/VOP call so the NFSv4.2 cli= ent can do > a file copy locally on the NFS server instead of doing Reads/Writes acros= s the wire. > The current code has gotten fairly complex, so I'll try and ask "how comp= lex" this > syscall/VOP call should be? > > The range of variants I can think of are: > 0) - Don't do it at all. > 1) - The syscall could just do a VOP_COPY_FILE_RANGE() and return whateve= r error > it returns. > --> This implies an error return for all file systems for now, wi= th support for > NFSv4.2mounts being added later (FreeBSD13 hopefully). This option would require applications or the C library to fallback to a copy loop. While doable, nothing in userland would be able to range-lock the file, making the copy loop non-atomic. So the in-kernel copy is superior. > 2) - The syscall could fall back on a simple copy loop, but not try to de= al with holes. > --> The Linux man page mentions using copy_file_range(2) in a loop= with > lseek(SEEK_DATA)/lseek(SEEK_HOLE) for sparse files. This sug= gests that > the Linux fallback code doesn't try to handle holes. Same problem as 1. Or if you do the copy loop in-kernel it would waste CPU time and expand sparse files, which isn't good either. > 3) - The current patch which tries to handle holes and copy the entire by= te range > in one call. Definitely the best option, despite its complexity. I would argue that the complexity calls for a robust test suite, rather than abandoning the feature. -Alan From owner-freebsd-fs@freebsd.org Sun Jun 30 15:08:05 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B2D015D83A0 for ; Sun, 30 Jun 2019 15:08:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 932A389DE5 for ; Sun, 30 Jun 2019 15:08:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4DC0415D839F; Sun, 30 Jun 2019 15:08:04 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BF7F15D839E for ; Sun, 30 Jun 2019 15:08:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD88289DE4 for ; Sun, 30 Jun 2019 15:08:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0B583EAB5 for ; Sun, 30 Jun 2019 15:08:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x5UF822q068353 for ; Sun, 30 Jun 2019 15:08:02 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x5UF82TT068351 for fs@FreeBSD.org; Sun, 30 Jun 2019 15:08:02 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 232175] [panic][zfs] solaris assert: HDR_EMPTY(hdr), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c, line: 6213 Date: Sun, 30 Jun 2019 15:08:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: dch@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Unable to Reproduce X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jun 2019 15:08:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D232175 Dave Cottlehuber changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed Resolution|--- |Unable to Reproduce --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Jun 30 21:00:05 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 946B015DF786 for ; Sun, 30 Jun 2019 21:00:05 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0396A70342 for ; Sun, 30 Jun 2019 21:00:05 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id B2EFD15DF780; Sun, 30 Jun 2019 21:00:04 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 906D315DF77F for ; Sun, 30 Jun 2019 21:00:04 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 25E7F70338 for ; Sun, 30 Jun 2019 21:00:04 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 6879511D7D for ; Sun, 30 Jun 2019 21:00:03 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x5UL03an061708 for ; Sun, 30 Jun 2019 21:00:03 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x5UL03rY061706 for fs@FreeBSD.org; Sun, 30 Jun 2019 21:00:03 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201906302100.x5UL03rY061706@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 30 Jun 2019 21:00:03 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jun 2019 21:00:05 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 211491 | System hangs after "Uptime" on reboot with ZFS Open | 221909 | [ZFS] Add a sysctl to toggle send_corrupt_data Open | 235665 | panic: handle_written_inodeblock: live inodedep Open | 237067 | ZFS: Crash in vdev_dtl_reassess when using GELI w 6 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Mon Jul 1 14:01:48 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 210E815DB987 for ; Mon, 1 Jul 2019 14:01:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id ABB2D77A8F for ; Mon, 1 Jul 2019 14:01:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6A0E315DB986; Mon, 1 Jul 2019 14:01:47 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55E6C15DB985 for ; Mon, 1 Jul 2019 14:01:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E671D77A8A for ; Mon, 1 Jul 2019 14:01:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 3AAF91B0B4 for ; Mon, 1 Jul 2019 14:01:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x61E1kRT035975 for ; Mon, 1 Jul 2019 14:01:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x61E1k9x035974 for fs@FreeBSD.org; Mon, 1 Jul 2019 14:01:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 237517] ZFS parallel mounting sometimes misses mounting intermediate filesystems Date: Mon, 01 Jul 2019 14:01:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ben@morrow.me.uk X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2019 14:01:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237517 --- Comment #10 from Ben Morrow --- You can turn the parallel mounting off by setting ZFS_MOUNT_SERIAL=3D1 in t= he environment when you run either `zfs mount -a` or `zfs import` (without -N). IMHO rc.d/zfs should do this until the code is fixed properly. I had a brief look at the code and it's trying to do the right thing; I can= 't yet see why it's not working as it should. Does anyone know if the same pro= blem exists on other OSes? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Jul 2 04:59:27 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4945B15C572E for ; Tue, 2 Jul 2019 04:59:27 +0000 (UTC) (envelope-from pigs@solidcool.icu) Received: from solidcool.icu (zky5jw2w.ni.net.tr [89.252.174.251]) by mx1.freebsd.org (Postfix) with ESMTP id 76AFB6DB91 for ; Tue, 2 Jul 2019 04:59:26 +0000 (UTC) (envelope-from pigs@solidcool.icu) From: " Harry Santiago" Date: Mon, 01 Jul 2019 23:55:52 -0500 MIME-Version: 1.0 Subject: The closest thing to flying a REAL plane! To: Message-ID: X-Rspamd-Queue-Id: 76AFB6DB91 X-Spamd-Bar: +++++ X-Spamd-Result: default: False [5.26 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[solidcool.icu:s=mail]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:89.252.174.251]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; HTML_SHORT_LINK_IMG_1(2.00)[]; NEURAL_SPAM_MEDIUM(1.00)[0.999,0]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_SHORT(0.96)[0.960,0]; DKIM_TRACE(0.00)[solidcool.icu:+]; DMARC_POLICY_ALLOW(-0.50)[solidcool.icu,quarantine]; NEURAL_SPAM_LONG(1.00)[1.000,0]; MX_GOOD(-0.01)[cached: aspmx.l.google.com]; SUBJECT_ENDS_EXCLAIM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; IP_SCORE(1.31)[ipnet: 89.252.174.0/24(3.51), asn: 51559(2.98), country: TR(0.07)]; ASN(0.00)[asn:51559, ipnet:89.252.174.0/24, country:TR]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2019 04:59:27 -0000 Have You Always Dream of Taking To The Skies Like A Real Pilot? Fly With A Worldwide Network of Players Land Anywhere Worldwide with 25,000 Airports! Experience the REAL EFFECTS of the wind and weather... If you're flying through turbulent weather.. the controls will lag correctly.. just like in real life. If you don't want to receive these email from us pleaseUse me 28 W. Colonial Ave. Ellenwood, GA 30294 From owner-freebsd-fs@freebsd.org Tue Jul 2 14:58:43 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE4DE15D5E1F for ; Tue, 2 Jul 2019 14:58:43 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from dg.fsn.hu (dg.fsn.hu [84.2.225.196]) by mx1.freebsd.org (Postfix) with ESMTP id 48CAB89911 for ; Tue, 2 Jul 2019 14:58:42 +0000 (UTC) (envelope-from bra@fsn.hu) Received: by dg.fsn.hu (Postfix, from userid 1003) id A88EF7640; Tue, 2 Jul 2019 16:58:33 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 14.8305] X-CRM114-CacheID: sfid-20190702_16583_0D8A80A5 X-CRM114-Status: Good ( pR: 14.8305 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Tue Jul 2 16:58:32 2019 X-DSPAM-Confidence: 0.9899 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 5d1b7118409731295823851 X-DSPAM-Factors: 27, To*fs+FreeBSD.org, 0.01000, 13G, 0.01000, import, 0.01000, import, 0.01000, after+it, 0.01000, it+came, 0.01000, Received*(localhost+[127.0.0.1]), 0.01000, with+ZFS), 0.01000, 2+35, 0.01000, right, 0.01000, 5+1, 0.01000, zpool, 0.01000, zpool, 0.01000, 20+0, 0.01000, 20+0, 0.01000, solve, 0.01000, 24%25, 0.01000, 00%25, 0.01000, 00%25, 0.01000, and+not, 0.01000, 01%25+sshd, 0.01000, 01%25+sshd, 0.01000, Received*Tue, 0.01000, 10, 0.01000, 10, 0.01000, Date*2+Jul, 0.01000, X-Spambayes-Classification: ham; 0.00 Received: from [127.0.0.1] (localhost [127.0.0.1]) by dg.fsn.hu (Postfix) with ESMTP id 38028763E for ; Tue, 2 Jul 2019 16:58:30 +0200 (CEST) To: freebsd-fs@FreeBSD.org From: "Nagy, Attila" Subject: ZFS exhausts kernel memory just by importing zpools Message-ID: Date: Tue, 2 Jul 2019 16:58:29 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 48CAB89911 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of bra@fsn.hu designates 84.2.225.196 as permitted sender) smtp.mailfrom=bra@fsn.hu X-Spamd-Result: default: False [4.26 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_MEDIUM(0.96)[0.957,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[fsn.hu]; NEURAL_SPAM_SHORT(0.76)[0.764,0]; MX_GOOD(-0.01)[dg.fsn.hu]; NEURAL_SPAM_LONG(1.00)[0.999,0]; IP_SCORE(1.75)[ipnet: 84.2.224.0/21(4.92), asn: 15555(3.81), country: HU(0.02)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15555, ipnet:84.2.224.0/21, country:HU]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2019 14:58:43 -0000 Hi, Running latest stable/12 on amd64 with 64 GiB memory on a machine with 44 4T disks. Each disks have its own zpool on it (because I solve the redundancy between machines and not locally with ZFS). One example zpool holds 2.2 TiB of data (according to df) and have around 75 million files in hashed directories, this is the typical usage on them. When I import these zpools, top says around 50 GiB wired memory (ARC is minimal, files weren't yet touched) and after I start to use (heavy reads/writes) the pools, the free memory quickly disappears (ARC grows) until all memory is gone and the machine starts to kill processes, ends up in a deadlock, where nothing helps. If I import the pools one by one, each of them adds around 1-1.5 GiB of wired memory. Top shows this, right after it came to a halt and nothing else works (I can't log in even on the console): last pid: 61878;  load averages:  5.05,  4.42,  2.50    up 0+01:07:23  15:45:17 171 processes: 1 running, 162 sleeping, 1 stopped, 1 zombie, 6 waiting CPU:  0.0% user,  0.0% nice,  0.2% system,  0.0% interrupt, 99.8% idle Mem: 7716K Active, 8192 Inact, 84K Laundry, 57G Wired, 180M Buf, 14M Free ARC: 21G Total, 10G MFU, 4812M MRU, 4922M Anon, 301M Header, 828M Other      5739M Compressed, 13G Uncompressed, 2.35:1 Ratio Swap:   PID USERNAME    THR PRI NICE   SIZE    RES STATE    C TIME    WCPU COMMAND 61412 root          1  20    0    14M  3904K CPU14   14 0:06   1.55% top 57569 redis        57  20    0  1272M    64M uwait   22 4:28   0.24% consul  5574 root          1  20    0    13M  3440K nanslp  10 0:02   0.05% gstat  5557 root          1  20    0    20M  7808K select  20 0:00   0.01% sshd  5511 root          1  20    0    20M  7808K select   4 0:01   0.01% sshd  4955 root          1  20    0    10M  1832K select   9 0:00   0.01% supervis  5082 root          1  20    0    25M    14M select   0 0:00   0.00% perl  4657 _pflogd       1  20    0    12M  2424K bpf      1 0:00   0.00% pflogd  5059 elasticsea    2  20  -20  6983M   385M STOP     5 1:29   0.00% java 61669 root          1  26    0    23M      0 pfault   4 0:14   0.00% Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BBE9F15D7822 for ; Tue, 2 Jul 2019 16:13:41 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 435888C606 for ; Tue, 2 Jul 2019 16:13:40 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: by mail-ed1-x529.google.com with SMTP id r12so27839528edo.5 for ; Tue, 02 Jul 2019 09:13:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=9HqRpJfo40SzQSqcyGhaWMBfhZlssY00Ore4kRiwK6Q=; b=IID3vQXVLpApeS2hV4IvFt5fe5VVhlsgRPdTCR1jKpx1FAZXM4UC5elhRVDmbO57du OiTp4583292i7RWT798CPs1IMQZiPeCBtCjFeSLpX6Ao0NUmoOXjb6AQhB4f2Yg04PIm /CeeXC2llAe+ucckPzZ+ps+4qIFEF4sNtIIz102BVRKtoAPziTSLpTBOGlHE79pfWNCp xwD3g73UCxWDY6qFZ9uBwO/uz/9RO8hUYVxKz/jeFxKPnDfxTdnJHFMgdDT8KUUOni0n M6zj+AnCE3QvBojLsCVm4MZ5Uy1zlVxrd1QFJEqrDbHqO3uxKqrWkzvBFvcBgrDFOV9o RtBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=9HqRpJfo40SzQSqcyGhaWMBfhZlssY00Ore4kRiwK6Q=; b=gSPfJDjTWS3RufgV6ehcbX0YC/ZONJ2ngjA8fyfDRJkKrY7eZZ9FPERsEHx0RQBewk +5qhMO0lZ9MkFejffKX4HtaCN+jjgcswiM+b5c/WcHEhAznD2xd8oMwCTdKUamAUcMos MmUhWm4hgnx6NxnjY6UfLikrBda5vO8LP1dr3mPUsPv7QHXjhMjr3RaO1SYIY4H8K/Q4 TateFGbwxKVlnFOxNB/Xdr3uR2Ru082/GQkCc8AaqJFxtTaGRGvW8bnGVJHE+IDkTM9d 4wlzVtI8ui2dKz4VWJbtCqcT0tPTol7e99Uf8R/G2nmB3IqkDRoJNxDmACF7MeSh4btU sEFQ== X-Gm-Message-State: APjAAAUG8iwhBaP6G6TmgzJ8bTr2MEk9LknLURY+RaGbjzrWLxYS1rGc Lw4NQqBncCsbMG3uAfLrR9j9ovg7im1YyBBiRWUjSiLV X-Google-Smtp-Source: APXvYqw6a5HOLK5Ul0RCpfAZCIqShsqXEuPc3pC8RAWs5Bskz+0WlIxK+ZsY8qUhMmNMBhDskrKZz3wauXZHP5Eh9ZU= X-Received: by 2002:a17:906:6055:: with SMTP id p21mr29971071ejj.35.1562084019129; Tue, 02 Jul 2019 09:13:39 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: "Sam Fourman Jr." Date: Tue, 2 Jul 2019 12:13:28 -0400 Message-ID: Subject: Re: ZFS exhausts kernel memory just by importing zpools To: "Nagy, Attila" Cc: FreeBSD FS X-Rspamd-Queue-Id: 435888C606 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=IID3vQXV; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of sfourman@gmail.com designates 2a00:1450:4864:20::529 as permitted sender) smtp.mailfrom=sfourman@gmail.com X-Spamd-Result: default: False [-6.87 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.970,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[9.2.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.89)[ip: (-9.39), ipnet: 2a00:1450::/32(-2.65), asn: 15169(-2.36), country: US(-0.06)]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2019 16:13:42 -0000 Hello, My initial guess is that you may have de-duplication enabled on one (or more) of the underlying datasets. **if** this is the case, a simple solution is to add more memory to the machine. (64GB of memory is not sufficient for dedup to be enabled ) -- Sam Fourman Jr. On Tue, Jul 2, 2019 at 10:59 AM Nagy, Attila wrote: > Hi, > > Running latest stable/12 on amd64 with 64 GiB memory on a machine with > 44 4T disks. Each disks have its own zpool on it (because I solve the > redundancy between machines and not locally with ZFS). > > One example zpool holds 2.2 TiB of data (according to df) and have > around 75 million files in hashed directories, this is the typical usage > on them. > > When I import these zpools, top says around 50 GiB wired memory (ARC is > minimal, files weren't yet touched) and after I start to use (heavy > reads/writes) the pools, the free memory quickly disappears (ARC grows) > until all memory is gone and the machine starts to kill processes, ends > up in a deadlock, where nothing helps. > > If I import the pools one by one, each of them adds around 1-1.5 GiB of > wired memory. > > Top shows this, right after it came to a halt and nothing else works (I > can't log in even on the console): > > last pid: 61878; load averages: 5.05, 4.42, 2.50 up 0+01:07:23 > 15:45:17 > 171 processes: 1 running, 162 sleeping, 1 stopped, 1 zombie, 6 waiting > CPU: 0.0% user, 0.0% nice, 0.2% system, 0.0% interrupt, 99.8% idle > Mem: 7716K Active, 8192 Inact, 84K Laundry, 57G Wired, 180M Buf, 14M Free > ARC: 21G Total, 10G MFU, 4812M MRU, 4922M Anon, 301M Header, 828M Other > 5739M Compressed, 13G Uncompressed, 2.35:1 Ratio > Swap: > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > COMMAND > 61412 root 1 20 0 14M 3904K CPU14 14 0:06 1.55% top > 57569 redis 57 20 0 1272M 64M uwait 22 4:28 0.24% consul > 5574 root 1 20 0 13M 3440K nanslp 10 0:02 0.05% gstat > 5557 root 1 20 0 20M 7808K select 20 0:00 0.01% sshd > 5511 root 1 20 0 20M 7808K select 4 0:01 0.01% sshd > 4955 root 1 20 0 10M 1832K select 9 0:00 0.01% > supervis > 5082 root 1 20 0 25M 14M select 0 0:00 0.00% perl > 4657 _pflogd 1 20 0 12M 2424K bpf 1 0:00 0.00% > pflogd > 5059 elasticsea 2 20 -20 6983M 385M STOP 5 1:29 0.00% java > 61669 root 1 26 0 23M 0 pfault 4 0:14 0.00% > 61624 root 1 20 -20 24M 14M buf_ha 9 0:09 0.00% > python3. > 61626 root 1 20 -20 23M 16K pfault 0 0:08 0.00% > python3. > 61651 root 1 20 -20 23M 14M buf_ha 10 0:08 0.00% > python3. > 61668 root 1 20 -20 23M 13M buf_ha 20 0:08 0.00% > python3. > > I've already tried to shrink ARC and vm.kmem_size without too much success. > > Any ideas what causes this? > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > -- Sam Fourman Jr. From owner-freebsd-fs@freebsd.org Tue Jul 2 16:14:04 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60E8C15D7850 for ; Tue, 2 Jul 2019 16:14:04 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene.sentex.ca (unknown [IPv6:2607:f3e0:0:3::18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B6B818C614 for ; Tue, 2 Jul 2019 16:13:53 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.26] ([192.168.43.26]) by pyroxene.sentex.ca (8.15.2/8.15.2) with ESMTP id x62GDoiY061907; Tue, 2 Jul 2019 12:13:51 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: ZFS exhausts kernel memory just by importing zpools To: "Nagy, Attila" , freebsd-fs@FreeBSD.org References: From: Mike Tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= mQENBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAG0HG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT6JATgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnm5AQ0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAYkBHwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: <820ceee3-95aa-9925-066d-5d22884ce001@sentex.net> Date: Tue, 2 Jul 2019 12:13:48 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: B6B818C614 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::18 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-1.19 / 15.00]; ARC_NA(0.00)[]; RDNS_NONE(1.00)[]; NEURAL_HAM_MEDIUM(-0.84)[-0.840,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; NEURAL_HAM_LONG(-1.00)[-0.997,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[sentex.net]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.sentex.ca]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.82)[-0.818,0]; IP_SCORE(-1.72)[ipnet: 2607:f3e0::/32(-4.95), asn: 11647(-3.59), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jul 2019 16:14:04 -0000 On 7/2/2019 10:58 AM, Nagy, Attila wrote: > Hi, > > Running latest stable/12 on amd64 with 64 GiB memory on a machine with > 44 4T disks. Each disks have its own zpool on it (because I solve the > redundancy between machines and not locally with ZFS). > > One example zpool holds 2.2 TiB of data (according to df) and have > around 75 million files in hashed directories, this is the typical > usage on them. > > When I import these zpools, top says around 50 GiB wired memory (ARC > is minimal, files weren't yet touched) and after I start to use (heavy > reads/writes) the pools, the free memory quickly disappears (ARC > grows) until all memory is gone and the machine starts to kill > processes, ends up in a deadlock, where nothing helps. > > If I import the pools one by one, each of them adds around 1-1.5 GiB > of wired memory. Hi,     You mean you have 44 different zpools ?  75mil files per pool sounds like a lot. I wonder for testing purposes, you made 1 or two zpools with 44 (or 22) different datasets and had 3.3billion files, would you run into the same memory exhaustion ?     ---Mike From owner-freebsd-fs@freebsd.org Wed Jul 3 14:34:16 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9916F15D5876 for ; Wed, 3 Jul 2019 14:34:16 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from dg.fsn.hu (dg.fsn.hu [84.2.225.196]) by mx1.freebsd.org (Postfix) with ESMTP id 4065476177 for ; Wed, 3 Jul 2019 14:34:15 +0000 (UTC) (envelope-from bra@fsn.hu) Received: by dg.fsn.hu (Postfix, from userid 1003) id 7EA997B9F; Wed, 3 Jul 2019 16:34:13 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 27.7999] X-CRM114-CacheID: sfid-20190703_16340_61A47AA6 X-CRM114-Status: Good ( pR: 27.7999 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Wed Jul 3 16:34:11 2019 X-DSPAM-Confidence: 0.9899 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 5d1cbce3556461935321310 X-DSPAM-Factors: 27, 13G, 0.01000, 13G, 0.01000, underlying, 0.01000, underlying, 0.01000, import, 0.01000, import, 0.01000, nanslp%c2%a0+10, 0.01000, nanslp%c2%a0+10, 0.01000, after+it, 0.01000, after+it, 0.01000, it+came, 0.01000, it+came, 0.01000, To*, 0.01000, 23M%c2%a0%c2%a0%c2%a0+16K, 0.01000, 23M%c2%a0%c2%a0%c2%a0+16K, 0.01000, >+(or, 0.01000, 5+1, 0.01000, 5+1, 0.01000, X-Spambayes-Classification: ham; 0.00 Received: from [127.0.0.1] (localhost [127.0.0.1]) by dg.fsn.hu (Postfix) with ESMTP id A1FED7B9D; Wed, 3 Jul 2019 16:34:09 +0200 (CEST) Subject: Re: ZFS exhausts kernel memory just by importing zpools To: "Sam Fourman Jr." Cc: FreeBSD FS References: From: "Nagy, Attila" Message-ID: <21b04b21-8850-c3c3-36c9-a0d0ede4dc22@fsn.hu> Date: Wed, 3 Jul 2019 16:34:08 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Rspamd-Queue-Id: 4065476177 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of bra@fsn.hu designates 84.2.225.196 as permitted sender) smtp.mailfrom=bra@fsn.hu X-Spamd-Result: default: False [3.89 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_SPAM_SHORT(0.39)[0.389,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[fsn.hu]; NEURAL_SPAM_MEDIUM(0.98)[0.975,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: dg.fsn.hu]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(1.00)[0.996,0]; IP_SCORE(1.74)[ipnet: 84.2.224.0/21(4.90), asn: 15555(3.78), country: HU(0.02)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:15555, ipnet:84.2.224.0/21, country:HU]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jul 2019 14:34:16 -0000 Hi, Oh, should've written about that: no I don't use (and never used) dedup. On 2019. 07. 02. 18:13, Sam Fourman Jr. wrote: > Hello, > > My initial guess is that you may have de-duplication enabled on one > (or more) of the underlying datasets. > **if** this is the case, a simple solution is to add more memory to > the machine. (64GB of memory is not sufficient for dedup to be enabled ) > > -- Sam Fourman Jr. > > On Tue, Jul 2, 2019 at 10:59 AM Nagy, Attila > wrote: > > Hi, > > Running latest stable/12 on amd64 with 64 GiB memory on a machine > with > 44 4T disks. Each disks have its own zpool on it (because I solve the > redundancy between machines and not locally with ZFS). > > One example zpool holds 2.2 TiB of data (according to df) and have > around 75 million files in hashed directories, this is the typical > usage > on them. > > When I import these zpools, top says around 50 GiB wired memory > (ARC is > minimal, files weren't yet touched) and after I start to use (heavy > reads/writes) the pools, the free memory quickly disappears (ARC > grows) > until all memory is gone and the machine starts to kill processes, > ends > up in a deadlock, where nothing helps. > > If I import the pools one by one, each of them adds around 1-1.5 > GiB of > wired memory. > > Top shows this, right after it came to a halt and nothing else > works (I > can't log in even on the console): > > last pid: 61878;  load averages:  5.05,  4.42,  2.50    up 0+01:07:23 > 15:45:17 > 171 processes: 1 running, 162 sleeping, 1 stopped, 1 zombie, 6 waiting > CPU:  0.0% user,  0.0% nice,  0.2% system,  0.0% interrupt, 99.8% idle > Mem: 7716K Active, 8192 Inact, 84K Laundry, 57G Wired, 180M Buf, > 14M Free > ARC: 21G Total, 10G MFU, 4812M MRU, 4922M Anon, 301M Header, 828M > Other >       5739M Compressed, 13G Uncompressed, 2.35:1 Ratio > Swap: > >    PID USERNAME    THR PRI NICE   SIZE    RES STATE    C TIME    WCPU > COMMAND > 61412 root          1  20    0    14M  3904K CPU14   14 0:06 1.55% top > 57569 redis        57  20    0  1272M    64M uwait   22 4:28 0.24% > consul >   5574 root          1  20    0    13M  3440K nanslp  10 0:02   > 0.05% gstat >   5557 root          1  20    0    20M  7808K select  20 0:00   > 0.01% sshd >   5511 root          1  20    0    20M  7808K select   4 0:01   > 0.01% sshd >   4955 root          1  20    0    10M  1832K select   9 0:00   0.01% > supervis >   5082 root          1  20    0    25M    14M select   0 0:00   > 0.00% perl >   4657 _pflogd       1  20    0    12M  2424K bpf      1 0:00   > 0.00% pflogd >   5059 elasticsea    2  20  -20  6983M   385M STOP     5 1:29   > 0.00% java > 61669 root          1  26    0    23M      0 pfault   4 0:14 0.00% > 61624 root          1  20  -20    24M    14M buf_ha   9 0:09 0.00% > python3. > 61626 root          1  20  -20    23M    16K pfault   0 0:08 0.00% > python3. > 61651 root          1  20  -20    23M    14M buf_ha  10 0:08 0.00% > python3. > 61668 root          1  20  -20    23M    13M buf_ha  20 0:08 0.00% > python3. > > I've already tried to shrink ARC and vm.kmem_size without too much > success. > > Any ideas what causes this? > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to > "freebsd-fs-unsubscribe@freebsd.org > " > > > > -- > > Sam Fourman Jr. From owner-freebsd-fs@freebsd.org Wed Jul 3 14:37:50 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09E3F15D5AD1 for ; Wed, 3 Jul 2019 14:37:50 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from dg.fsn.hu (dg.fsn.hu [84.2.225.196]) by mx1.freebsd.org (Postfix) with ESMTP id 360B6763B1 for ; Wed, 3 Jul 2019 14:37:49 +0000 (UTC) (envelope-from bra@fsn.hu) Received: by dg.fsn.hu (Postfix, from userid 1003) id 0FBE37BA4; Wed, 3 Jul 2019 16:37:49 +0200 (CEST) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 12.8040] X-CRM114-CacheID: sfid-20190703_16374_5A5B0BD2 X-CRM114-Status: Good ( pR: 12.8040 ) X-DSPAM-Result: Whitelisted X-DSPAM-Processed: Wed Jul 3 16:37:48 2019 X-DSPAM-Confidence: 0.9899 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 5d1cbdbc557181928486798 X-DSPAM-Factors: 27, To*fs+FreeBSD.org, 0.01000, import, 0.01000, import, 0.01000, run+>, 0.01000, but, 0.01000, Received*(localhost+[127.0.0.1]), 0.01000, with+ZFS), 0.01000, zpool, 0.01000, zpool, 0.01000, AM, 0.01000, To*>, 0.01000, 02+18, 0.01000, of, 0.01000, of, 0.01000, zpool+on, 0.01000, To*Mike, 0.01000, 75+million, 0.01000, From*"Nagy, Attila" , 0.01000, scales, 0.01000, X-Spambayes-Classification: ham; 0.00 Received: from [127.0.0.1] (localhost [127.0.0.1]) by dg.fsn.hu (Postfix) with ESMTP id 1F6407BA2; Wed, 3 Jul 2019 16:37:47 +0200 (CEST) Subject: Re: ZFS exhausts kernel memory just by importing zpools To: Mike Tancsa , freebsd-fs@FreeBSD.org References: <820ceee3-95aa-9925-066d-5d22884ce001@sentex.net> From: "Nagy, Attila" Message-ID: <78882cea-c1aa-0d08-d2e8-7f7ae7131bb6@fsn.hu> Date: Wed, 3 Jul 2019 16:37:46 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: <820ceee3-95aa-9925-066d-5d22884ce001@sentex.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 360B6763B1 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of bra@fsn.hu designates 84.2.225.196 as permitted sender) smtp.mailfrom=bra@fsn.hu X-Spamd-Result: default: False [4.39 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_SPAM_SHORT(0.91)[0.908,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[fsn.hu]; NEURAL_SPAM_MEDIUM(0.96)[0.960,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: dg.fsn.hu]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(1.00)[0.998,0]; IP_SCORE(1.73)[ipnet: 84.2.224.0/21(4.88), asn: 15555(3.75), country: HU(0.02)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15555, ipnet:84.2.224.0/21, country:HU]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jul 2019 14:37:50 -0000 On 2019. 07. 02. 18:13, Mike Tancsa wrote: > On 7/2/2019 10:58 AM, Nagy, Attila wrote: >> Hi, >> >> Running latest stable/12 on amd64 with 64 GiB memory on a machine with >> 44 4T disks. Each disks have its own zpool on it (because I solve the >> redundancy between machines and not locally with ZFS). >> >> One example zpool holds 2.2 TiB of data (according to df) and have >> around 75 million files in hashed directories, this is the typical >> usage on them. >> >> When I import these zpools, top says around 50 GiB wired memory (ARC >> is minimal, files weren't yet touched) and after I start to use (heavy >> reads/writes) the pools, the free memory quickly disappears (ARC >> grows) until all memory is gone and the machine starts to kill >> processes, ends up in a deadlock, where nothing helps. >> >> If I import the pools one by one, each of them adds around 1-1.5 GiB >> of wired memory. > Hi, > >     You mean you have 44 different zpools ?  75mil files per pool sounds > like a lot. I wonder for testing purposes, you made 1 or two zpools with > 44 (or 22) different datasets and had 3.3billion files, would you run > into the same memory exhaustion ? > Yes, 44 different pools. I think this is related to how ZFS stores pool metadata in memory. I don't think these scales with the number of the files, but maybe with the number of stored blocks. Sadly, I can't put the same amount of data to a machine with a different setup ATM. From owner-freebsd-fs@freebsd.org Wed Jul 3 23:05:50 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AFA7D15E3429 for ; Wed, 3 Jul 2019 23:05:50 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from smtp.simplesystems.org (smtp.simplesystems.org [65.66.246.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8635B6EF9F for ; Wed, 3 Jul 2019 23:05:49 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from scrappy.simplesystems.org (scrappy.simplesystems.org [65.66.246.73]) by smtp.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id x63MqN5X016427; Wed, 3 Jul 2019 17:52:24 -0500 (CDT) Date: Wed, 3 Jul 2019 17:52:23 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@scrappy.simplesystems.org To: "Nagy, Attila" cc: freebsd-fs@freebsd.org Subject: Re: ZFS exhausts kernel memory just by importing zpools In-Reply-To: <78882cea-c1aa-0d08-d2e8-7f7ae7131bb6@fsn.hu> Message-ID: References: <820ceee3-95aa-9925-066d-5d22884ce001@sentex.net> <78882cea-c1aa-0d08-d2e8-7f7ae7131bb6@fsn.hu> User-Agent: Alpine 2.20 (GSO 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (smtp.simplesystems.org [65.66.246.90]); Wed, 03 Jul 2019 17:52:26 -0500 (CDT) X-Rspamd-Queue-Id: 8635B6EF9F X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of bfriesen@simple.dallas.tx.us designates 65.66.246.90 as permitted sender) smtp.mailfrom=bfriesen@simple.dallas.tx.us X-Spamd-Result: default: False [-1.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.66)[-0.660,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-0.96)[-0.959,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dallas.tx.us]; NEURAL_SPAM_SHORT(0.60)[0.600,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.simplesystems.org]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(0.03)[asn: 7018(0.22), country: US(-0.06)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7018, ipnet:65.64.0.0/13, country:US]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jul 2019 23:05:50 -0000 On Wed, 3 Jul 2019, Nagy, Attila wrote: > > I think this is related to how ZFS stores pool metadata in memory. I don't > think these scales with the number of the files, but maybe with the number of > stored blocks. There should be a zfs tunable to disable zfs metadata caching. This would make repeated access slower due to more reads, but might allow importing the pool. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt From owner-freebsd-fs@freebsd.org Thu Jul 4 13:31:26 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2A0015D5C18 for ; Thu, 4 Jul 2019 13:31:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 857F891FB5 for ; Thu, 4 Jul 2019 13:31:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 499CA15D5C10; Thu, 4 Jul 2019 13:31:25 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3837115D5C0F for ; Thu, 4 Jul 2019 13:31:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA3F691FA7 for ; Thu, 4 Jul 2019 13:31:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0EACC17658 for ; Thu, 4 Jul 2019 13:31:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x64DVNad084529 for ; Thu, 4 Jul 2019 13:31:23 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x64DVNjt084528 for fs@FreeBSD.org; Thu, 4 Jul 2019 13:31:23 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 238663] [UFS] Corrupted files since migration to FreeBSD11 Date: Thu, 04 Jul 2019 13:31:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: nicolas.masse@stormshield.eu X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jul 2019 13:31:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D238663 --- Comment #6 from Masse Nicolas --- Small update because some people here are reporting the fact that they got = the issue while not using the -async flag. Still, the issue happens a lot less often in this case. --=20 You are receiving this mail because: You are the assignee for the bug.=