From owner-freebsd-current@freebsd.org Sat Oct 31 02:26:32 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 93B26464F18 for ; Sat, 31 Oct 2020 02:26:32 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com [IPv6:2607:f8b0:4864:20::830]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CNNN72FWMz3ZtB for ; Sat, 31 Oct 2020 02:26:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x830.google.com with SMTP id k90so14236qte.2 for ; Fri, 30 Oct 2020 19:26:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xANjZJBGTuiEBJvECNrW+zBXJgEPHo/mVCz/+B1FXok=; b=Udh05liZYICKSHrDgd2SAXUS4Q2QcjYgj3QWKtSopHm2Oa71+VFeINMW7tZDlVtoFO NSZDNDmXutoKrqOdDXrQLb+DvCNfcS/W9REMyLFsXBgBkcaR/dy5st4vFXx3UwNOi9rW 8Bl73MdkpkiiQ/6ckvl/bbhbOIARcXFtIQxwsop+1tJMh4T0oEA1AE756DZqODr3upvN 91EU1q2e0blWkmgZ2GqDzrmryC5LJZjGrSH0olX79HZo1tFAAtKx2fLz2KKp35ERtwZn t9VBD9K5c2GatqM3c7iyEPf/g+CAPOReXC91Y6s/grPBcIyaGALLUkZ09AIdKZke2Mhj IX0w== X-Gm-Message-State: AOAM531VdDiOXZRgCtc14e4TsiOVuxD5OJ3bFQWpga2/wqXNaZHlLPfL 1Mvn/aBt1WCMtGLd57/aUaxJ4G01/m16MQdbqpYA2Q== X-Google-Smtp-Source: ABdhPJzQ2bAj1dOJf4dIpiNrQpBDPgp/xY4llPMsk3qiQmWiRk087N2WPRrbW7jg359WhE10PQnKWMkXg7c13DkE0OU= X-Received: by 2002:a05:622a:10b:: with SMTP id u11mr5117687qtw.235.1604111190087; Fri, 30 Oct 2020 19:26:30 -0700 (PDT) MIME-Version: 1.0 References: <202010300313.09U3D0KZ006216@slippy.cwsent.com> <20201030204622.GF2033@zxy.spb.ru> <202010302053.09UKrAXc031272@slippy.cwsent.com> <20201030220809.GG2033@zxy.spb.ru> <202010302234.09UMYA5d032018@slippy.cwsent.com> <20201030224734.GH2033@zxy.spb.ru> <202010302300.09UN0t4A032372@slippy.cwsent.com> <20201030233138.GD34923@zxy.spb.ru> <202010302350.09UNoVcM033686@slippy.cwsent.com> <202010310041.09V0ffBL035185@slippy.cwsent.com> In-Reply-To: <202010310041.09V0ffBL035185@slippy.cwsent.com> From: Warner Losh Date: Fri, 30 Oct 2020 20:26:19 -0600 Message-ID: Subject: Re: OpenZFS: kldload zfs.ko freezes on i386 4GB memory To: Cy Schubert Cc: Matthew Macy , Slawa Olhovchenkov , qroxana , freebsd-current X-Rspamd-Queue-Id: 4CNNN72FWMz3ZtB X-Spamd-Bar: / X-Spamd-Result: default: False [-0.50 / 15.00]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; MAILMAN_DEST(0.00)[freebsd-current]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_LONG(-1.02)[-1.022]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::830:from]; NEURAL_HAM_MEDIUM(-1.03)[-1.031]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; DMARC_NA(0.00)[bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; NEURAL_SPAM_SHORT(0.05)[0.050]; FREEMAIL_CC(0.00)[gmail.com,zxy.spb.ru,mail.ru,freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; SUSPICIOUS_RECIPS(1.50)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Oct 2020 02:26:32 -0000 On Fri, Oct 30, 2020 at 6:42 PM Cy Schubert wrote: > In message > om> > , Matthew Macy writes: > > On Fri, Oct 30, 2020 at 4:50 PM Cy Schubert > wrote > > : > > > > > > In message <20201030233138.GD34923@zxy.spb.ru>, Slawa Olhovchenkov > writes: > > > > On Fri, Oct 30, 2020 at 04:00:55PM -0700, Cy Schubert wrote: > > > > > > > > > > > > More stresses memory usually refers to performance penalty. > > > > > > > > Usually way for better performance is reduce memory access. > > > > > > > > > > > > > > The reason filesystems (UFS, ZFS, EXT4, etc.) cache is to > avoid dis > > k > > > > > > > accesses. Nanoseconds vs milliseconds. > > > > > > > > > > > > I mean compared ZoL ZFS ARC vs old (BSD/Opensolaris/Illumos) ZFS > ARC. > > > > > > Any reaason to rise ARC hit rate in ZoL case? > > > > > > > > > > That's what hit rate is. It's a memory access instead of a disk > access. > > > > > That's what you want. > > > > > > > > Is ZoL ARC hit rate rise from FreeBSD ARC hit rate? > > > > > > We don't know that. You should be able to find out by running some > tests > > > that would populate your ARC and run the test again. I see that my > > > -DNO_CLEAN buildworlds run faster, when I run them a second or third > time > > > after making a minor edit, than they did before. Thus I assume it uses > > > memory more efficiently. By default it stores more metadata in ARC, 75% > > > instead of IIRC 25% by default. > > > > > > Getting back to your original question. A more efficient ARC would > exercise > > > your memory more intensely because you are replacing disk reads with > memory > > > reads. And as I said before the old ZFS "found" weak RAM on three > separate > > > occasions in three different machines over the last ten years. You're > > > advised to replace the marginal memory. > > > > Ryan has been able to reproduce this in a VM with 4GB, similarly a VM > > with 2GB loads just fine. It would seem that 4GB triggers a bug in > > limit handling. We're hoping that we can simply lower one of the > > default limits on i386 and make the problem go away. > > > > Please don't shoot the messenger when I observe that, generally > > speaking, i386 is considered a self supported platform due to ZFS > > general inability to perform well with limited memory or KVA. Long > > mode has been available on virtually all processors shipped since > > 2006. > > Yes, I was able to use ZFS on a 2 GB Pentium-M (i386) laptop for many > years. ZFS worked well with a little tuning on such a small machine. Last > time I booted it was late last year or early this year. It's in a drawer > right now. I'll try to pull it out this coming week to test it out. > > Serendipitous that I was thinking about pulling out that old laptop to > test > out the new ZFS just last week. > History has shown that as we tune by default for modern machines, the older machines will need more and more tuning to get reasonable performance. We had issues with the default number of bufs, for example, on small memory footprint machines. The first order tuning helped, but it was only a matter of time before even that was not enough. I suspect that using a smaller ARC on 32-bit platforms will suffice, for now. We should learn from history and understand that it's the first step down the path to the setup not working and gently encourage people to use this time to retool their platforms. This likely will take a year or four if history is any indicator, so there's plenty of time to retool... Warner