From nobody Mon Jul 5 16:33:24 2021 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9587A7CF3D5 for ; Mon, 5 Jul 2021 16:33:26 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJWTL38GQz4kCW; Mon, 5 Jul 2021 16:33:26 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-ej1-x632.google.com with SMTP id b2so29755308ejg.8; Mon, 05 Jul 2021 09:33:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=FuFXdH+JTNk9Ag6vADkWg+V5d7GcIXQpX/2ZiYja4DM=; b=MpX+BV8I8aY25KQxt9OtgRG/ra6nvgwvtv11IdC+ndGbsXOwvLwKxgenBt/uX7QjMJ W4RzF0/+3+78EkkCOE+E2TzAiAJkrkkbsZVIRVCcWNhVfo7ppD4Qusnkrg6vBgOiuNVH 0PF9+B5vyCJxEgtrr7ZPXJ11Wpxhguvv0UTqYcJkinmATKTrw0IiW2X7JQUHQC97lazw V1DJGOx3iyPi/b0MbV9dL27fftFIVEYGD0acgPHVevZeIVGQl3Ed1RWFUrBy7mjOFhIC TiMCR1d4ZMWvpdp0znRwlniZKQYCoMdvWmGgCeKK5VpAeb2I5RIMUikr9uxiV/YstKuX h18A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=FuFXdH+JTNk9Ag6vADkWg+V5d7GcIXQpX/2ZiYja4DM=; b=lMnL7o3cxpanSxFUF1iGGFeao6VpLpVKfv5kSBrogMSRrynqZUrkNwGybDQotx0YUa Rf3YsY60ZXIORGZBh+XjJrmLiPmnqvQF1tWIom47SkkDEX18E3pf4BYAXFFXzu/rDX5i ZbG9Y+XoVAagybFIkXLhsqo87Z/Q3Y0thD8cFznkZz0XG1rMD+ZA5wtH7SZ5xj9QpUhr XpSb7MJOLaAoTrS+0NpO5vaRx7W2jtid/NJUfuUwBdC1GnlO/8rmXrujMC46aKiPMUu5 rZaSm5qDlkAjQ5U0opAfSCVYUTu/dkbZiP8NkVQQPtPiYZUjXKNNEMViHEM+3hB902vW B7ig== X-Gm-Message-State: AOAM530Xr9gnZQbS00+KfDFTAJt/u2AOOb5PARZUCVW4CcMLy9S2x9dJ uxahDNZ9o7tmEH4dj0128CtwahJNIS4= X-Google-Smtp-Source: ABdhPJy47rYoWthd2QQkNFWtezORxDRiR2Hx4xfxP6DLUQs8es2Zia1zTy1F9tJE2XPs9WTMo58FWw== X-Received: by 2002:a17:906:4c8c:: with SMTP id q12mr14181356eju.254.1625502805260; Mon, 05 Jul 2021 09:33:25 -0700 (PDT) Received: from ernst.home (pd9e2360f.dip0.t-ipconnect.de. [217.226.54.15]) by smtp.gmail.com with ESMTPSA id s5sm5820027edi.93.2021.07.05.09.33.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 09:33:24 -0700 (PDT) Date: Mon, 5 Jul 2021 18:33:24 +0200 From: Gary Jennejohn To: Edward Tomasz =?ISO-8859-1?Q?Napiera=3Fa?= Cc: Warner Losh , FreeBSD Current Subject: Re: panic: Unaligned free (was: kernel panic while copying files) Message-ID: <20210705163324.29466849@ernst.home> In-Reply-To: References: <20210608084646.6a7e1bc7@ernst.home> <20210608155405.5cf0e200@ernst.home> <20210610095041.38d7597c@ernst.home> <20210629094201.77ef5f22@ernst.home> <20210630125703.2b5544e7@ernst.home> <20210701035800.410d2376@ernst.home> <20210701113026.59f864e9@ernst.home> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GJWTL38GQz4kCW X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, 5 Jul 2021 15:04:48 +0100 Edward Tomasz Napiera__a wrote: > On 0701T1330, Gary Jennejohn wrote: > > Gary Jennejohn wrote: > > > I noticed that the value of vm.debug.divisor affects what value is > > > returned in uma_core.c:uma_dbg_kskip(), so I decided to try a few > > > different values. > > > > > > The returned value is used to set skipdbg in uma_core.c:item_dtor(). > > > > > > The default is vm.debug.divisor=1. > > > > > > vm.debug.divisor is only present when INVARIANTS is defined. > > > > > > kskipdbg eventually affects the value of freei. > > > > > > With these values: > > > vm.debug.divisor: 0 > > > kern.cam.da.enable_uma_ccbs: 1 > > > I can turn on the disk and it comes up without a panic! > > > > > > However, I didn't try to do any large data transfers to the disk. > > > > > > So, it appears that at least vm.debug.divisor is a big factor in > > > whether or not a panic happens with INVARIANTS. > > > > > > > I decided to do a real test. So I built a kernel w/o INVARIANTS and > > installed it to /boot/test. > > > > Then I stuck a 160GB disk I had around into an external USB3 enclosure > > and put a filesystem on it. > > > > The I booted the new kernel from /boot/test and set the sysctls so: > > kern.cam.da.enable_uma_ccbs: 1 > > kern.cam.ada.enable_uma_ccbs: 1 > > > > After that I plugged in the external USB3 enclosure and copied about > > 114GiB of data from an internal SSD to it - without a kernel panic: > > Filesystem Size Used Avail Capacity Mounted on > > /dev/da0p1 144G 114G 18G 86% /mnt > > > > I'm pretty sure that's more than I could copy without a kernel panic > > prior to the recent changes made in cam and umass. > > > > My test may not be real proof that all bugs have been squashed, but it > > certainly seems to be a better situation than we had before. > > I think the vm.debug.divisor simply masks the problem; the underlying > bug is still there. > > Could you go back to the setup which panics, and then test the patch > at https://reviews.freebsd.org/D31054? It fixes the scenario described > by Warner. > It looks like this patch fixes things. I used the default value vm.debug.divisor=1 and both enable_uma_ccbs=1 (which are now the default values on my system). I used the 8TiB disk, which spins up very slowly and usually resulted very quickly in a panic - no panic with the patch. Then using dd to /dev/null (bs=1m) I transferred: 308755+0 records in 308755+0 records out 323753082880 bytes transferred in 1366.162410 secs (236979938 bytes/sec) from the disk, so about 324GiB without a panic. -- Gary Jennejohn