From nobody Thu Oct 20 14:32:55 2022 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MtVSV0Kk3z4fT1L; Thu, 20 Oct 2022 14:32:58 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc2f.google.com (mail-oo1-xc2f.google.com [IPv6:2607:f8b0:4864:20::c2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MtVST2nRNz3y1h; Thu, 20 Oct 2022 14:32:57 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-oo1-xc2f.google.com with SMTP id r11-20020a4aa2cb000000b004806f49e27eso4010019ool.7; Thu, 20 Oct 2022 07:32:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=/F8ysxKQPNWjg9y9yqeGD1E4zX5t7vzcXV5NeFpk/UY=; b=Q2cDrNpah/F6Gz0FQTecgVcd0N8kYxOSXozjLT8tKZ7Y2ntjCGv5cLSufMGbnW+tT5 gYwvJtDx9F4aYaGCPaqlHJabyRWYAIId9aNAo/ugTmcWbgZ7VHI4AbE4wU4FbBAdTj2m JyWJM4tzIsbeHrvaPn1jYNgSswYbw2mQ6HrinhnsYHGT4lQ9d1Cbr0ZzVsHPzk0bH4az LRQASTr36Ym7eAFBdW42hVibcbTyrKull0HLPS75v64Q/KZ/1M3n0YKr+8sMMyu3l3e1 e9eiW9gX84IkO56OqnEAjfK76/ZGFFM+G3gMLViuKo9HvlkQwcS9Eq7EnYokHKRsEyn0 8hNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/F8ysxKQPNWjg9y9yqeGD1E4zX5t7vzcXV5NeFpk/UY=; b=PSmmIJudnfgF9XowZVRYx0a78u+D/F+U5pZkQogk6oqpUFZ+1nP4njNn0EC8lfpbRS FXVEMR4FnxrItdhEZfLeEBixJx8+z9vybXmrX88PTuOcQuDT68tdFVPCTxAMgnVOtE23 Ekn7N+tAZLpoi9HPmtfgi2nJ9pY6oF8l4eFYUDls+Fomy2iMf81a67dpP5+lZQ3FPLRw 65WvtSluGCRkxIZr0ngIRlheH+inM3Ks23+OO+EW4Bt7c6o6CfRKDaUvbIert3s4qiAR HmijlRpnA7bm0/NvUq3j+CF9LBMEIBDQ2NwbVWr/DKJnq0xmo1tvcWkarc6L6tt7scqW bERA== X-Gm-Message-State: ACrzQf2zq1fEonxbS9zjpZuuJx+heWKkpFeiO1QjUD1xIbCMzmzqXfLe MNiqF3itW6YPzOsv0swtS2cJVoBWAraop6qJ5kqj9A2w X-Google-Smtp-Source: AMsMyM6C4nm3b5xVGnIPKLuwISkVD5vqpp24aAPmRxrFBQ8HgYuX8OauKTqmu6ZyLF27xAiYpS69IjfjCN3wgKZYV9w= X-Received: by 2002:a4a:b447:0:b0:480:9c98:46a3 with SMTP id h7-20020a4ab447000000b004809c9846a3mr6449078ooo.14.1666276376322; Thu, 20 Oct 2022 07:32:56 -0700 (PDT) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Received: by 2002:a8a:351:0:b0:478:5e54:a9c8 with HTTP; Thu, 20 Oct 2022 07:32:55 -0700 (PDT) In-Reply-To: <93085f3c-4592-8f46-fadb-ba0ee2d2ffa9@FreeBSD.org> References: <93085f3c-4592-8f46-fadb-ba0ee2d2ffa9@FreeBSD.org> From: Mateusz Guzik Date: Thu, 20 Oct 2022 16:32:55 +0200 Message-ID: Subject: Re: panic: condition seqc_in_modify not met, while replaying ZIL To: Andriy Gapon Cc: freebsd-stable List , freebsd-fs Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4MtVST2nRNz3y1h X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=Q2cDrNpa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2607:f8b0:4864:20::c2f as permitted sender) smtp.mailfrom=mjguzik@gmail.com X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[stable@freebsd.org,fs@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::c2f:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_ALL(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N This is a false-positive -- the seqc stuff is not modified on replay and zfs internal asserts check for it. I think the easiest way forward for this one is to appease the assert by entering it seqc. That said, could it be it is an invariant there are no nc entries at this stage anyway and the call to cache_vop_rmdir can be avoided? I can add cache_assert_no_entries or similar routine to be called instead. On 10/20/22, Andriy Gapon wrote: > > This happens on stable/13, custom kernel compiled with DEBUG_VFS_LOCKS, from > > mid-September after an ungraceful reboot (unrelated crash). As far as I can > see > in kgdb, both dvp and vp have v_seqc == 0. > > VNASSERT failed: ({ seqc_t __seqc = (_vp->v_seqc); __builtin_expect((__seqc > & > 1), 0); }) not true at > /usr/home/avg/devel/freebsd-src-new/machines/trant/sys/kern/vfs_cache.c:2976 > > (cache_vop_rmdir) > 0xfffff8004bc0bd58: type VDIR > usecount 1, writecount 0, refcount 1 seqc users 0 mountedhere 0 > hold count flags () > flags () > lock type zfs: UNLOCKED > panic: condition seqc_in_modify(_vp->v_seqc) not met at > /usr/home/avg/devel/freebsd-src-new/machines/trant/sys/kern/vfs_cache.c:2976 > > (cache_vop_rmdir) > cpuid = 5 > time = 1666250711 > KDB: stack backtrace: > db_trace_self_wrapper() at 0xffffffff8061549b = > db_trace_self_wrapper+0x2b/frame > 0xfffffe01ce6c7090 > kdb_backtrace() at 0xffffffff80942927 = kdb_backtrace+0x37/frame > 0xfffffe01ce6c7140 > vpanic() at 0xffffffff808f4fb4 = vpanic+0x184/frame 0xfffffe01ce6c71a0 > panic() at 0xffffffff808f4d63 = panic+0x43/frame 0xfffffe01ce6c7200 > cache_vop_rmdir() at 0xffffffff809b9b9f = cache_vop_rmdir+0xdf/frame > 0xfffffe01ce6c7220 > zfs_rmdir_() at 0xffffffff80391cff = zfs_rmdir_+0x1df/frame > 0xfffffe01ce6c7290 > zfs_rmdir() at 0xffffffff80391af8 = zfs_rmdir+0x48/frame 0xfffffe01ce6c7310 > zfs_replay_remove() at 0xffffffff804fc53b = zfs_replay_remove+0x7b/frame > 0xfffffe01ce6c7340 > zil_replay_log_record() at 0xffffffff80508462 = > zil_replay_log_record+0x212/frame 0xfffffe01ce6c7490 > zil_parse() at 0xffffffff80502f4b = zil_parse+0x5cb/frame > 0xfffffe01ce6c76a0 > zil_replay() at 0xffffffff805081d8 = zil_replay+0xd8/frame > 0xfffffe01ce6c7700 > zfsvfs_setup() at 0xffffffff8038f09d = zfsvfs_setup+0x24d/frame > 0xfffffe01ce6c7930 > zfs_mount() at 0xffffffff8038c9b2 = zfs_mount+0x652/frame > 0xfffffe01ce6c7ad0 > vfs_domount_first() at 0xffffffff809d0d66 = vfs_domount_first+0x216/frame > 0xfffffe01ce6c7c00 > vfs_domount() at 0xffffffff809cdd43 = vfs_domount+0x2d3/frame > 0xfffffe01ce6c7d30 > vfs_donmount() at 0xffffffff809ccb6f = vfs_donmount+0x81f/frame > 0xfffffe01ce6c7dc0 > sys_nmount() at 0xffffffff809cc318 = sys_nmount+0x108/frame > 0xfffffe01ce6c7df0 > amd64_syscall() at 0xffffffff80c31d16 = amd64_syscall+0x186/frame > 0xfffffe01ce6c7f30 > fast_syscall_common() at 0xffffffff80c0889b = fast_syscall_common+0xf8/frame > > 0xfffffe01ce6c7f30 > --- syscall (378, FreeBSD ELF64, sys_nmount), rip = 0xb6aac9dd1a, rsp = > 0xb6b538f468, rbp = 0xb6b538f4d0 --- > Uptime: 37s > Dumping 1426 out of 32644 MB: (CTRL-C to abort) > ..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > -- > Andriy Gapon > -- Mateusz Guzik