From owner-freebsd-current@freebsd.org Fri Oct 9 23:27:20 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 151EE43024F for ; Fri, 9 Oct 2020 23:27:20 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C7PP36jRLz3YV8 for ; Fri, 9 Oct 2020 23:27:19 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Received: from Ryans-MBP.attlocal.net (unknown [IPv6:2600:1700:358a:c660:3c7d:a09a:5a3f:b268]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: freqlabs/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id ABEA1103BB for ; Fri, 9 Oct 2020 23:27:19 +0000 (UTC) (envelope-from freqlabs@FreeBSD.org) Subject: Re: ZFS crash -- zvol_geom_bio_getattr called when volmode=dev To: freebsd-current@freebsd.org References: <1dfaf797-f23b-af97-d2d3-efa0f5bb88d9@vangyzen.net> <95636bbf-2367-f226-f462-d5581581c18f@vangyzen.net> From: Ryan Moeller Message-ID: Date: Fri, 9 Oct 2020 19:27:18 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.12.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Oct 2020 23:27:20 -0000 On 10/9/20 6:22 PM, Alan Somers wrote: > This sounds like it might be a regression introduced by the OpenZFS merge. > Have you compared vdev_geom.c in OpenZFS vs the old version? > -Alan I don't think vdev_geom.c is involved, we're taking a wrong path in zvol_os.c because it seems the volume is created using the default volmode and later changed to volmode=dev. > On Fri, Oct 9, 2020 at 3:48 PM Eric van Gyzen wrote: > >> On 10/9/20 4:39 PM, Eric van Gyzen wrote: >>> Does this look familiar? I'm creating a zvol with volmode=dev, but some >>> geom code paths were taken. If this looks new, I'll provide more >> details. >> >> primarycache=none also seems to be a factor. I can easily repro with: >> >> zfs create -s -V 10G -o primarycache=none -o volmode=dev .../testvol I don't think primarycache is a factor, I can easily repro with primarycache left at the default. The volmode property is being set asynchronously and losing the race with the initial creation of the minors. When volmode is changed the minor is supposed to be destroyed and then recreated in the correct mode, but that does not seem to be working correctly. Setting vfs.zfs.debug=1 you can see in dmesg the zvol is created once in volmode=geom and then a second attempt to create the zvol fails, because zvol_free did not occur. zvol_create_minor_impl:1250[1]: Creating ZVOL p0/testvol... name=p0/testvol error=0 volmode=1 zvol_create_minor_impl:1372[1]: ZVOL p0/testvol created. zvol_create_minor_impl:1250[1]: Creating ZVOL p0/testvol... So something is preventing zv_free from being called by zvol_remove_minors_impl. -Ryan >> >>> 13.0-CURRENT r366500+84ccaf49083c-c272054 GENERIC >>> >>> #8 >>> #9 zvol_geom_bio_getattr (bp=0xfffff80376132900) >>> at /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zvol_os.c:545 >>> #10 zvol_geom_bio_start (bp=0xfffff80376132900) >>> at /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zvol_os.c:519 >>> #11 0xffffffff80b1c684 in g_io_schedule_down (tp=) >>> at /usr/src/sys/geom/geom_io.c:848 >>> #12 0xffffffff80b1cfcc in g_down_procbody (arg=) >>> at /usr/src/sys/geom/geom_kern.c:111 >>> >>> (kgdb) f 9 >>> #9 zvol_geom_bio_getattr (bp=0xfffff80376132900) >>> at /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zvol_os.c:545 >>> 545 spa_t *spa = dmu_objset_spa(zv->zv_objset); >>> >>> (kgdb) l >>> 540 zvol_state_t *zv; >>> 541 >>> 542 zv = bp->bio_to->private; >>> 543 ASSERT(zv != NULL); >>> 544 >>> 545 spa_t *spa = dmu_objset_spa(zv->zv_objset); >>> 546 uint64_t refd, avail, usedobjs, availobjs; >>> 547 >>> 548 if (g_handleattr_int(bp, "GEOM::candelete", 1)) >>> 549 return (0); >>> >>> (kgdb) p zv >>> $1 = (zvol_state_t *) 0x0 >>> >>> (kgdb) p *bp >>> $3 = { >>> bio_cmd = 4, >>> bio_flags = 0, >>> bio_cflags = 0, >>> bio_pflags = 0, >>> bio_dev = 0x0, >>> bio_disk = 0x0, >>> bio_offset = 0, >>> bio_bcount = 0, >>> bio_data = 0xfffff801fa687c00 "", >>> bio_ma = 0x0, >>> bio_ma_offset = 0, >>> bio_ma_n = 0, >>> bio_error = 0, >>> bio_resid = 0, >>> bio_done = 0x0, >>> bio_driver1 = 0x0, >>> bio_driver2 = 0x0, >>> bio_caller1 = 0x0, >>> bio_caller2 = 0x0, >>> bio_queue = { >>> tqe_next = 0xffffffffffffffff, >>> tqe_prev = 0xffffffffffffffff >>> }, >>> bio_attribute = 0xffffffff81223c03 "GEOM::physpath", >>> bio_zone = { >>> zone_cmd = 0 '\000', >>> zone_params = { >>> disk_params = { >>> zone_mode = 0, >>> flags = 0, >>> optimal_seq_zones = 0, >>> optimal_nonseq_zones = 0, >>> max_seq_zones = 0 >>> }, >>> rwp = { >>> id = 0, >>> flags = 0 '\000' >>> }, >>> report = { >>> starting_id = 0, >>> rep_options = 0 '\000', >>> header = { >>> same = 0 '\000', >>> maximum_lba = 0, >>> reserved = '\000' >>> }, >>> entries_allocated = 0, >>> entries_filled = 0, >>> entries_available = 0, >>> entries = 0x0 >>> } >>> } >>> }, >>> bio_from = 0xfffff80006b92880, >>> bio_to = 0xfffff80006972500, >>> bio_length = 1024, >>> bio_completed = 0, >>> bio_children = 0, >>> bio_inbed = 0, >>> bio_parent = 0x0, >>> bio_t0 = { >>> sec = 50, >>> frac = 10248368299661698441 >>> }, >>> bio_task = 0x0, >>> bio_task_arg = 0x0, >>> bio_spare1 = 0x0, >>> bio_spare2 = 0x0, >>> bio_track_bp = 0x0, >>> bio_pblkno = 0 >>> } >>> >>> (kgdb) p *bp->bio_to >>> $4 = { >>> name = 0xfffff80006972598 "zvol/disco_fast/vm/onefs1-1/disk7", >>> provider = { >>> le_next = 0x0, >>> le_prev = 0xfffff80006972428 >>> }, >>> geom = 0xfffff80006972400, >>> consumers = { >>> lh_first = 0xfffff80006b92880 >>> }, >>> acr = 1, >>> acw = 0, >>> ace = 0, >>> error = 0, >>> orphan = { >>> tqe_next = 0x0, >>> tqe_prev = 0x0 >>> }, >>> mediasize = 5368709120, >>> sectorsize = 512, >>> stripesize = 8192, >>> stripeoffset = 0, >>> stat = 0xfffff80006d3d120, >>> spare1 = 0, >>> spare2 = 0, >>> flags = 48, >>> aliases = { >>> lh_first = 0x0 >>> }, >>> private = 0x0, >>> index = 0 >>> } >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"