From owner-freebsd-questions@freebsd.org Mon Mar 15 15:35:17 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D489457E7D9 for ; Mon, 15 Mar 2021 15:35:17 +0000 (UTC) (envelope-from gobble.wa@gmail.com) Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DzgTx0H4Kz3FXf for ; Mon, 15 Mar 2021 15:35:16 +0000 (UTC) (envelope-from gobble.wa@gmail.com) Received: by mail-ed1-x535.google.com with SMTP id o19so17876997edc.3 for ; Mon, 15 Mar 2021 08:35:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=ODxskmiEnCxiKQCtwA5uS69qK33l6WbsyrsQ6QgPmAc=; b=R+5ldlyRiZpWXhpatUhs/kRZY1hq4KWqLR9QNF0yrIPVdc1hJhgshCb9rwd9tbvQ6y Get+MNk+VS216k7Bhv8QNCP1VvBBJvbZ8Mb7JCafvk3dZZ/ZKfsQKarlTQXlDDMpQj5B MLAz++Yk+BAWPPA01yKG4Xuq4UQ91k0SlwyRIn+WiUvG1zlE00G7aRxGH0C1hzLEBE5p p8PxGwpyuD9D1/49r3kCHPOBoUEv5N/2jyKedvgMdQJyRAnz2YpsT7wTAKnpwu3xGWNh VlDhIWHgbWQBLr03gbAxy5q1N/lu2NOfhlI9+v0jSOouLMAZpPv84HLKLctAOwXklsxA edBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=ODxskmiEnCxiKQCtwA5uS69qK33l6WbsyrsQ6QgPmAc=; b=to0VFHMIxt+OkiA7IRpQGd0r2xCT0AhI1+243A1Fi77iZb53F8ynEHgspyFHDniKlJ RPH5BgTLfxZx6GG8MfjGvq4NzFuuTU5U/zRVukyRwnp2KIlS3DrZSi5oOoXi/IUGNpCF YHnj8lEAxLbOnV9grGBHiN6IZOG353rmEaUVaDRqmRXQ8ws311Ti3AVw1fQe02VpONJT 93YIGgDrLiBQtpvDEtVq5qhYDikef8ejCTovt6q0mjm73Bb+1BdV0YwojOlvPxvd+6Cn gUffP4X/l3Pz5CVyrAg6eV++6Pll47c/E+BGWU9C6huldKlXJj904w4qzT9AOpu49LKZ qf+w== X-Gm-Message-State: AOAM533lrU5U7iX0eRJuz8OvaBpDHqsphcT/3N0vpeAsFv2B5Jo2km+Y Ce4eBU07TIESOpcArMJRBlqJCXHaBrO6AwZ47I3SInP87P4= X-Google-Smtp-Source: ABdhPJx9+Mb4Hq0E8xpTJ7PrqWFzenS+FSFw9WjJrzomieLc0AUGOnc1788PQrsY7dqpFbHb68Delc6MfImqsisxuE4= X-Received: by 2002:aa7:dd97:: with SMTP id g23mr30317066edv.154.1615822515256; Mon, 15 Mar 2021 08:35:15 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Waitman Gobble Date: Mon, 15 Mar 2021 11:35:03 -0400 Message-ID: Subject: Re: Disappearing files in FreeBSD 13.0-R2 To: FreeBSD Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4DzgTx0H4Kz3FXf X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=R+5ldlyR; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gobblewa@gmail.com designates 2a00:1450:4864:20::535 as permitted sender) smtp.mailfrom=gobblewa@gmail.com X-Spamd-Result: default: False [-2.30 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::535:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::535:from:127.0.2.255]; NEURAL_SPAM_SHORT(0.70)[0.704]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::535:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-questions] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2021 15:35:17 -0000 On Mon, Mar 15, 2021 at 2:44 AM David Christensen wrote: > > On 3/14/21 6:22 PM, Waitman Gobble wrote: > > On Sun, Mar 14, 2021 at 9:20 PM Waitman Gobble wrote: > >> > >> On Sun, Mar 14, 2021 at 8:00 PM David Christensen > >> wrote: > >>> > >>> On 3/14/21 4:03 PM, Waitman Gobble wrote: > >>>> I did a fresh install using ZFS with encryption. I copied the files on a > >>>> second drive (UFS) to /usr/home/backup (ZFS). I reformatted the second > >>>> drive ZFS and created a new pool "home" for that drive. It decided to mount > >>>> the drive as /home. AFAIK i never told the system to do that. But /home and > >>>> /usr/home are different, there is no link. > >>>> > >>>> I can only see /usr/home/backup if i boot into single user mode. If i mount > >>>> read write or boot normally then /usr/home is empty. > >>>> > >>>> I copied the files to a usb drive. > >>>> > >>>> How do i delete the backup? Its taking up 100 gb. I can see them read only > >>>> and copy to usb drive, but as soon as i mount read write they disappear. I > >>>> did not import the home pool, it does not show up in the status command. > >>> > >>> > >>> Please run the following commands and post your console session: > >>> > >>> # gpart show > >>> > >>> # zpool list > >>> > >>> # zfs list -d 1 > >>> > >>> # zfs list -r -t all home > >>> > >>> > >>> David > >> > >> Thanks for the reply. > >> > >> Ok, Here's the info from multi-user mode. > >> > >> In single user mode the 'home' pool (second drive) doesn't show up in > >> status or list or anything, which is fine. > >> When I boot into single user mode / read only then /usr/home has 100gb > >> of backup. If I reboot to multi-user or mount read-write in single > >> user then the files in /usr/home disappear. (even if the second drive > >> isn't mounted). The 80 GB avail should be like 180GB I think, it's a > >> 250GB drive. The usage stats don't change if the files in /usr/home > >> appear or not. > >> > >> The home directory for the second drive is /home which isn't a link to > >> /usr/home or vice-versa AFAIK. in multi-user mode /home has my files, > >> /usr/home is empty. in single-user read only , /home is empty and > >> /usr/home has the backup files. > >> > >> root@ayo:/home # gpart show > >> => 40 488397088 ada0 GPT (233G) > >> 40 532480 1 efi (260M) > >> 532520 1024 2 freebsd-boot (512K) > >> 533544 984 - free - (492K) > >> 534528 4194304 3 freebsd-swap (2.0G) > >> 4728832 483667968 4 freebsd-zfs (231G) > >> 488396800 328 - free - (164K) > >> > >> => 40 976773088 ada1 GPT (466G) > >> 40 976773088 1 freebsd-zfs (466G) > >> > >> => 40 976773088 da0 GPT (466G) > >> 40 2008 - free - (1.0M) > >> 2048 976771072 1 freebsd-ufs (466G) > >> 976773120 8 - free - (4.0K) > >> > >> => 40 976773088 da1 GPT (466G) > >> 40 2008 - free - (1.0M) > >> 2048 976771072 1 freebsd-ufs (466G) > >> 976773120 8 - free - (4.0K) > >> > >> root@ayo:/home # zpool list > >> NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > >> HEALTH ALTROOT > >> home 464G 294G 170G - - 0% 63% 1.00x ONLINE - > >> zroot 230G 142G 87.6G - - 0% 61% 1.00x ONLINE - > >> root@ayo:/home # zfs list -d 1 > >> NAME USED AVAIL REFER MOUNTPOINT > >> home 294G 156G 294G /home > >> zroot 142G 80.4G 96K /zroot > >> zroot/ROOT 140G 80.4G 96K none > >> zroot/tmp 184K 80.4G 184K /tmp > >> zroot/usr 1.38G 80.4G 96K /usr > >> zroot/var 877M 80.4G 96K /var > >> root@ayo:/home # zfs list -r -t all home > >> NAME USED AVAIL REFER MOUNTPOINT > >> home 294G 156G 294G /home > >> root@ayo:/home # ls /usr/home/ > >> [nada] > >> root@ayo:/home # > >> > >> > > > > oops, missed a couple. sorry. > > > > root@ayo:/home # zfs list -r -t all home > > NAME USED AVAIL REFER MOUNTPOINT > > home 294G 156G 294G /home > > > > root@ayo:/home # zfs list -d 1 > > NAME USED AVAIL REFER MOUNTPOINT > > home 294G 156G 294G /home > > zroot 142G 80.4G 96K /zroot > > zroot/ROOT 140G 80.4G 96K none > > zroot/tmp 184K 80.4G 184K /tmp > > zroot/usr 1.38G 80.4G 96K /usr > > zroot/var 877M 80.4G 96K /var > > > It looks like 'ada0' is a 250 GB drive with GPT and your system > partitions. ada0p4 should be your root partition with the ZFS pool > 'zroot', containing the root filesystem. 'zroot/usr/home' should be a > ZFS filesystem mounted at /usr/home. These commands will confirm: > > # zfs list -d 1 zroot/usr > > # zfs list -r -t all zroot/usr/home > > > '/home' should be a symbolic link to '/usr/home'. These commands will > confirm: > > # ls -l /home > > # ls -l /usr > > > It looks like 'ada1' is a 500 GB drive with the ZFS pool 'home'. ZFS > mounts this at '/home' by default. This mount overlays the root > filesystem symbolic link '/home', but I do not understand why > '/usr/home/backup' disappears. > > > I would do the following as root in single-user mode: > > 1. Record your console session with script(1). Exact details will be > useful later; sooner if something goes wrong. > > 2. Take a recursive snapshot of the 'zroot' and 'home' ZFS filesystems. > Pick a meaningful SNAPNAME (I do date/ time strings): > > # zfs snapshot -r zroot@SNAPNAME home@SNAPNAME > > 3. Unmount the ZFS filesystem 'zroot/usr/home', make it read-only, > change its mountpoint, and mount it:: > > # zfs unmount zroot/usr/home > > # zfs set readonly=on zroot/usr/home > > # zfs set mountpoint=/usr/oldhome zroot/usr/home > > # zfs mount zroot/usr/home > > /usr/oldhome and /usr/oldhome/backup should now be visible. > > (If you previously created a ZFS filesystem > 'zroot/usr/home/backup', repeat the first, second, and fourth steps > above; adjusting the filesystem name. The 'mountpoint' property should > be inherited.) > > 4. Set the mountpoint of the ZFS pool 'home' and mount it: > > # zfs set mountpoint=/usr/home home > > # zfs mount home > > /usr/home should now be visible. > > 5. It is recommended practice not to put files and directories into the > base filesystem of a ZFS pool (e.g. '/usr/home') -- it is better to > create ZFS filesystems at the base level of a pool and put files, > directories, and/or additional ZFS filesystems into those. Assuming > '/usr/oldhome/backup' represents your old home directory, create a ZFS > filesystem for your new home directory: > > # zfs create home/username > > Do the same when adding more accounts in the future. > > 6. Assuming '/usr/oldhome/backup' represents one user account, copy its > contents to '/usr/home/username'. > > 7. Reboot and check everything. > > 8. Wait a while (hours, days, weeks). When you are certain everything > is okay, destroy the old home filesystem: > > # zfs destroy -r zroot/usr/home > > This should reclaim space in the 'zroot' pool and filesystem. > > > David > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" Thanks for everyone's help. It seems my problem was that /home was an actual directory. I don't recall creating that, normally I would not do that. This was a fresh install but I rsynched some files from the old drive to the new one, like most of /etc, /root, /usr/local, /var. Anyhow, when I deleted /home and created a symlink, the backup directory magically appeared in /usr/home. So I deleted that and reclaimed 100G of drive space. Then I mounted the home pool and it mounted to /usr/home instead of /home. Everything seems to work. So in summary, for some reason there was a /home directory. and if you create a pool named "home" it does magic stuff like it knows it's supposed to be home i guess. If /home is a directory then it acts crazy. Deleting /home makes it sane again. -- Waitman Gobble