From owner-freebsd-arm@freebsd.org Sat Aug 24 10:15:40 2019 Return-Path: Delivered-To: freebsd-arm@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B15EDDBFAA for ; Sat, 24 Aug 2019 10:15:40 +0000 (UTC) (envelope-from zhangchaowang@gmail.com) Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46FvKl5VY6z4ptg for ; Sat, 24 Aug 2019 10:15:39 +0000 (UTC) (envelope-from zhangchaowang@gmail.com) Received: by mail-qt1-x841.google.com with SMTP id b11so13578826qtp.10 for ; Sat, 24 Aug 2019 03:15:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=CilPi2fI/6NkjbvNSEbnrKCJnThpTqF2qdHzE5gf3JM=; b=IyeSLetPaDRfy2+hDDhCyNzKrAfwO6e8vJ9qIeCL5C9+TV/FJtFJ/WUs3+4OiGrerT limMVe/5SganQiFouNdzZSZi91IBkY12rczkBPzGr4FznNCGbzdXlMeQwFEySZ7SSZKN buUyl0ciKnb8zxBgyBri78E/DLF1pXJXMSEogRpZI87uQWiKTygzFqd6rj6BsTUclUrV voO2rdX/qr8yb1ZsbIGUkGi96inqoJG5EfDS7ho8w5SCxE/W9JyUYEhmKa/3TzGTorsF +zdsEOEgogIWka8sUI+SzEtEileWRni0Bl4S2zkz+30hy8B3C9LbvwIx0risZLWnJuNJ jURw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=CilPi2fI/6NkjbvNSEbnrKCJnThpTqF2qdHzE5gf3JM=; b=NU1pWNJFnz3GYuJwjAfOqr/5M3PoZiRzAfFVoMlAQNY7X/P3XnTl860K9wig3cu1il 9DDmcvsgO9u72LxVQ5tpNc5+f3O8CZHGf7A3QuvpDyFDdNNCooljve6fO9dPSJLaRCLy 6oQ7wGSM+DPm5XRE8NvsJ2xcaIVSB6C/EX1m3HWsImiH09YZnl/LuS+GtwsIfkMCjVCM lB2gi+BgomfSN+CZGZ64f5UjzfEUwNAe8TMJ6cStFpXnHUXJa+N7Ao+hBi9viKQNH9iH bAShIOZwDazRBGg73LiTA+kJAGaWNKrGD31CUH/8KPZ5kgLqDg2RFF4uRFDq/8IHz1DD IBKg== X-Gm-Message-State: APjAAAW2STp+hLvfe+reYvG0NWdKoAzhSZE7H4SGj0mc3uMpGujoR7DE og4v874zyArMIITP66ovobBHtDsunvc= X-Google-Smtp-Source: APXvYqzmC/w6IVsLJL22DZjYuAKZim1kZW5YCrPKqhjSE4eCBPTtIVp+pZ/CIpJBrbazmrNbsGSEVw== X-Received: by 2002:aed:3b30:: with SMTP id p45mr8664017qte.84.1566641738219; Sat, 24 Aug 2019 03:15:38 -0700 (PDT) Received: from [192.168.2.15] (cpe-76-182-13-165.nc.res.rr.com. [76.182.13.165]) by smtp.gmail.com with ESMTPSA id x28sm3120308qtk.8.2019.08.24.03.15.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 24 Aug 2019 03:15:37 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: How to change rootfs from official RPI3 image From: Ricky Zhang X-Priority: 3 (Normal) In-Reply-To: <61580006.14.1566389702495@localhost> Date: Sat, 24 Aug 2019 06:15:35 -0400 Cc: freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <0FC8815E-58D7-4196-BF7E-0D6B127B314D@gmail.com> <1668312327.7.1566381369480@localhost> <61580006.14.1566389702495@localhost> To: Ronald Klop X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46FvKl5VY6z4ptg X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=IyeSLetP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of zhangchaowang@gmail.com designates 2607:f8b0:4864:20::841 as permitted sender) smtp.mailfrom=zhangchaowang@gmail.com X-Spamd-Result: default: False [-3.48 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (2.59), ipnet: 2607:f8b0::/32(-2.87), asn: 15169(-2.34), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[1.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Aug 2019 10:15:40 -0000 Hi Ronald, After migrating to UFS, I read from BSD Journal May/2019 = (https://www.freebsdfoundation.org/past-issues/freebsd-for-makers/). = RPI3 can boot from ZFS rootfs. So I tried it on my own (see the step details below in method 2): = https://github.com/rickyzhang82/FreeBSDWiki/wiki/1.-Migrate-SD-card-rootfs= -to-SSD-for-RPI3#method-2-sd-card-kernel-boot--ssd-zfs-rootfs Unfortunately, it hangs when try to mount root from dataset tank/rootfs.=20= I read between lines that the the author in the BSD Journal only mount = /var and /usr from ZFS dataset. The rest of rootfs still comes from SD = card ufs partition. My way moved the whole rootfs and home to tank/rootfs and tank/home = dataset. When it boots, I saw my USB SSD LED is flashing periodically. = The screen prints =E2=80=9CTry to mount root from tank/roofs[]...". It = seems to me that the kernel struggled to find the zpool from SSD. I saw = that zfs module gets loaded properly from the screen. I have added two = module in /boot/loader.conf as Internet suggest: ... opensolaris_load=3D"YES" zfs_load=3D"YES=E2=80=9D ... If I unplug USB SSD without any settings change, hit Ctrl-C. I can = override mount root from command line at run time with = "ufs:ufs/rootfs=E2=80=9D rathe than =E2=80=9Czfs:tank/rootfs". Then I = can boot from SD card as before. Once I login, I can run zfs list and = show both tank/rootfs and tank/home data set. So that confirmed my ZFS = pool can be found. But somehow the boot process messed up somewhere. Do you have any suggestion? thanks Ricky > On Aug 21, 2019, at 8:15 AM, Ronald Klop wrote: >=20 > Thanks. I will not take the credits for the idea though. :-) > The FreeBSD wiki has some more examples of similar setups: > https://wiki.freebsd.org/ZFSOnRoot describes how to do /boot on UFS = and the rest on ZFS, which is similar to /boot on SD-card and the rest = on SSD. > https://wiki.freebsd.org/RootOnZFS/UFSBoot >=20 > Nice that you made a write up also. More exposure for FreeBSD is = always a good thing. :-) >=20 > Good luck and keep hacking. >=20 > Ronald. > Van: Ricky Zhang > Datum: woensdag, 21 augustus 2019 13:16 > Aan: Ronald Klop > CC: freebsd-arm@freebsd.org > Onderwerp: Re: How to change rootfs from official RPI3 image >=20 > Your idea is brilliant. It solved the dilemma.=20 > =20 > Ricky@router ~ $ sudo cat /etc/fstab > # Custom /etc/fstab for FreeBSD embedded images > /dev/ufs/rootfs /bootdir ufs rw 1 1 > /dev/label/gpt/ssdrootfs / ufs rw 1 1 > /dev/msdosfs/MSDOSBOOT /boot/msdos msdosfs rw,noatime 0 0 > tmpfs /tmp tmpfs rw,mode=3D1777,size=3D50m 0 0 > =20 > Ricky@router ~ $ mount > /dev/label/gpt/ssdrootfs on / (ufs, local, soft-updates) > devfs on /dev (devfs, local, multilabel) > /dev/ufs/rootfs on /bootdir (ufs, local, soft-updates) > /dev/msdosfs/MSDOSBOOT on /bootdir/boot/msdos (msdosfs, local, = noatime) > tmpfs on /tmp (tmpfs, local) > =20 > I wrote the whole thing down in my wiki: = https://github.com/rickyzhang82/FreeBSDWiki/wiki/1.-Migrate-SD-card-rootfs= -to-SSD-for-RPI3 > =20 > Thanks > =20 > Ricky > =20 > On Aug 21, 2019, at 5:56 AM, Ronald Klop wrote: > =20 > Sorry, I did only reply to the mailinglist. I will use reply-all now. >=20 > You are right about uboot finding /boot/kernel and /boot/loader.conf. >=20 > What you need to do is mount the SD-card on /bootdir; in fstab: > /dev/yoursdcard /bootdir ufs rw,noatime 1 2 >=20 > And a symlink from /boot -> /bootdir/boot on your SSD. >=20 > Then installworld/installkernel will do the proper thing. > So the complete /boot stays on the SD-card. >=20 > Regards, >=20 > Ronald. > Van: Ricky Zhang > Datum: woensdag, 21 augustus 2019 04:58 > Aan: freebsd-arm@freebsd.org > Onderwerp: Re: How to change rootfs from official RPI3 image >=20 > > BTW: Yes, u-boot is opensource: /usr/ports/sysutils/u-boot-rpi3 > > There are sysutils/u-boot-* ports for different system. With = sysutils/u-boot-master as the main part of it. > > > > https://www.freshports.org/sysutils/u-boot-rpi3 = > > http://www.denx.de/wiki/U-Boot > > > > Regards, > > Ronald. >=20 >=20 > Hi Ronald, >=20 > Sorry, if I messed up the mailing list thread. I didn=E2=80=99t = receive your email directly. Instead, I got your reply from daily = digest. I have to copy subject and quote your reply manually in my email = client. I have no idea how to fix it after reading all FAQ = (https://www.freebsd.org/doc/en_US.ISO8859-1/articles/mailing-list-faq/art= icle.html#etiquette = ). Any other mailing list I subscribed didn=E2=80=99t = work this way... >=20 > In any case, the magic works. I did rsync: >=20 > rsync -aAXvr --progress --delete /* /mnt/USB \ > --exclude=3D'/boot/msdos/*' \ > --exclude=3D'/dev/*' \ > --exclude=3D'/proc/*' \ > --exclude=3D'/net/*' \ > --exclude=3D'/tmp/*' \ > --exclude=3D'/mnt/*' \ > --exclude=3D'/media/*' >=20 > I confirmed that it mount ssd as rootfs: >=20 > Ricky@router ~ $ df -h > Filesystem Size Used Avail Capacity Mounted on > /dev/label/gpt/ssdrootfs 407G 5.5G 369G 1% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/msdosfs/MSDOSBOOT 50M 13M 37M 26% /boot/msdos > tmpfs 50M 4.0K 50M 0% /tmp >=20 > As you said, kernel still comes from SD card. I don=E2=80=99t fully = understand the FreeBSD boot process. Neither am I familiar with UEFI. >=20 > - I guess /boot/msdos/uboot.bin finds the SD card roofs system. Load = the kernel from /boot/kernel in SD card and scan /boot/loader.conf to = find the rootfs. Please correct me if I=E2=80=99m wrong. > - Should I remove /boot folder from SSD to avoid confusion? > - Are there any guide how to compile and deploy kernel? >=20 >=20 >=20 >=20 > _______________________________________________ > freebsd-arm@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org"