From nobody Sun Apr 30 02:44:13 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q89fM49Yqz48c33 for ; Sun, 30 Apr 2023 02:44:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-21.consmr.mail.gq1.yahoo.com (sonic317-21.consmr.mail.gq1.yahoo.com [98.137.66.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q89fL5L1Bz4FKw for ; Sun, 30 Apr 2023 02:44:26 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=s0flWfP4; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682822665; bh=Dg1zRP6K+BkPLSZIDNaWBC+aEEpxAZJKuTftAcqwykQ=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=s0flWfP4B9WE88d56JpOwM21gfvfnRN+Jnkve3V3JuQam+1cFDlHx1uUxTnbrhcm+9AJb3oVkzHN9aIO3cwYZlrx2qx6OM50MhEjCccLLGHuTPorsVB0tkzjyG9+nq+hoNp9bHrJBuK6RrByBdQwJNEtzmDUtYj3yPL8gtzeGWPqfNRxXeuGTM331OYbglut8Gc1ZRHbG+UBLyc9PeG3HSG4oAwpfBcrX/5DnqG7xTEs3B6J4hbbpQgS3hLB2qc9PyO9doeCwIyOU7afSypHMlPXuyAw6QcKy1Yj10yXrsOExL7wIyxGWoLOpQwg5Q42M9HgDUkMK8wSpy0hGwiVIQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682822665; bh=nd5Ii+dBXUB/ugS8bU8qUae5jVgyBEn1AL++8gTUnta=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=cWxeM5J23bZN7CSYnlcGkq4wIFvPdRlFeABqJvyC8NWp5iYjYfoQAZHmYNbvUWjeEJ0IObn2Ig7fRPcpx+n6DfyujnQPI7PH9rGC1SkWaXfNScXR9c3xP2Od7XkFMt4eso6CS1bPih8N6+wFqNyUcscPWxnfcPhabA/C9daX4VviY2cAhZYcRog8q4W/XZ4uUCKNUwLvyoQF4D41opO4ab6nOaP0v6iz5j1CTs+OnZz1z2mRAneQGFM1btvs8hDhDVQFZBUXSHrPIaCIzcLrKoYYMlkq1mG268a4lOkl5zFvH7SsHaC+kGVupN7nKydnmMI3j3l1p++6hu8Dd0JoFA== X-YMail-OSG: WVuzG08VM1md.qqGM.tX2xSZ0snDT4G9Xsp6Lxrk_hed5Ud7wTtkNBm2wVIVmVx 0TZqzngtAmmloe1JSFVvZokTb53JBd7xCqJ539XlSF7ifXayryJvvC2IaC3T9apRiUEj15piK3U4 JhrbRFm_m0OBK8.Lg1wNTpycC_IYbbTNRC4X8KspLhmGKp4.hXvCTyrH02cOftrrH67VqeNfXo2V Xwi.m8uSQntmVepI3O857EroCqXXSeMtsQhDxzK7koa8lfICYYtJUpjSNY2F8sKJIph624RQGKv3 nQPM5bCJqn8dWQfWNtF40QX9CsuwGe5XELhGEfhGwfTraGz.LM.NsaDNd5CQ0l4UI1HFIfwIzhou vuKt78CgvkAnlYMYcg2n08Zu7orj.0b_6iaED1vANu97Nn6m7fkf6rSI9WpfA3ktRbQ_7oneEBeA N912iAmMANH7n2DYRHWCfaefqUPynrTCjiLTJTdXszz327oLiOJ1AckYzav1daAhidljjWJi18Xa FIp7gIjb.261mcFgEy7lpPimtlf1C8rmrFwjeFul_SWsV4Ds5dS7dcRHt6EykJ7KvgNhtf0qQIyE nLh6Vw2Xf782KX8_kcTgw5I35lXQECqIKDcLAFQohEZszHu0nH3FxxORhsF06UOyaZDxwx_Vu3tc 6iDgnDw.SLHih7sxoarYyywQfPaLto_TwHmzGSmtSFyruSzIFj4zRNDwcLGkatM2cwG_k.QU0gEQ U3KtrqGSaV6kC4tHgKnq67sKQkykuDqtgYo07x9mEWS6ul.6ExuR8GGh4yesM0JZseEOlPUi2idU df0K9Ynca9geYTu2woPXPox21nE2q.mjfT3FUpcQLcAFF5szJo0BmnBgZRYR5dFOfhWJHqQr_LNc _KO_eS7.tpShej0bDpScbaqYDqPITdgc3fQJBb9VcAZYP.UrtW36a8ug7F0OFcrbDggiABL1HUPN kGvzgy3A0N_0EJJI14HW04ro0yZEcUqnkK.KH.GyueKfQkmjaL.fcBEgN6Vva_dl0tMCzAXKG7UP xPMue.B.QFjSOUFqnruQmcNLcbFw14Z23yWlORXraXPyxPlpUd8ofxLn1QK7UwF5ZaP.Ul9xq21u hEtxNrUWqv8r32nYqEMP_d1GIgxVoPXp1g7z97bzOZrB9eG0q0xUz_prkwRkBueG22unZJMhmXJ4 qfjsivJlERgJ2xNg8i4MfEhMWLU4RsHnS2jiNafFnkmyizL7oSj2PaP6nhB6ES1NPwKJPWsnx897 GIvaVF4OPJkM.4qgCx6TbM97QAXMVeyTIp.hK7e3XcMIqghp8XT0TWEJ_9ykC8XFjdK8nEvSkhk4 Qzcb0lgCcVNUBBtqMCv2sdSFWi5I96WXh7PrGOgUrh8Adbi7zxN7GxxB_R_gGq2oWSkEMbeMRt.z McN.U6Zn80cn3su0ijyXJnGbFKr47Kt5hwI1JCc7udBCNV9HIiQMhqMyQA.S4a9O995dhrlLVknz oMM7J9nCHGbUJ.LGEc9DKpcsf8SUxQSG_KOC2bhOqFhje8idH.NXoGyiNpbLegOtp5ozoAi59uWf Kz1tzjFEnmkqcBdOxQ6.laaYolcQJM5MSJA4O5MPCuXk8nrMRTU_XXEeZYpfHFiYrTHkRq4i_vAQ _HPgJ5V8ze6p82PRMnXD5V2HvE7_rfqazu3Ry813gXcLRVCQmoMNbpVU68.6S.XzqpZDqyUOI6Wa YhQ4IJsKG.5hmOxgEpxKJ2hbXzOnVD2EQ7jG2nz1.h5U_sauRKndCaUvb7WDkeXOqkORZmsTpVec 1eU1R1ri75g1tsUvgSVd0gYTSFGW2YkHwRnQEKjqDJK5Ka_.jwfOgkGT9g9qR7hLHQvx63LkUeqC kFmLJUs7I8a1KE9HeFM90gVQVdHkrVQc9tZh3hibEkXjVZAyAdikaHcj5siC8.coSnH.dSaQtbtk ik9KpcaQ.apO1x_huFuZ176RJqqCS78ZTvXHGglsl2uSQElnbAuQTrO72KUgkUfICGID6lEEiG9B vW1RVp9H2fdCv2uYcKme46ryZzuFtQcVCkyT3jRXKy97ZnOJG.7h5RwelCr8G50TPjezC5Jmkvav _Q_AUghz8r_BH5qLbkF_t6tY26EUnFg77ec.JeqY572lbziWPghnRfs9dIkdCM5.P0RU1I4h3PHK wEtnGwiCIfqXfhZH7XcBVnQZ5wK_uUCi_ksvVL43vhUstIu_irZfQNRoz7bRibP4Q9wVgeS9rTFT oYI7xdcCvHP2XJSC2p94IDFfuCMzefmT5HS9H7yHPeASCTKIjo6QgKRgXOdwrVvWuy7m6oJCqzLF m X-Sonic-MF: X-Sonic-ID: 600319dc-fa0d-4f7c-9d89-4504435516d5 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Sun, 30 Apr 2023 02:44:25 +0000 Received: by hermes--production-gq1-546798879c-g88f5 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 1023595f13d6ecaf5c33461fb2a882a1; Sun, 30 Apr 2023 02:44:24 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes? Message-Id: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> Date: Sat, 29 Apr 2023 19:44:13 -0700 To: FreeBSD Hackers , freebsd-arm X-Mailer: Apple Mail (2.3731.400.51.1.1) References: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D.ref@yahoo.com> X-Spamd-Result: default: False [-2.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.996]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.147:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCVD_TLS_LAST(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.147:from] X-Rspamd-Queue-Id: 4Q89fL5L1Bz4FKw X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N This is based on: main-n262658-b347c2284603-dirty, b347c2284603 being from late Apr 28, 2023 UTC. (The "-dirty" is from some historical patches that I use.) The build is a non-debug build (but with symbols not stripped). World or Kernel had been built using: -mcpu=3Dcortex-a78C+flagm+nofp16fml just for testing purposes. (Worked nicely for -j8 buildworld buildkernel testing for the 4 cortex-a78c's plus 4 cortex-x1c's present.) Monitoring poudriere bulk related activity via top and gstat -spod I see a lot of the odd result of one process doing something like: CPU4 4 1:39 99.12% /usr/local/sbin/pkg-static create -f tzst -r = /wrkdirs/usr/ports/devel/cmake-core/work/stage while other processes sit in the likes of: tx->tx zcq->z zcw->z zilog- select wait But sometimes there is no CPU bound process and the top CPU process is the likes of: 1.24% [usb{usbus0}] "gstat -spod" basically shows da0 dedicated to write activity most of the time. After: sysctl kern.tty_info_kstacks=3D1 Then using ^T at various times, I see a lot of: load: 0.48 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7534.91r 11.06u = 22.66s 0% 3800k #0 0xffff0000004fd564 at mi_switch+0x104 #1 0xffff000000463f40 at _cv_wait+0x120 #2 0xffff00000153fa34 at txg_wait_open+0xf4 #3 0xffff0000014a40bc at dmu_free_long_range+0x17c #4 0xffff000001448254 at zfs_rmnode+0x64 #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 #10 0xffff0000005fc430 at vinactivef+0x180 #11 0xffff0000005fba50 at vput_final+0x200 #12 0xffff00000060c4d0 at kern_funlinkat+0x320 #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 #14 0xffff0000008f8514 at do_el0_sync+0x594 #15 0xffff0000008d4904 at handle_el0_sync+0x40 load: 0.34 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7566.69r 11.06u = 22.66s 0% 3800k #0 0xffff0000004fd564 at mi_switch+0x104 #1 0xffff000000463f40 at _cv_wait+0x120 #2 0xffff00000153fa34 at txg_wait_open+0xf4 #3 0xffff0000014a40bc at dmu_free_long_range+0x17c #4 0xffff000001448254 at zfs_rmnode+0x64 #5 0xffff0000014557c4 at zfs_freebsd_reclaim+0x34 #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c #7 0xffff0000005fd6c0 at vgonel+0x450 #8 0xffff0000005fde7c at vrecycle+0x9c #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 #10 0xffff0000005fc430 at vinactivef+0x180 #11 0xffff0000005fba50 at vput_final+0x200 #12 0xffff00000060c4d0 at kern_funlinkat+0x320 #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 #14 0xffff0000008f8514 at do_el0_sync+0x594 #15 0xffff0000008d4904 at handle_el0_sync+0x40 load: 0.44 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7693.52r 11.24u = 23.08s 0% 3800k #0 0xffff0000004fd564 at mi_switch+0x104 #1 0xffff000000463f40 at _cv_wait+0x120 #2 0xffff00000153fa34 at txg_wait_open+0xf4 #3 0xffff0000014a40bc at dmu_free_long_range+0x17c #4 0xffff000001448254 at zfs_rmnode+0x64 #5 0xffff0000014557c4 at zfs_freebsd_reclaim+0x34 #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c #7 0xffff0000005fd6c0 at vgonel+0x450 #8 0xffff0000005fde7c at vrecycle+0x9c #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 #10 0xffff0000005fc430 at vinactivef+0x180 #11 0xffff0000005fba50 at vput_final+0x200 #12 0xffff00000060c4d0 at kern_funlinkat+0x320 #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 #14 0xffff0000008f8514 at do_el0_sync+0x594 #15 0xffff0000008d4904 at handle_el0_sync+0x40 The system (Windows Dev Kit 2023) has 32 GiBytes of RAM. Example output from a top modified to show some "Max[imum]Obs[erved]" information: last pid: 17198; load averages: 0.33, 0.58, 1.06 MaxObs: 15.49, = 8.73, 5.75 = up 0+20:48:10 19:14:49 426 threads: 9 running, 394 sleeping, 1 stopped, 22 waiting, 50 = MaxObsRunning CPU: 0.0% user, 0.0% nice, 0.2% system, 0.1% interrupt, 99.7% idle Mem: 282760Ki Active, 7716Mi Inact, 23192Ki Laundry, 22444Mi Wired, = 2780Ki Buf, 848840Ki Free, 2278Mi MaxObsActive, 22444Mi MaxObsWired, = 22752Mi MaxObs(Act+Wir+Lndry) ARC: 11359Mi Total, 3375Mi MFU, 5900Mi MRU, 993Mi Anon, 93076Ki Header, = 992Mi Other 8276Mi Compressed, 19727Mi Uncompressed, 2.38:1 Ratio Swap: 120832Mi Total, 120832Mi Free, 2301Mi MaxObs(Act+Lndry+SwapUsed), = 22752Mi MaxObs(Act+Wir+Lndry+SwapUsed) The poudriere bulk has 8 builders but has ALLOW_MAKE_JOBS=3Dyes without any explicit settings for the likes of MAKE_JOBS_NUMBER . So it is a configuration that allows a high load average compared to the number of hardware threads (here: cores) in the system. I've rebooted to do a test with filemon not loaded at the time (here it was loaded from prior buildworld buildkernel activity). We will see if it still ends up with such problems. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Apr 30 07:50:02 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q8JRH4sDXz46SQ3 for ; Sun, 30 Apr 2023 07:50:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-23.consmr.mail.gq1.yahoo.com (sonic304-23.consmr.mail.gq1.yahoo.com [98.137.68.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q8JRG4rHwz3sCr for ; Sun, 30 Apr 2023 07:50:18 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=PaHrZaCX; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682841016; bh=YOoe200HacnbBfywZapJrkgkLiUxWYnsNRKevxbTQFE=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=PaHrZaCXD6R3a/v29DM8qhgEVswwDpJ1tf7Q0ZY727MpRhUxdIzRdzvU4psbPpq/MDsGTXUi+kRMtkML7s2Lb1O3A1srOdhc/L8mL4t75mhHt36L4FPWydQ2e2f1E66TW3SXAolXP2lYnoOeP4bfH2fQ6TDD65MwBEDqLZnFmmhFwvhBGYuBDHV+a/JqJrlYUyhN/netIN4eweJy9OQXEaaJzJp2GJQgsp+YZYgcfB0mRbe9GiPI5okaOxiHwPaL3I4lZBzgP71T0qc2Zwx0uosGyyKvtJToqUNGYWbEL1su94QlLfVrCEcH6M4B88s+ANwtEVLXEJCuvk17RC1aSQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682841016; bh=kZyvH3ygK1SqV9irDpKYtA84HCwKxKi8/5CQ/A21vWw=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=UrV7huuq2E2OqFhMPhxSHbKiEgOHcfY4h5RFyUiI8ruc24lxmDLbTtDJtp27KEnTTgilvdD1tR3mwa44PoHpa9D70XDGqVSOLMgeF8kbNVHtQVFjjLTcJRCY0K+S9FyTZ9x0ps/x8UDibtlnEt59GrDHVthkrBFA+zCQEY1yoljbF/mITzgDHk7C0/iAgm3vsn2LiZqiEGa/f8Sg3xeOEWc1nFUXYhdV3AKa2UGeSARRrsDnYaCYLBmF5H7b6nNjul1e6BhXs7j0nxIowbeWNPOxDbOheVe0G8Rtoqiv3PrJaLbkXivIZBlRhnp36aU6HCSMoaWxWoQhVy9wdow5Mw== X-YMail-OSG: mFkGbdQVM1lGhrCb7qxlbOZLSA6BJWpCIud8W4te6Bs1gQMNGWKNURFKPekmKzF wt4d6suy0czfJxsuj8.kjepuThLnF8z2tonJ8uyEs6js1MOIr.rKotmq8mtzB3dSaTw_ztRnSgZi hH9tU0fv1IZwu8S96JyiUZwXalMguT58FnhRMiNl4J_mZMFjqV..n1CxC5pzP5liKuVq4tcLytm9 Ks8yFGKa3NHufmvnhCyCvwxEzzbpFNotijYBvvRcdobl0CoACioV8PQfzX6gDP8.uDISoP_T8WyK gcmX3qKG3iMC7GspCK8t.tFziFgs0e4jvNrt2hBu52MKKiYGRy0qJiFrVPK_8_GcRMpVlEUWUvqV 75vAeKx.KsF.fOIosXCq7HeHvZwaM8EpyxrrD4fDakZ.JB7.x8k.H3BbxPWAhT3a8.Md57uvhEij 9b6PKiGB2J5Dp60X.qO6IkxLCekbAb.umq3ZgaqeEOonhIx_zVKn63wmaDaNLfoB4_JmvyHttS9N t743X.EjC9tza1U0KS6PfvIU7SkCpcl.DLisdL06F8hVGuacwJ.WnjM2f_Pp22PJ4qa7SGEVpEGO 0eXT0I7zRtvO0wOQYBv.6pKte7JXqOH61RrFYEGQkVlqIOIDx7KrFLHSVgmjrxzsiBqxSjNiprxS culvrfIHC5VgobxFFUQUPWNSejFGPKUTmFFcrTOGaR_Z8ZEGT0GYfpfLRr3Mkyn6pWRnvMRwpeEW C6vXKiBx9n37I8IOkEfR4R9_hPNLbZ9nu4LdoJ6CxmWN9jJkwSTtspdyQKqj65k6xGxEyN51IfOB YJd.WNf5JDsh18G1VYmmW.cmAmBZ0P8MzkCUVMpIcJRbvbxLdow6NiWAn0XwgQsHefU0BxF7goXc RgRsVYl6543a1NUqM1cPabD2m5g4M1P30D6oCrmT2dturdxy_hYBIXJmDqX8dfqENejASGrDW2oI mxZt54AXGEGxx3J.YjG46BxFGE.oUWbh57MdulcGGrF07g7ydIQH4lOuwtVDH6mIVji1xA52Te8O tRQwDZiLjR8Wy68IDm2a7F0mLIAHPk0br8ysmzGmO2ZjHQcxUaFQ_1LdkJ0euNrnzOtZ6VL70n0X ZkFf4XzQH8EGyjGCdDz1rHjm8nmp3LEoT42WJTiS0nvHXEYzFRzPK73E_Ms0pf_ssEuVz.cp7zKq sMTy03DTpljtrVR1jachMnxOzh5W3zNUCQVpnDtkxTHV4moaOBonEZJxmaCUaXxBqGjEz55hoWP7 JWvi7qJQAwP5_YA1.YAT6bliJO4TP5pgRwIUpRbhYHdzCgf1bhNmNqh13dXqEsU9UU56HvWWRF._ H204DlcetlspUjx0Zfna0fYsgqyOlM7_yvBpzQvw3P7ycdtwmpmt0d7QyEwekV_h.53.QTd6_rU2 bOkpOdr6H7QI7Bebe5MdQ_..BGScau1h21akZ8NNDJ4YDCeRheOlqL6lw14Xz06zmYxxO9Tjnt1Y zIIubZmZ47ROzx_qmJ4J_Kbwjp1B1Wlht0eXb1LmIYsoxFJwmNMel2LNoBn8mwf8z9t1c0f5oma9 p1Xz0wpSLXwcwWJTnRGREpdwO7hKLxdzIoYLzyJS2G4OjE6JH6LxL0ZVxl8aKrb2Xfhfg3Ri_4t0 t_yIhIoFUuDSkO._LU_zfcGq.L.FItrdycFBKHy79RaFKwqoonTskuWY5Xx0sObMPvjMID975Fx. uaRNRidePyHS6j7WPzF2qRBpgaPFIW96lr0Sgu8CHahsr_tXPOpw1pWYiF6n1GLDRoc6PgPmbLwf aNrULHi2StD3OPp6NfVj39a5w5S1RLHHQ7qTLjagqVX1Xp1oB5rg8ewipDciP4oJh5gAMsCITc_v crTklNHCRN.H2iA0jlFPKu3ZTj5a_SlA42FCrCv4L4OvJZ_k2uoGX65yKuaKqQnVK.Lqj68gH37p n_snsMUb9AGoitdQn6wETWTkE6Hekd.q2V9TDlkdioh8EDmQ7G4ZQkQP6yuK57cO8oDIjMU7pXpI 3tAEXSImaS_D3VZG6ot0hNKB1fdCmK3nK7SSzth..8sfIiQ2r0g8K6vt8j5jL3oCXIidsomEwv2g gGhiv3dtn5FyM.k49ocpjjX15_X7lYVrW_9G17XI4a8Lwh025pC63AzgaJw4pQ.uIeoqQQ6L9kPp X8hgpqiLgYY6_.AVSBmD6EOczkh.2GJ1U2kmyBto4KK2DFCWtqSG_tsunF3Mj.fgBaS.EjoGHxTi Xsd8fPQ1IYU7MJ1MRRjxpz7lE597NRAmmhktbNyCq1N2caVodPhHbWHq3z4rMn8WpNe4J6McAVsV Bwh4- X-Sonic-MF: X-Sonic-ID: 110a1bb2-13c1-4169-bc79-053d31d03f1a Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Sun, 30 Apr 2023 07:50:16 +0000 Received: by hermes--production-gq1-546798879c-qtf56 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 887f2c2aaf197c4439425249f4901f60; Sun, 30 Apr 2023 07:50:13 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes? Date: Sun, 30 Apr 2023 00:50:02 -0700 References: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> To: FreeBSD Hackers , freebsd-arm In-Reply-To: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> Message-Id: <02DC03AE-E082-4FB5-AA0D-396F64CC23CB@yahoo.com> X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-2.44 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.94)[-0.944]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.204:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCVD_TLS_LAST(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.204:from] X-Rspamd-Queue-Id: 4Q8JRG4rHwz3sCr X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N On Apr 29, 2023, at 19:44, Mark Millard wrote: > This is based on: main-n262658-b347c2284603-dirty, b347c2284603 > being from late Apr 28, 2023 UTC. (The "-dirty" is from some > historical patches that I use.) The build is a non-debug build > (but with symbols not stripped). World or Kernel had been built > using: >=20 > -mcpu=3Dcortex-a78C+flagm+nofp16fml >=20 > just for testing purposes. (Worked nicely for -j8 buildworld > buildkernel testing for the 4 cortex-a78c's plus 4 cortex-x1c's > present.) >=20 > Monitoring poudriere bulk related activity via top and gstat -spod > I see a lot of the odd result of one process doing something > like: >=20 > CPU4 4 1:39 99.12% /usr/local/sbin/pkg-static create -f tzst -r = /wrkdirs/usr/ports/devel/cmake-core/work/stage >=20 > while other processes sit in the likes of: >=20 > tx->tx > zcq->z > zcw->z > zilog- > select > wait >=20 > But sometimes there is no CPU bound process and the top CPU process is > the likes of: >=20 > 1.24% [usb{usbus0}] >=20 > "gstat -spod" basically shows da0 dedicated to write activity most > of the time. >=20 > After: sysctl kern.tty_info_kstacks=3D1 > Then using ^T at various times, I see a lot of: >=20 > load: 0.48 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7534.91r 11.06u = 22.66s 0% 3800k > #0 0xffff0000004fd564 at mi_switch+0x104 > #1 0xffff000000463f40 at _cv_wait+0x120 > #2 0xffff00000153fa34 at txg_wait_open+0xf4 > #3 0xffff0000014a40bc at dmu_free_long_range+0x17c > #4 0xffff000001448254 at zfs_rmnode+0x64 > #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 > #10 0xffff0000005fc430 at vinactivef+0x180 > #11 0xffff0000005fba50 at vput_final+0x200 > #12 0xffff00000060c4d0 at kern_funlinkat+0x320 > #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 > #14 0xffff0000008f8514 at do_el0_sync+0x594 > #15 0xffff0000008d4904 at handle_el0_sync+0x40 >=20 > load: 0.34 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7566.69r 11.06u = 22.66s 0% 3800k > #0 0xffff0000004fd564 at mi_switch+0x104 > #1 0xffff000000463f40 at _cv_wait+0x120 > #2 0xffff00000153fa34 at txg_wait_open+0xf4 > #3 0xffff0000014a40bc at dmu_free_long_range+0x17c > #4 0xffff000001448254 at zfs_rmnode+0x64 > #5 0xffff0000014557c4 at zfs_freebsd_reclaim+0x34 > #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c > #7 0xffff0000005fd6c0 at vgonel+0x450 > #8 0xffff0000005fde7c at vrecycle+0x9c > #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 > #10 0xffff0000005fc430 at vinactivef+0x180 > #11 0xffff0000005fba50 at vput_final+0x200 > #12 0xffff00000060c4d0 at kern_funlinkat+0x320 > #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 > #14 0xffff0000008f8514 at do_el0_sync+0x594 > #15 0xffff0000008d4904 at handle_el0_sync+0x40 >=20 > load: 0.44 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7693.52r 11.24u = 23.08s 0% 3800k > #0 0xffff0000004fd564 at mi_switch+0x104 > #1 0xffff000000463f40 at _cv_wait+0x120 > #2 0xffff00000153fa34 at txg_wait_open+0xf4 > #3 0xffff0000014a40bc at dmu_free_long_range+0x17c > #4 0xffff000001448254 at zfs_rmnode+0x64 > #5 0xffff0000014557c4 at zfs_freebsd_reclaim+0x34 > #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c > #7 0xffff0000005fd6c0 at vgonel+0x450 > #8 0xffff0000005fde7c at vrecycle+0x9c > #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 > #10 0xffff0000005fc430 at vinactivef+0x180 > #11 0xffff0000005fba50 at vput_final+0x200 > #12 0xffff00000060c4d0 at kern_funlinkat+0x320 > #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 > #14 0xffff0000008f8514 at do_el0_sync+0x594 > #15 0xffff0000008d4904 at handle_el0_sync+0x40 >=20 >=20 > The system (Windows Dev Kit 2023) has 32 GiBytes of RAM. Example > output from a top modified to show some "Max[imum]Obs[erved]" > information: >=20 > last pid: 17198; load averages: 0.33, 0.58, 1.06 MaxObs: = 15.49, 8.73, 5.75 = up 0+20:48:10 19:14:49 > 426 threads: 9 running, 394 sleeping, 1 stopped, 22 waiting, 50 = MaxObsRunning > CPU: 0.0% user, 0.0% nice, 0.2% system, 0.1% interrupt, 99.7% idle > Mem: 282760Ki Active, 7716Mi Inact, 23192Ki Laundry, 22444Mi Wired, = 2780Ki Buf, 848840Ki Free, 2278Mi MaxObsActive, 22444Mi MaxObsWired, = 22752Mi MaxObs(Act+Wir+Lndry) > ARC: 11359Mi Total, 3375Mi MFU, 5900Mi MRU, 993Mi Anon, 93076Ki = Header, 992Mi Other > 8276Mi Compressed, 19727Mi Uncompressed, 2.38:1 Ratio > Swap: 120832Mi Total, 120832Mi Free, 2301Mi = MaxObs(Act+Lndry+SwapUsed), 22752Mi MaxObs(Act+Wir+Lndry+SwapUsed) >=20 >=20 > The poudriere bulk has 8 builders but has ALLOW_MAKE_JOBS=3Dyes > without any explicit settings for the likes of MAKE_JOBS_NUMBER . > So it is a configuration that allows a high load average compared > to the number of hardware threads (here: cores) in the system. >=20 >=20 > I've rebooted to do a test with filemon not loaded at the time > (here it was loaded from prior buildworld buildkernel activity). > We will see if it still ends up with such problems. It still ends up with things waiting, but the detailed STATE valiues generally lsited are somewhat different. I also tried a chroot into a world from use of -mcpu=3Dcortex-a72 and got similar results. Suggesting that only the -mcpu=3Dcortex-a78c+flagm+nofp16fml kernel is required to see the issues. This even got some examples like: load: 1.20 cmd: sh 95560 [tx->tx_quiesce_done_cv] 2022.55r 3.21u 9.24s = 0% 3852k #0 0xffff0000004fd564 at mi_switch+0x104 #1 0xffff000000463f40 at _cv_wait+0x120 #2 0xffff000001518a34 at txg_wait_open+0xf4 #3 0xffff00000147d0bc at dmu_free_long_range+0x17c #4 0xffff000001421254 at zfs_rmnode+0x64 #5 0xffff00000142e7c4 at zfs_freebsd_reclaim+0x34 #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c #7 0xffff0000005fd6c0 at vgonel+0x450 #8 0xffff0000005fde7c at vrecycle+0x9c #9 0xffff00000142e678 at zfs_freebsd_inactive+0x48 #10 0xffff0000005fc430 at vinactivef+0x180 #11 0xffff0000005fba50 at vput_final+0x200 #12 0xffff00015d8ceab4 at null_reclaim+0x154 #13 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c #14 0xffff0000005fd6c0 at vgonel+0x450 #15 0xffff0000005fde7c at vrecycle+0x9c #16 0xffff00015d8ce8e8 at null_inactive+0x38 #17 0xffff0000005fc430 at vinactivef+0x180 (The chroot use involves null mounts.) which is sort of analogous to the filemon related backtraces I showed earlier. The common part=20 across the examples looks to be #0..#11: #0 0xffff0000004fd564 at mi_switch+0x104 #1 0xffff000000463f40 at _cv_wait+0x120 #2 0xffff00000153fa34 at txg_wait_open+0xf4 #3 0xffff0000014a40bc at dmu_free_long_range+0x17c #4 0xffff000001448254 at zfs_rmnode+0x64 #5 0xffff0000014557c4 at zfs_freebsd_reclaim+0x34 #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c #7 0xffff0000005fd6c0 at vgonel+0x450 #8 0xffff0000005fde7c at vrecycle+0x9c #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 #10 0xffff0000005fc430 at vinactivef+0x180 #11 0xffff0000005fba50 at vput_final+0x200 There were a lot more nanslp examples in all the alter testing (i.e, those avoided filemon.ko being loaded). Starting from having pkgclean -A'd the ports, the experiments got about the same number of ports built as of the end of the 1st hour. UFS vs. ZFS? Different media types? . . . So I decided to create and try a UFS test context instead of a ZFS one. But the media that was best to update was a U2 960GB Optane in a USB3 adapter, something that would perform noticeably better than my normal USB3 NVMe drives, even with USB involved. This combination maintained reasonable load averages (instead of having long periods of <1) and finish building 172 ports in the 1st hour, far more than the around 83 each time I tried the other device/ZFS combination. NO evdience of the earlier reported oddities. I should also time a from-scratch buildworld buildkernel. I'll look into setting up another U2 960GB Optane for use in the USB3 adpater, but with ZFS. That should help isolate media vs. file system contributions to the varying behaviors. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Apr 30 21:00:06 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q8dyb1Xffz48Hnl for ; Sun, 30 Apr 2023 21:00:07 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q8dyZ6ygbz45vF for ; Sun, 30 Apr 2023 21:00:06 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682888407; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=c31Byfb52nyCUkiG2jplJub68Hinz6FDqmHNIUPhxII=; b=AZ2MzPCx1GjLhRcC7kIzSyCEQriVPBE56zZO+jNUIwR5P4VraC/ifa0ssuqbgexwWgzZFS X2FavWpiQGP8sCTaW0fZXE7wFY5V4LmRL09AS1Vnytk0CsRri5NYwHfpelsDZXFhjMILZM QkWfe7qJn1BFp9Y/VQMI5/ARD6b0scOhAZYDGPQzVWAM15PHtyiWoWG07/nTCcUFCsJv44 pdmasY3Ha+o1XENzxtGpIvJO10sZBKasogv6FY5TPG0S6c3GJZ+FSvPiwTBs2ugCDUadCF Owajgqi0P43P8s+wcoGgssJYilJr70mpeVfJ6SQih4k8+owS87D/prlx7vLrpg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1682888407; a=rsa-sha256; cv=none; b=xAGQw84+sMjvNr6AngfaCUNbESKe9BpjPYqMoMR4kLjzw3RGZOKOixYK6bVTTIVae+Ce1i QF13mvAubh8LuLPt+z13U2mTAugwcq89qshnLPC2oQqj+38XoKOm60qbhhi7SEvv4YEgHb iVr9QT3wP/rdFXczgbsJXErvT5sWhyM33sp3CRDgX2HYoAenXCJRyDHAwNW3Zf/80cJYPi MpbYt//0CxqDsnT4i9GCPobg+sAXm54wstRAUO4dAsibcJOjfbTeRHBJHIQANUaj2J3D8+ CfnaXgyoBTGrSCfflaWSHx60a1yRdJ/ahY3y3LAyUUonirAOyWT7qqNcs6LXvA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Q8dyZ63Hlzf2D for ; Sun, 30 Apr 2023 21:00:06 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 33UL06h3046688 for ; Sun, 30 Apr 2023 21:00:06 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 33UL06Ca046687 for freebsd-arm@FreeBSD.org; Sun, 30 Apr 2023 21:00:06 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202304302100.33UL06Ca046687@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: freebsd-arm@FreeBSD.org Subject: Problem reports for freebsd-arm@FreeBSD.org that need special attention Date: Sun, 30 Apr 2023 21:00:06 +0000 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="16828884061.9EBd3b.45297" Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N --16828884061.9EBd3b.45297 Date: Sun, 30 Apr 2023 21:00:06 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 238576 | Raspberry Pi 3B+ "shutdown -p" does not shut off Open | 257670 | mpr(4): SAS3008 PCI-Express Fusion-MPT SAS-3: Fat 2 problems total for which you should take action. --16828884061.9EBd3b.45297 Date: Sun, 30 Apr 2023 21:00:06 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
Open        |    238576 | Raspberry Pi 3B+ "shutdown -p" does not shut off 
Open        |    257670 | mpr(4): SAS3008 PCI-Express Fusion-MPT SAS-3: Fat

2 problems total for which you should take action.
--16828884061.9EBd3b.45297-- From nobody Sun Apr 30 21:44:25 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q8gV65HLGz48Mgl for ; Sun, 30 Apr 2023 22:09:02 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q8gV54LjMz4KC0 for ; Sun, 30 Apr 2023 22:09:01 +0000 (UTC) (envelope-from lojas@arroway.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of lojas@arroway.org designates 173.199.118.77 as permitted sender) smtp.mailfrom=lojas@arroway.org; dmarc=none Received: from [10.67.152.49] (unknown [179.240.25.90]) by hobbes.arroway.org (Postfix) with ESMTPA id AB6A814D1B5 for ; Sun, 30 Apr 2023 19:09:00 -0300 (-03) Date: Sun, 30 Apr 2023 18:44:25 -0300 User-Agent: K-9 Mail for Android List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----D5D7QWUMI4FVXGDFKLR3K1Q9TOPW0N" Content-Transfer-Encoding: 7bit Subject: Nanopi R5S support and build guide To: freebsd-arm@FreeBSD.org From: Matheus Message-ID: X-Spamd-Result: default: False [-0.06 / 15.00]; NEURAL_HAM_LONG(-0.86)[-0.855]; NEURAL_SPAM_MEDIUM(0.63)[0.633]; NEURAL_SPAM_SHORT(0.37)[0.367]; R_SPF_ALLOW(-0.20)[+a:c]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@FreeBSD.org]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DMARC_NA(0.00)[arroway.org]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Q8gV54LjMz4KC0 X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N ------D5D7QWUMI4FVXGDFKLR3K1Q9TOPW0N Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, I am trying to have FreeBSD 14 running on this SBC=2E I could not find any= guides in how to build images for it=2E I found the people=2Efreebsd=2Eorg= /~sos/ site that has some images, and one for it, but that seldom boots my = board, and when it did, there was no console over serial or vga=2E If anyone can give any hints=2E Unfortunately my dev skills are not good= =2E But I can test and help build once I figure out how :) Thanks, Matheus --- "We will call you Cygnus, the God of balance you shall be=2E" ------D5D7QWUMI4FVXGDFKLR3K1Q9TOPW0N Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi,

I am trying to have FreeBSD 14 running on this SBC=2E I could no= t find any guides in how to build images for it=2E I found the people=2Efre= ebsd=2Eorg/~sos/ site that has some images, and one for it, but that seldom= boots my board, and when it did, there was no console over serial or vga= =2E

If anyone can give any hints=2E Unfortunately my dev skills are = not good=2E But I can test and help build once I figure out how :)

T= hanks,

Matheus
"We will call you Cygnus,
the God of balance yo= u shall be=2E" ------D5D7QWUMI4FVXGDFKLR3K1Q9TOPW0N-- From nobody Mon May 1 00:33:36 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q8kjC73DFz48Wbh for ; Mon, 1 May 2023 00:33:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-25.consmr.mail.gq1.yahoo.com (sonic311-25.consmr.mail.gq1.yahoo.com [98.137.65.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q8kjC0Wqmz4X1x for ; Mon, 1 May 2023 00:33:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=XBJX4QLV; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682901229; bh=pX018TPo4mKsuDQ3bxYUyNWN8vID7MyjbE1GruXiTWk=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=XBJX4QLVka0i2H66aMH1t8INCWRcaR+krqrKuFUar7QFmE2/qzTSGBtE55qfFvRmIm8v4wcmuCWfTdBSN2Bed3dyyjeOz34xqn2EF8EKRJYf+9CYPTLemvvnPL+fedJvjrewg83AN+laeHdONbC4RwJgiT7dpLYkdvRRBR8jUz7d/K0ujb47RQ0l0QoP13ymqflmZpzHLjzp8LsfpuSwOepmgCaLSUEDCmIkg8S6wHSNtsmqwhAcYSh3ik3NLik6QwYr8NvWFjL1fL7hmKpWd8Sp/VVmDVCkdd4SEZNrKOU2v4bYGgJ8wpuIcQkWT7AEjAqkvfRuEXr4W9xvqcchRw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682901229; bh=6HzOao75VAf9SYbPegXArZRt+zqYGDxD2fLHjfu4Mfj=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=n326ccuVJJJiunFrPlwwXtpF9K0kZWx8Ql73IFE5D43cd7ijIAdZyRPjIINGg0xhagU2UWf0yU/tuc0XpyV2TTIm7ghxso4AeyIu9AQ7JJaMEFY8ocnWTOsL5Q7J8wGYPtBTfm00xmM6gu5keWOFZfiIqOyGnVURXK59LAYRS5JuAG5nEwJUku6ANz9OfpXJgPzrl2IACvrqUZ8Na0SM7qHJlDZmKtf3ipMJlwBf5wHGPHWjofeYSt9q0xcEnPCziwdr/5vnZLdenwYPvzDkpgw84TZSpGovm9v+0BgaXf5jZzZKTKUQqztEF7/JTqyomJtWKHalzMtWCtVvQbkFpA== X-YMail-OSG: 5DT5xD4VM1kcuCQLjQ_qJ1FeyMQ_NT2kt1.Wz.M9bGl5ftmn0_LDyDjNVjVVpvm 3bRl5346LGpsTv3soSYhgIe3hfvsl7l2AL54NuiTQbjdgnDQhlh1WMSVwz7VuLVo3UUwE4piWdar 1Y8QgkwtipCHV6KgwX_j_ntOhI7HqLargLuiEiS_yn.UdqM8KNI0owPjlBrM9JOltstUFWRwe7Pw NQ7ecWQbQvryzCd71o5W0F9tmXLW7I4kqJbTYoUvhA6WMxxsnUlbTWWghrs3JME4.x1CufM13pHV cOymR.Q9yJ3jQMPMEb5zsBFgfaRnILITEX3G.b.7S4HJupu31Fnv9g9_.Rxyu6iSs33GAD3ddQNH 6WdsSdQ03AlAMpx._kDZl2vbjBfOyZRHAfJXQeMqHAcrEmVCWbpt1KLpzrdOvKcC2XEzT69TrEz. EYg1IM87jLmtSxJDOGD1PS8rXMVcTeODS8vogDf9pE2t0BGBPT_eDG7TsWQVcBaIdLAys0tlhbii 0dYha7kAIGcCCpLbQxwNPDhuAShzlC4t8p8WKssub5HmwDRssTlOwKSBpUzJwaEVq2NrrEDwlE1r sxBl_JJscPYAqcgGbBYddAHwCzaNbJob8xoLAJDKKanmDtMmiK4QF9_rmfQu0vL4f5KrXNI7cF.h TkTJ8vA4Chp_wf1.vueTjPlLL61Tl4hsxZMc4R8X7_mRhwBa5dBY8mHB4WrDB4hV.LG7h3aJvFI. KYixQgNQJuPkF0JYqCruE44xf4VKBxtl2oNHNx7dzWC1aMXxVZnLvx0kxVM4Sud6XgUJ2Y0_.DLZ bxUTomd_AB0JW7ZKJkH5vpfmMFaocfJgaZSvCgjgX3C9zN4wgcohPMoG1YyxI52vlUqEjlPjgP0Y ecCGgURBp7qhAqZPVu0wXz_Y71vfTeFHTxGozot9SO0r7IgLSKKjTQd3E3099wYiglDFTwmsH8r_ oksa9c_TTJGFLQ_HqVnLSFsOHuOQg08IQ8rV.lI6IUcPDhcDouVgWRcEEdx5EdGehNSiQvITJvVi tNKNJo8oTNYwUL_skdDi59XnFDqLBNgBBx1hBgNTHUtnRytHCtS8YugPacWlIjmOGjIN56wqbSdS QnMLEOr43GrVgq3iSN8D6p99sKEikrKWw_deheg4Ow_LQtOhbXqusYagkhgUil4US75UpMgwhbF_ P9a_xpv2jXnfucDYa2Kt.ClYguCpxjwMmxd2dj1Y.vMgUZ0aSNwIOdZH.Cuaz3TZEpOgZ5zpSdhz H_19Uc9vOIqBLRs6UJJ6PATzEyLd7oA_4fZTI4l0BZJ5.orCFmyRFUQ7fYaBW3fsj3BsRmkI8iK4 HE3N_mAupSfNwvvtDDvD9..Xu0oZo_PPNluCruU_pLxPo4B9LRdWUKizTXz0C3tpDRX3kOC9bfYu _LUfTvXSo1fsIrjA3CVA7yrW1cJu1nc4d02_g85sriFH0X1sbW5q.4KGp.lIt.uLubO.2mGdwVmo 0edS.oxb4kocafy3FRheSntdtPQV0HMF36t6AVWmkj6.vAoVHsAQJoOyJ0TTIIWcXPx4MIOqAQg9 vPl2k_eFww1rRhgi9SkEzwIHHbHdWX79hJt1MHJ.GwfNM30fqGIuQVbIlGVdoi.4Vj4ZA4glZH9O rNjFr54r8wKxlak_aFUvLSISAtxWI2VWVOQZiu0a.wn7bvUtScJ5Qbf4o2zqdmx8ocpx8bTxAJdA cndaat3Be7h9Aa8RQDfMhKof84Q5omi5R1iV5PoQEHOodLxQZUfW1.sMKXGjeDRMGJYtL5GHDg6S vJ8zt81nb0leGVAKx8Gti7HOjhqVc6qcEny6fVryuGktLwYEU8zGhZG_vfCafleoeXI.RGDD7pqK CG5tbYT1oFz3VKj023OQLZSt5TBBwrDeUqdHkJroPmonCIb.ZiG1h6wUuPKheFw82rWeTU0Lhezg ifFVTyW84jwJw62V1_QfUrWanUeJVRGnVhy8gypkQZIpJ9W8lOl8ma00VMN.XcrctJ0jcOgE9l9k eVRk0Y.fJaxZNlk06JiWLQRrbB2HyrRo7UO.satOyeji8UOnrwE0YQTzJSpLxINWJLMVmef3Ui2D _Qhep2AtSKf0pMjp5sRd6IL8iUNY_a310O14d3g1Wr2Svkb5ph8GsaJpRjvw1h6PEmDH.DCfYa7x d0.v244STR3ZTf4Bys7mi9Ic5ULJwh.mnn6izdV51w.Is2q2I5i7HED80Ieh2pO5.K6109TiB15a A3r6f7k_FKkolBifdvwojiiiZChdehGBEM2r1EU4gjVVKVmVojlsFSZpMfv6KOqdF1s5KeOk3j4v vgg-- X-Sonic-MF: X-Sonic-ID: d00b87de-aef3-4224-83b9-56bd9d43a311 Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Mon, 1 May 2023 00:33:49 +0000 Received: by hermes--production-gq1-546798879c-sfj59 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 4fbe0c6f94bed2bf88666b843be83362; Mon, 01 May 2023 00:33:47 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes? Date: Sun, 30 Apr 2023 17:33:36 -0700 References: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> <02DC03AE-E082-4FB5-AA0D-396F64CC23CB@yahoo.com> To: FreeBSD Hackers , freebsd-arm In-Reply-To: <02DC03AE-E082-4FB5-AA0D-396F64CC23CB@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-2.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.998]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.206:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCVD_TLS_LAST(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.206:from] X-Rspamd-Queue-Id: 4Q8kjC0Wqmz4X1x X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N As the evidence this time is largely independent of the details reported previousy, I'm top posting this. The previous ZFS on USB3 results were based on poudriere using "USE_TMPFS=3Ddata", meaning that almost all file I/O was via ZFS to the USB3 media. The UFS on U2 960GB Optane via USB3 adapter results did not not suffer the reported problems, despite "USE_TMPFS=3Ddata". (I let it run to completion.) But this had both a media and a file system difference. This time the results are for just changing poudriere to use "USE_TMPFS=3Dall" instead but back on the original ZFS on USB3 media. The implication is that the vast majority of the file I/O is not via ZFS to the USB3 media. The context has 32 GiBytes of RAM and about 118 GiBytes of swap/paging space. It would need to page if pet run to completion. Observing, the load average is behaving normally: Things are not stuck waiting. "gstat -spod" indicates the ZFS I/O is not sustained (no paging in swap space use yet). First 1 hr: 262 ports built. But this had both a media and a file system difference again. I'm stopping after this, in order to set up the next just- ZFS experiments. Next experiments: I expect to move the ZFS context to U2 960GB Optane media used with the USB3 adapter and to test both "USE_TMPFS=3Ddata" and "USE_TMPSF=3Dall", probably letting USE_TMPFS=3Dall run to completion. If Optane based USE_TMPFS=3Ddata context still has the problem, even the more performance media would have been not enough to avoid what would then appear to be a ZFS problem, two other file systems not having the problem. The Optane based USE_TMPFS=3Dall context should just handle the paging and more rare ZFS I/O quicker. I do not expect problems for this combination, given the UFS on Optane USB3 results and the partial USE_TMPFS=3Dall non-Optane USB3 results. Even with ZFS working, this likely is the more performant type of context for the Windows Dev Kit 2023, given that I'm leaving Windows 11 Pro in place on the internal media. Hypothesis for the original problem: I wonder if ZFS write activity to the USB3 media was largely blocking the ZFS read activity to the same media, causing lots of things to have to spend much time waiting for data instead of making progress, leading to long periods of low load averages. Older material: On Apr 30, 2023, at 00:50, Mark Millard wrote: > On Apr 29, 2023, at 19:44, Mark Millard wrote: >=20 >> This is based on: main-n262658-b347c2284603-dirty, b347c2284603 >> being from late Apr 28, 2023 UTC. (The "-dirty" is from some >> historical patches that I use.) The build is a non-debug build >> (but with symbols not stripped). World or Kernel had been built >> using: >>=20 >> -mcpu=3Dcortex-a78C+flagm+nofp16fml >>=20 >> just for testing purposes. (Worked nicely for -j8 buildworld >> buildkernel testing for the 4 cortex-a78c's plus 4 cortex-x1c's >> present.) >>=20 >> Monitoring poudriere bulk related activity via top and gstat -spod >> I see a lot of the odd result of one process doing something >> like: >>=20 >> CPU4 4 1:39 99.12% /usr/local/sbin/pkg-static create -f tzst = -r /wrkdirs/usr/ports/devel/cmake-core/work/stage >>=20 >> while other processes sit in the likes of: >>=20 >> tx->tx >> zcq->z >> zcw->z >> zilog- >> select >> wait >>=20 >> But sometimes there is no CPU bound process and the top CPU process = is >> the likes of: >>=20 >> 1.24% [usb{usbus0}] >>=20 >> "gstat -spod" basically shows da0 dedicated to write activity most >> of the time. >>=20 >> After: sysctl kern.tty_info_kstacks=3D1 >> Then using ^T at various times, I see a lot of: >>=20 >> load: 0.48 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7534.91r 11.06u = 22.66s 0% 3800k >> #0 0xffff0000004fd564 at mi_switch+0x104 >> #1 0xffff000000463f40 at _cv_wait+0x120 >> #2 0xffff00000153fa34 at txg_wait_open+0xf4 >> #3 0xffff0000014a40bc at dmu_free_long_range+0x17c >> #4 0xffff000001448254 at zfs_rmnode+0x64 >> #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 >> #10 0xffff0000005fc430 at vinactivef+0x180 >> #11 0xffff0000005fba50 at vput_final+0x200 >> #12 0xffff00000060c4d0 at kern_funlinkat+0x320 >> #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 >> #14 0xffff0000008f8514 at do_el0_sync+0x594 >> #15 0xffff0000008d4904 at handle_el0_sync+0x40 >>=20 >> load: 0.34 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7566.69r 11.06u = 22.66s 0% 3800k >> #0 0xffff0000004fd564 at mi_switch+0x104 >> #1 0xffff000000463f40 at _cv_wait+0x120 >> #2 0xffff00000153fa34 at txg_wait_open+0xf4 >> #3 0xffff0000014a40bc at dmu_free_long_range+0x17c >> #4 0xffff000001448254 at zfs_rmnode+0x64 >> #5 0xffff0000014557c4 at zfs_freebsd_reclaim+0x34 >> #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c >> #7 0xffff0000005fd6c0 at vgonel+0x450 >> #8 0xffff0000005fde7c at vrecycle+0x9c >> #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 >> #10 0xffff0000005fc430 at vinactivef+0x180 >> #11 0xffff0000005fba50 at vput_final+0x200 >> #12 0xffff00000060c4d0 at kern_funlinkat+0x320 >> #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 >> #14 0xffff0000008f8514 at do_el0_sync+0x594 >> #15 0xffff0000008d4904 at handle_el0_sync+0x40 >>=20 >> load: 0.44 cmd: sh 93914 [tx->tx_quiesce_done_cv] 7693.52r 11.24u = 23.08s 0% 3800k >> #0 0xffff0000004fd564 at mi_switch+0x104 >> #1 0xffff000000463f40 at _cv_wait+0x120 >> #2 0xffff00000153fa34 at txg_wait_open+0xf4 >> #3 0xffff0000014a40bc at dmu_free_long_range+0x17c >> #4 0xffff000001448254 at zfs_rmnode+0x64 >> #5 0xffff0000014557c4 at zfs_freebsd_reclaim+0x34 >> #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c >> #7 0xffff0000005fd6c0 at vgonel+0x450 >> #8 0xffff0000005fde7c at vrecycle+0x9c >> #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 >> #10 0xffff0000005fc430 at vinactivef+0x180 >> #11 0xffff0000005fba50 at vput_final+0x200 >> #12 0xffff00000060c4d0 at kern_funlinkat+0x320 >> #13 0xffff00015d6cbbf4 at filemon_wrapper_unlink+0x14 >> #14 0xffff0000008f8514 at do_el0_sync+0x594 >> #15 0xffff0000008d4904 at handle_el0_sync+0x40 >>=20 >>=20 >> The system (Windows Dev Kit 2023) has 32 GiBytes of RAM. Example >> output from a top modified to show some "Max[imum]Obs[erved]" >> information: >>=20 >> last pid: 17198; load averages: 0.33, 0.58, 1.06 MaxObs: = 15.49, 8.73, 5.75 = up 0+20:48:10 19:14:49 >> 426 threads: 9 running, 394 sleeping, 1 stopped, 22 waiting, 50 = MaxObsRunning >> CPU: 0.0% user, 0.0% nice, 0.2% system, 0.1% interrupt, 99.7% = idle >> Mem: 282760Ki Active, 7716Mi Inact, 23192Ki Laundry, 22444Mi Wired, = 2780Ki Buf, 848840Ki Free, 2278Mi MaxObsActive, 22444Mi MaxObsWired, = 22752Mi MaxObs(Act+Wir+Lndry) >> ARC: 11359Mi Total, 3375Mi MFU, 5900Mi MRU, 993Mi Anon, 93076Ki = Header, 992Mi Other >> 8276Mi Compressed, 19727Mi Uncompressed, 2.38:1 Ratio >> Swap: 120832Mi Total, 120832Mi Free, 2301Mi = MaxObs(Act+Lndry+SwapUsed), 22752Mi MaxObs(Act+Wir+Lndry+SwapUsed) >>=20 >>=20 >> The poudriere bulk has 8 builders but has ALLOW_MAKE_JOBS=3Dyes >> without any explicit settings for the likes of MAKE_JOBS_NUMBER . >> So it is a configuration that allows a high load average compared >> to the number of hardware threads (here: cores) in the system. >>=20 >>=20 >> I've rebooted to do a test with filemon not loaded at the time >> (here it was loaded from prior buildworld buildkernel activity). >> We will see if it still ends up with such problems. >=20 > It still ends up with things waiting, but the detailed STATE > valiues generally lsited are somewhat different. >=20 > I also tried a chroot into a world from use of -mcpu=3Dcortex-a72 > and got similar results. Suggesting that only the > -mcpu=3Dcortex-a78c+flagm+nofp16fml kernel is required to see the > issues. This even got some examples like: >=20 > load: 1.20 cmd: sh 95560 [tx->tx_quiesce_done_cv] 2022.55r 3.21u = 9.24s 0% 3852k > #0 0xffff0000004fd564 at mi_switch+0x104 > #1 0xffff000000463f40 at _cv_wait+0x120 > #2 0xffff000001518a34 at txg_wait_open+0xf4 > #3 0xffff00000147d0bc at dmu_free_long_range+0x17c > #4 0xffff000001421254 at zfs_rmnode+0x64 > #5 0xffff00000142e7c4 at zfs_freebsd_reclaim+0x34 > #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c > #7 0xffff0000005fd6c0 at vgonel+0x450 > #8 0xffff0000005fde7c at vrecycle+0x9c > #9 0xffff00000142e678 at zfs_freebsd_inactive+0x48 > #10 0xffff0000005fc430 at vinactivef+0x180 > #11 0xffff0000005fba50 at vput_final+0x200 > #12 0xffff00015d8ceab4 at null_reclaim+0x154 > #13 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c > #14 0xffff0000005fd6c0 at vgonel+0x450 > #15 0xffff0000005fde7c at vrecycle+0x9c > #16 0xffff00015d8ce8e8 at null_inactive+0x38 > #17 0xffff0000005fc430 at vinactivef+0x180 >=20 > (The chroot use involves null mounts.) >=20 > which is sort of analogous to the filemon related > backtraces I showed earlier. The common part=20 > across the examples looks to be #0..#11: >=20 > #0 0xffff0000004fd564 at mi_switch+0x104 > #1 0xffff000000463f40 at _cv_wait+0x120 > #2 0xffff00000153fa34 at txg_wait_open+0xf4 > #3 0xffff0000014a40bc at dmu_free_long_range+0x17c > #4 0xffff000001448254 at zfs_rmnode+0x64 > #5 0xffff0000014557c4 at zfs_freebsd_reclaim+0x34 > #6 0xffff000000a1340c at VOP_RECLAIM_APV+0x2c > #7 0xffff0000005fd6c0 at vgonel+0x450 > #8 0xffff0000005fde7c at vrecycle+0x9c > #9 0xffff000001455678 at zfs_freebsd_inactive+0x48 > #10 0xffff0000005fc430 at vinactivef+0x180 > #11 0xffff0000005fba50 at vput_final+0x200 >=20 > There were a lot more nanslp examples in all > the alter testing (i.e, those avoided filemon.ko > being loaded). >=20 > Starting from having pkgclean -A'd the ports, the > experiments got about the same number of ports built > as of the end of the 1st hour. >=20 >=20 >=20 > UFS vs. ZFS? Different media types? . . . >=20 > So I decided to create and try a UFS test context > instead of a ZFS one. But the media that was best > to update was a U2 960GB Optane in a USB3 > adapter, something that would perform noticeably > better than my normal USB3 NVMe drives, even with > USB involved. >=20 > This combination maintained reasonable load averages > (instead of having long periods of <1) and finish > building 172 ports in the 1st hour, far more than the > around 83 each time I tried the other device/ZFS > combination. NO evdience of the earlier reported > oddities. >=20 > I should also time a from-scratch buildworld > buildkernel. >=20 > I'll look into setting up another U2 960GB Optane > for use in the USB3 adpater, but with ZFS. That > should help isolate media vs. file system > contributions to the varying behaviors. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon May 1 01:57:00 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q8mYQ0yBJz48bRP for ; Mon, 1 May 2023 01:57:14 +0000 (UTC) (envelope-from ganbold@gmail.com) Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q8mYP4Jd0z3Gct for ; Mon, 1 May 2023 01:57:13 +0000 (UTC) (envelope-from ganbold@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-io1-xd34.google.com with SMTP id ca18e2360f4ac-76937d6d2b9so128999239f.2 for ; Sun, 30 Apr 2023 18:57:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682906232; x=1685498232; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=VNmv2YpaeI9CTKvOiwTu54LaJNIFBOw1eOBhzCSrjHU=; b=iMHhx96sbwIoQrqE3nVNwbIeyoUFMBp0O0B4tAq5V4jnX7XylV+fHuQi1eX07rBxLq kfSxrA0OylFFQJDxjxKIr+in1I6wOnPJUP11VDl6JGRJW7MW6t4EUbjHysheUgAghLU+ 9PYIrvBKKkboVSjVx0zL8o6cIBwWHaYYf6b8/MWKXld4NS3VznHJTkgERn5iyq/bSYiw UMQ85+2GFTR8cFHVZsfXDvz6/VUCZ8XqVemCHiwDJv8VR4YH/tZQ5FqOu58IgUZ5O2fB FkTrvIsBZ7zo2W65jrwxGHM/8iigEP7/WVbvs5EvrHjrZEJRSSE4aykJ7bGBeDCYX0Io A20w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682906232; x=1685498232; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VNmv2YpaeI9CTKvOiwTu54LaJNIFBOw1eOBhzCSrjHU=; b=HpOzDHmbcMctj0ndvA+fNdyz0Y7f/vlFlsMg6nSAne/Vk7AGDb9Q97nQ5HpDgSdtw7 v16BvW6SWAyj+QQlww3RhGwxrts4O35OkpAm9lNJkIsT5/2Hb5eATE/W6sVTbcZVgUuD Sc1gZ1il6puMfWVhxYYtm07gHNqt2bVUZaPHg9SbxOyODj4Sh5+tAOBMXiCHAFUI+M3Z gtUtify/Tug12xgLdeWmPifZ1TtzN+mE8bWhJF79FrIEPra9I7RlKKW5mm2iSOv6KN4v 3QS7pL8CMDcCHbv+lQZfgyOfc+PBPiUkBn0+03psF4LLkKQVHduXBFYs2A9qqPWHiUye 74Jw== X-Gm-Message-State: AC+VfDwUJb/66vfNv7zlhQHIbrYulkcCP5kKxdXqZdoeo/ADnN3if5n0 cEPP0TsCJpmsEb/ya9HO3w5eOXfcjHApQ0ywf8C9QgzBg4ZiCQ== X-Google-Smtp-Source: ACHHUZ6AlB+HUb3Kfx0/MWBcRAoE7fgFlWrmqBeu5yyfFtN7BLAA+TjzTogBl8nX3836Cy/vwHCF6BzTUfXCEySntVg= X-Received: by 2002:a5d:9a11:0:b0:759:95a5:327a with SMTP id s17-20020a5d9a11000000b0075995a5327amr8715417iol.11.1682906232116; Sun, 30 Apr 2023 18:57:12 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Ganbold Tsagaankhuu Date: Mon, 1 May 2023 09:57:00 +0800 Message-ID: Subject: Re: Nanopi R5S support and build guide To: Matheus Cc: freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="00000000000027dc7505fa982497" X-Rspamd-Queue-Id: 4Q8mYP4Jd0z3Gct X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000027dc7505fa982497 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, May 1, 2023 at 6:09=E2=80=AFAM Matheus wrote: > Hi, > > I am trying to have FreeBSD 14 running on this SBC. I could not find any > guides in how to build images for it. I found the people.freebsd.org/~sos= / > site that has some images, and one for it, but that seldom boots my board= , > and when it did, there was no console over serial or vga. > > If anyone can give any hints. Unfortunately my dev skills are not good. > But I can test and help build once I figure out how :) > You can try an image from personalbsd.org: https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Nano= Pi-R5S-20230402.img.xz There are no build instructions available now. One of the reasons is the DTS. Ganbold > > Thanks, > > Matheus > ------------------------------ > "We will call you Cygnus, > the God of balance you shall be." --00000000000027dc7505fa982497 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Mon, May 1, 2023 at 6:09=E2=80=AFA= M Matheus <lojas@arroway.org>= ; wrote:
Hi,
=
I am trying to have FreeBSD 14 running on this SBC. I could not find an= y guides in how to build images for it. I found the people.freebsd.org/~sos/ site th= at has some images, and one for it, but that seldom boots my board, and whe= n it did, there was no console over serial or vga.

If anyone can giv= e any hints. Unfortunately my dev skills are not good. But I can test and h= elp build once I figure out how :)

=
There are no build instructions available now. One of the reasons=C2= =A0is the DTS.

Ganbold

= =C2=A0

Thanks,
Matheus
"We will call you Cygnus,
the God of balance you s= hall be."
--00000000000027dc7505fa982497-- From nobody Mon May 1 01:57:16 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q8mYk6lLNz48bjF for ; Mon, 1 May 2023 01:57:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-21.consmr.mail.gq1.yahoo.com (sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q8mYk2HH7z3Grv for ; Mon, 1 May 2023 01:57:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682906249; bh=/cmO4ZCOIvGXCD1Fw2ps4OdrhftESgdb7dAOMUjlo6c=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=p8NJJ8Qn6sildfLBF4+3ZcGc7HY+g1R2EgcciFY+xC+uGLLVOR0KfhdYv0e75xkoxXEKJ25cGxZ47PivLzSWb6FElctekK37qMya1kdmfufaWMoW4O8UfDjolQTl9s3DOyBGbf0hZMU/QC9Y2toxqY6/7OVQUmyC+srPgH8Y1YPzprHL3GUG4zOgmb7by9hdrNzb4wBa6xfMH/KExrhWMXX67NHLrN+WjD0iNXacMPHqxRkJh8SO09Q+sHGZe8XDFBrSjrpGEwiWNvFmvxWULMYkftUdd73zKhmxnxxBatsGSEpGNdq6DjtcISIoZ9HyHb+rspkkPTszCFISQWz7Iw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682906249; bh=IcuLuPRM6Qx6D0iXZoRNleRklEyw1cQ7kmQHLDZt0mv=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=tStQzEkmf4UUpaAbB6aZRXBLMImmFHFcUdGVMzsJIdlE3fPLF5jGcNNSaRQo3HNr0UykCBbVVE+yP7rpl01ORIGiiM6B/5+WdgTC3nnsJENATkb7Kqr+Mw8mQuGslXHUMTW6qQJF4T1fQIOqVZ/lDf9m/KjQPKxc6s9Xu8mYfVOIZN2YjKHdU3rPq9sJ4fnwyCgfKtxKEhKp4ntvhJGPvuGxIwmleBV1dV/JQDkUIu0f0y9N0eUVbdK54hFwXd/3qGpQf5cbT2eUDHgruXAr0j7x7p+uS7knVQ/Q7/JzxzDBCgWiOjKnH3p3VyxyqX/X2mRxiqBNTkX4JBJMCJjA5A== X-YMail-OSG: .MpGC0EVM1lsBfC768LJ4BBmnh5_Q0CBQQSdx9k.E4NFGr285ZVBbl56PbCpy49 T09EvIc4JuojdTq6Gv.zUKIEsEpJD6a4kxYxSlVS8anIxGND3OnOdQkbgN092wWxYhyhZquzxxS0 M1US_zWdnwlBfQXAPPuLziXo.ykFClsNdVEG5347ul9f.XoJejsC0.KWtWvtUIVqc1XZDPmXyZ8m 9PHDHuftbjg5zDfinBzSSAhpeCibcvc4ehd24ePmwV1HjSeW1cD7b2aQs0ORFf1xUlr5613YsTr8 JfZnCx_1Wiszq5h1tVbu_Lc1.JdBE9mI7L6Lu57c37a4_n3RHQrW4Vzp_04TK2KbXOT8tgZ7.46R dfUIF8ZZjQexmHpt417su9r.U0bgraPVUB4BYC5WPCnc0UGhqg0ZqxsPRCsVwoNXpBp76WS7_plT wQcn9BhwhGlZ0YY0bPKVf4HX_4xlFJWc6hxmYf_M63Vz03iD2KPeqbVBS9lbeIz7c1ym55PE9mKB OhPSG0VN0X2E0UCOtSCMFxqyjppaLgN5EA2WfFwwiFBRxvEvpmsaQivr0htWNsrflHYmLlERLdBT .VrC27x.ZA4_HQp00srAY4Gb1XEn03kiBcKqMMpDYFMtqIbGHf3rvj0mv9Sxo5xDBqnDP1bIFqSk z4irFtqVkYx.QNIFICVTX2mbChpLE17DJjUWlOtm1oyoHqxXlx0KvkfDP635Zu1K7l30gGGTE9gt rSTm4y5AhHSBQqQWfSPpU5Sx_8ljZQzEM2qRwvNg05zjMRN__jQESYWLI7Phc_rcjCKbNwpJChd2 RR.t7kskIa5.cld8z2Fq1XnTHs2kQDNJZEOCAbTmXnJ.1gVmYsrMLuQ0YpY6rR2FT8WsFoFg2CqT wTkhCwSSlj4viYjPRsJPThmlKdvoQc56LAFvq.OZdT8k3ANmht1C1u3IrVqcyvvO6MwPWueCvTwK sqdlKTuMC6oD01TXm7OS2WT1xYjfgoAzi8nUbnF9n2UytD.mEgGQ.X.q76BfAFv2qTzEWWr.VWUY mZNFO5khCRjFyDmlxciD31SqLrJ_1M4gCqmkYPeKraZ7AJvny1tkKQGE0Vu7_xqEL5Se6v3Ceufo 5Hd0YpItb8ljNrwbt0Rx47eUlgjRP6HHHhXXua.EdhvAMLbngbtO5v1MYsIKLO9fXKBxBNgyd6pj AqgUifECsbqdNVf5nWEU0YDSZyMzj2020o6R2RuO5g44.3Dew.pV0oBpXFzaS0qvHqIV7I1rF4Qc jFrEFGaQKYKXTTvgh_MuPYyjTO2cCeqXB9OY.bnOwWZl7mTjxmU6Kb.08hMRv3db2chrGCEmx8aV .yrn8P2o7sXIRut4KXt40kK2rCiMIUgJANj6msTdYmTxGAWyp77pY4ovSK.bHnHOHhVN_fEa5YZC qw0Ujmr..lb2S6FLafibS3F5n_lOb5l4Fxt3lvWTGGR3ecu1TnDPaygyQm31NVoY_LQM8nsBGDT0 v8aTOxPF0yb_6XE7.tAjzsxu21C2KsTFZAzpZtnMptPPIZx5IlzNEMT3QeI_hv.3wJVkcaa0usmH zRr.lz7nwdsbCC1I0STxZ_gG2jn3Ifg2U_y7c8zM7DVqcCp_GDxiogQVR2EbyZ55K525o2moIljC c4JNdAm3YcMd94luW4ohync5KTy0zzZKzAox1ODIXIJy3INd8vf4zEhoEVaA22WDN78W05YfFWcS IKIXGE3fS8qI6LgwCSVnUi4V_Wmwxuavpy40hPS40C7qjAttytk_PtcDtOjFBY7gAQB4ELAgiKzr dj7rj3xeJ6A5vygfSuZsZoJN5Q1O1LxVM4HTU0pKb7X.jjFe66Jf3N2N0MPHG1kMjGgj2WD1yfo5 8na3J9dnv9k_JsI.QHZZS_jl8wMr.g0HQB.9fOIqDXySjnHJnP6B15BmGy0rpOfpPd1TWCPiWYuG 8xmyRX6TF79uKS9BfXMWR5.ZxtT7z8sqRIFF8fqFeYrL5.nYxXg.sS0YY0u407ZsBzgVrl83vwkW Hmiu6PUkwfhRzl_6cL29g5Pc6ExDEq2NgF9YzjEZk8W7ZbdCPSd_IHPFqWoqyndUFEVCdEvAtlvc 95k8CQinpmdjKI5SwX994pCWMGNmUSZd3x8KDLsnAzRfsakbvX0mYayXuy.7cBT2QyuIb4Ugbd4b y86NMG2gRJ5c_ybmyeb39zJID8DLj3MAMASuKjUlotPqIhqXnIsCyM4AlKp7uhexdcK6USg8W4c1 YsKXeubGKC1zNEpdGoyIMzjagzCkr4LZlhCqlVE2wTfve51eHOmss8DinHwXUfwml42UVxhA9OYm 1 X-Sonic-MF: X-Sonic-ID: 73c7f526-a957-41be-ac47-57f7d13d9aad Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Mon, 1 May 2023 01:57:29 +0000 Received: by hermes--production-gq1-546798879c-qx24x (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID b5c8fa2a09aca77006c3b1692a533c4b; Mon, 01 May 2023 01:57:27 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes? From: Mark Millard In-Reply-To: Date: Sun, 30 Apr 2023 18:57:16 -0700 Cc: FreeBSD Hackers , freebsd-arm Content-Transfer-Encoding: 7bit Message-Id: References: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> <02DC03AE-E082-4FB5-AA0D-396F64CC23CB@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4Q8mYk2HH7z3Grv X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Apr 30, 2023, at 17:44, Mateusz Guzik wrote: > can you redo zfs test with: > sysctl vfs.zfs.per_txg_dirty_frees_percent=5 Sure. Result summary: Seems to have avoided the sustained periods of low load average activity. Much better for the context. Context: Original ZFS USB3 media. World or Kernel in use had been built (non-debug style) using: -mcpu=cortex-a78C+flagm+nofp16fml Steps for this test . . . # poudriere pkgclean -A -jmain-CA78C . . . # sysctl vfs.zfs.per_txg_dirty_frees_percent=5 vfs.zfs.per_txg_dirty_frees_percent: 30 -> 5 # grep USE_TMPFS= /usr/local/etc/poudriere.conf # EXAMPLE: USE_TMPFS="wrkdir data" USE_TMPFS="data" #USE_TMPFS=all # poudriere bulk -jmain-CA78C -w -f ~/origins/CA78C-origins.txt . . . At 15 minutes into the build: 46 ports in 1st 15 minutes. Load average stayed reasonable for the configuration. At 30 minutes into the build: 102 ports in 1st 30 minutes. Load average still reasonable for the configuration. Looks good compared to before. I've no clue what optimal would be for the context, but vfs.zfs.per_txg_dirty_frees_percent=5 is vastly better for the context than the default 30 was. Thanks. I'm going to stop the test and do the conversion to the U2 960GB Optane media in the USB3 adaptor and then compare USE_TMPFS=data vs. USE_TMPFS=all --but using your vfs.zfs.per_txg_dirty_frees_percent=5 assignment. > On 5/1/23, Mark Millard wrote: >> As the evidence this time is largely independent of the details >> reported previousy, I'm top posting this. >> >> The previous ZFS on USB3 results were based on poudriere using >> "USE_TMPFS=data", meaning that almost all file I/O was via ZFS >> to the USB3 media. >> >> The UFS on U2 960GB Optane via USB3 adapter results did not not >> suffer the reported problems, despite "USE_TMPFS=data". (I let >> it run to completion.) But this had both a media and a file >> system difference. >> >> This time the results are for just changing poudriere to use >> "USE_TMPFS=all" instead but back on the original ZFS on >> USB3 media. The implication is that the vast majority of the >> file I/O is not via ZFS to the USB3 media. >> >> The context has 32 GiBytes of RAM and about 118 GiBytes of >> swap/paging space. It would need to page if pet run to >> completion. >> >> Observing, the load average is behaving normally: Things are >> not stuck waiting. "gstat -spod" indicates the ZFS I/O is >> not sustained (no paging in swap space use yet). >> >> First 1 hr: 262 ports built. >> >> But this had both a media and a file system difference again. >> >> I'm stopping after this, in order to set up the next just- >> ZFS experiments. >> >> >> Next experiments: >> >> I expect to move the ZFS context to U2 960GB Optane media >> used with the USB3 adapter and to test both "USE_TMPFS=data" >> and "USE_TMPSF=all", probably letting USE_TMPFS=all run to >> completion. >> >> If Optane based USE_TMPFS=data context still has the problem, >> even the more performance media would have been not enough to >> avoid what would then appear to be a ZFS problem, two other >> file systems not having the problem. >> >> The Optane based USE_TMPFS=all context should just handle the >> paging and more rare ZFS I/O quicker. I do not expect problems >> for this combination, given the UFS on Optane USB3 results >> and the partial USE_TMPFS=all non-Optane USB3 results. Even >> with ZFS working, this likely is the more performant type of >> context for the Windows Dev Kit 2023, given that I'm leaving >> Windows 11 Pro in place on the internal media. >> >> >> Hypothesis for the original problem: >> >> I wonder if ZFS write activity to the USB3 media was largely >> blocking the ZFS read activity to the same media, causing >> lots of things to have to spend much time waiting for data >> instead of making progress, leading to long periods of low >> load averages. >> >> >> Older material: [I've removed the older material.] === Mark Millard marklmi at yahoo.com From nobody Mon May 1 16:47:12 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q98JZ10syz48l1h for ; Mon, 1 May 2023 16:47:26 +0000 (UTC) (envelope-from soren.schmidt@gmail.com) Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q98JY5gXVz41rl for ; Mon, 1 May 2023 16:47:25 +0000 (UTC) (envelope-from soren.schmidt@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-3f19a80a330so15075345e9.2 for ; Mon, 01 May 2023 09:47:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682959644; x=1685551644; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=MER8srqqsi0Rnw1RWe2Q1sZmVxzvsq3CytFLObxoOTc=; b=Gtjg7O1J8BKVCzYDaORSUqdN/JdyX/6gRrua26j7CgW8+vZj7YRbmQMu9M+HL2nEYF vf9AH/pA0mblLZETwjLOhKpjSL8q15qPvlOLWI5IOlNdpjxcekSPFecQ9WG1k788xgHf owUQsxk398SWFa3Yqz9r3F2EqqFPWuTcnNQGP4tt2k87YSfhcFdw840x0MQDdgd1ugLE uR80Sl8NkeOWWGUP1xQ2pWn01M6HvimSXYKvaC9jMFvsOPQGdW8VSlFiMM1L2zNV0eyF zoZ3doDgj7aDAoLSttvGHU1s6OjOf0sOwU62Gmu/Hz1RSS4jRqeOWrjLV23ai3L+VzJQ IN9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682959644; x=1685551644; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MER8srqqsi0Rnw1RWe2Q1sZmVxzvsq3CytFLObxoOTc=; b=bFIHXiWJF2hUmLe+sX6FUKEcCQGBwSJRlBko0xFUxqlAeaNes4p/LJt3j9qrmu5Ds6 zbQFd5l3XzutY73aVQ0B8kpu642oZ6XifUC9ZKGvly1pyNMB1HyTTMv1bMQKcylTuP+n ZH0VB0ll0PevZ3s1Md+35u0fdREwA0385L9Q3jzo7STM1ZljUT9YCpd28UjobaSeKKI9 Pi2l+ipfkp76R5RlR/Xs2LVOM9doyL22juanx/MS6w1XcHoGb/9+7ThZP2vi7koIWxRI z/h6YbdqJczbdLLGRUNNoliEdp5mMMwzF4oRISjVIkc1YPD6NV5q5bTwvZvf3esj99Ul qFuw== X-Gm-Message-State: AC+VfDxc2uqOqNyQgjSIBsRhozikZx+aXx+4Q+FlzTOC24a+SKiaZDmy 4rTVj5AlMNttxC6XjjHwxRuqOIRePzX5IA== X-Google-Smtp-Source: ACHHUZ43J5RlpmhN2CfS6F2uUW2sBsOdFKUahvcvqQEcod/1ciXHr73ATEyMJ57AolyP9BbKE+In7A== X-Received: by 2002:a5d:4a89:0:b0:306:2d28:d556 with SMTP id o9-20020a5d4a89000000b003062d28d556mr2472942wrq.34.1682959644009; Mon, 01 May 2023 09:47:24 -0700 (PDT) Received: from smtpclient.apple ([85.27.186.9]) by smtp.gmail.com with ESMTPSA id e8-20020adfdbc8000000b003047d5b8817sm19646365wrj.80.2023.05.01.09.47.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 May 2023 09:47:23 -0700 (PDT) From: =?utf-8?Q?S=C3=B8ren_Schmidt?= Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_73C69236-D40D-43C4-AAB8-7C0C0EFF5989" List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.500.231\)) Subject: Re: Nanopi R5S support and build guide Date: Mon, 1 May 2023 18:47:12 +0200 In-Reply-To: Cc: "freebsd-arm@freebsd.org" To: Matheus References: X-Mailer: Apple Mail (2.3731.500.231) X-Rspamd-Queue-Id: 4Q98JY5gXVz41rl X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_73C69236-D40D-43C4-AAB8-7C0C0EFF5989 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 30 Apr 2023, at 23.44, Matheus wrote: >=20 > Hi, >=20 > I am trying to have FreeBSD 14 running on this SBC. I could not find = any guides in how to build images for it. I found the = people.freebsd.org/~sos/ site that has some images, and one for it, but = that seldom boots my board, and when it did, there was no console over = serial or vga. >=20 > If anyone can give any hints. Unfortunately my dev skills are not = good. But I can test and help build once I figure out how :) Hi Matheus The image at = https://people.freebsd.org/~sos/ARM64/current-RK356X-images/nano5-sdcard.i= mg.gz for the Nanopi R5S does indeed boot with both HDMI output and serial = console (1500000baud). The boot loader (EDK2 in FDT mode) is very picky on SD card quality = though from experience, I works for me with Sandisk Ultra / Extreme = cards but not with Samsung and cheap noname SD cards YMMV. You can build a stock ARM64 generic kernel and most things will be = usable, however as Ganbold wrote the DTS files is not in there yet (and = not even in linux where our DTS files are fetched from). However the EDK2 boot loader provided (and used in above image) on=20 https://people.freebsd.org/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb=EF=BF= =BC=09 NANOPI-R5S_EFI File =C2=B7 1,7 MB does hand over the =E2=80=9Cright=E2=80=9D DTB file if you want to = experiment. If you need the used DTS file and build guidance let me know in private = mail... -- S=C3=B8ren Schmidt sos@deepcore.dk / sos@freebsd.org "So much code to hack, so little time" --Apple-Mail=_73C69236-D40D-43C4-AAB8-7C0C0EFF5989 Content-Type: multipart/related; type="text/html"; boundary="Apple-Mail=_729D9769-0090-412E-8CE7-99F17248EDD8" --Apple-Mail=_729D9769-0090-412E-8CE7-99F17248EDD8 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On 30 Apr 2023, at 23.44, Matheus = <lojas@arroway.org> wrote:

Hi,

I am trying to have = FreeBSD 14 running on this SBC. I could not find any guides in how to = build images for it. I found the people.freebsd.org/~sos/ site that has = some images, and one for it, but that seldom boots my board, and when it = did, there was no console over serial or vga.

If anyone can give = any hints. Unfortunately my dev skills are not good. But I can test and = help build once I figure out how :)

Hi = Matheus

The image at


for the Nanopi = R5S does indeed boot with both HDMI output and serial console = (1500000baud).

The boot loader = (EDK2 in FDT mode) is very picky on SD card quality though from = experience, I works for me with Sandisk Ultra / Extreme cards but not = with Samsung and cheap noname SD cards YMMV.

You can build a = stock ARM64 generic kernel and most things will be usable, however as = Ganbold wrote the DTS files is not in there yet (and not even in linux = where our DTS files are fetched from).
However the EDK2 = boot loader provided (and used in above image) on 


does hand over = the =E2=80=9Cright=E2=80=9D DTB file if you want to = experiment.

If you need the = used DTS file and build guidance let me know in private = mail...

--
S=C3=B8ren = Schmidt
sos@deepcore.dk / sos@freebsd.org
"So much code to hack, = so little time"

= --Apple-Mail=_729D9769-0090-412E-8CE7-99F17248EDD8 Content-Transfer-Encoding: base64 Content-Disposition: inline; filename=preview.png Content-Type: image/png; x-unix-mode=0666; name="preview.png" Content-Id: iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAAAXNSR0IArs4c6QAAAERlWElmTU0A KgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAIKADAAQAAAAB AAAAIAAAAACshmLzAAACvklEQVRYCeVXO5LaQBAd8VlB2YHLrlpiTgABN9iYwKnLqS+yl9grOHCy zog4BCkRGVVsKQFjBAL3G+u5WkNLSA422akavf73mxmJnXXurY/I2IBovV5/juP46XK53MMv6MNC ZG6WZel+v/+xWCy+TafTX7TXwZYRdCfjKYqie5neXYYq904IfJlMJj9ns9k7Zb8pWgTidrvtV47G Zc1pR4fz+eyEgOv3+w/j8fi5CQmLQBtF2aAOCRxNmqau2+26Xq/XiIRFQHoWtz4kFOryDngCnU7H YTYhYRFA/asdqNoJHMHxeHRydJ5AExImgXAHtK5lz1Qe2IHT6eRarVZjEiaBcAdCPSQBHeeP9yAk IZ/zw2g0ep7P5+9JWKP1O/Bxu92+MCj89rUOGfNwOLjNZuOSJPFfBI4ChHAU+Yvp5NP+PhwOv0rd jLWBJoHdbvfCRgiibCFsfAdwDNCxI9gJDGC+Y8fBYPBBTIUfqo6PuvFAARYOEaloIluNVfpKiMEg esW5riACCgTMdwANOfNkrsLEfIWmj/k5XvW7MjAhLAp7aKNe5dMxrK3RJMCkEMsaIY6zLEY31bJJ 4FaRkJjWtWzV0c0hmwTKiujkshjYy3w6n7JJAE6rCG1Mpm6hZWOeRpNAWTLsnCxyKxZxjGGORpOA TmJyiLpI6NO6lnUOZZNAmGTptLEQdQtpY6xGkwACmFSF9LEgdQtpYyzRJMDgOogYxqEo5TJkY6JJ oE6hqgZVPjYmlhJ4LRImAa7gf0hwZVau9lE2CehkLZNYFdLHBtSJtBMtAvJn/LLRjbXMQlVIH5tA l0sLav69KNAhaBHIlsvloyQkiNPFKNdBxDBOLq3JarV6lHKF65ivj0cwYtE/5bMnaF3bgpRKFav+ LRP3TMyDzH/DKg4brk4ggv+SrBgx1x4ggJWjcSqzcAx/AH/H2pgSKdsEAAAAAElFTkSuQmCC --Apple-Mail=_729D9769-0090-412E-8CE7-99F17248EDD8-- --Apple-Mail=_73C69236-D40D-43C4-AAB8-7C0C0EFF5989-- From nobody Mon May 1 19:57:01 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9DWj4YzMz48xL8 for ; Mon, 1 May 2023 19:57:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9DWh13tTz4Pry for ; Mon, 1 May 2023 19:57:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=hBJGCRje; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682971038; bh=xuLSkyPWETSS+3g7ArElxHEzmCDoZIj4wr/z1L33qJ4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=hBJGCRjesL+OhLPjvqGZipel5WepPF1987oHJ3N6sdKHmzuVS1FVF+RlMrNBZqJ3ITF1peVtRhA0nRNo4tSPTCneAB5QR+vKYXoPmXqbgh7hepqK2qRIRk8qdxF9CQi3aQwZsxGLEzXrh0uMPZPLB49zCgepdCy235P96div5/QlNxkipp7zXg3GvCnsitJBnCF5fiCr51e7OTPCc/yQqdDGBzai+H3iViWQDUI4ehvK3hINvVDf++GQdLpPrQZtwh63pEpLJ4NZeQG+K0bIv7ZQtV7jjS9xFzCIES8U0d5pD3T1M5LqT9wqe51mio1yI49BgX3oobLTJOrqvWAYeQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682971038; bh=Ra7XaJuAE/zkOrGuNBU+9mhZ9jsJmpcpFinDNIzoq78=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=MwMXQ3PGAT4NBYHfvk4EGL+XrhVjvxGYrgnOa01hIDCCduhcNSJM3jzpX7RcdmUL3uHin3zQpa/8xKbl+eiO03FXiN3WHG9DhWqXrn6OgUEKJNEhVRuMnUKq+EkkZisdkML7TmVoqwqiJKr59w1v+Tc2OHqNkyp7dBRAeDKmCpXce2LQ7Nm4ncj82B0t6gjH8/qds6IPZ63V/+k3uCLTYsAJJzFnD99ZeEyDU7yLr0VZuRmF9QgvIwhZR8yozs+TZXp3h0VairMAxleNXu2WQmwNQ9IUi1axo4DaR8i2ewflRRIMCMk+zcRwHz0X+Bv+mr2edz5guPw/mr9PhPctCA== X-YMail-OSG: PqQt0j8VM1kkdc7MA5NWMAmvhluv9gq1CFi63iMrijg423MtlF.J.icce02Z7xt gQ9zzd3a3VHJQckL4zdVW_q_TVNRGdr0MEyUhF7t1Qx7j6anSF6RUUsr.LV1joNBKBNezS1NBUt9 w7PWPV.BQd.Gb7NOShDRBRJ6mbqWHs4R9IJOMuSBX1ZJtWwKPle4Y22K6ASaPe9thnrofxuv4cfa PGPw1E9.hy1DqZwsKzeWMS1jAJeWBIk1XZoc8sGs.5YJm3McRaSAJhu_DDDWi9aZc66IMzag4lTC 0T10sB1BsOaQItYMnf5yu_nSSlZjXog8fMahtQZ_J43eHKho85WqkjD_mGrO_LqlIrj9smvCCUIq cQUtc3ASCn5nAGIKyGFfKyaZS..wQRqmVzaOLV4irgWmi.gfj4wT4WGne_pmlwDohW4N6HTVeFTZ 9UiPE8lZP.aiCZG5uv1rznnv_hLvHe7omV0Ol6e_R1rhGylS5.l0f8Pc50yNX7zqP8HPKR6H0WHp JY68ie3_IAV7nfBr5mPL8.PLh.5YxRBK_ew_jYBZ2cN09i_iRDsXEoaKYrc.dX0CbFUXQHhrdJ7S BcsQ9DLiOZRh2qTJJXnifYj30InzWbLazAz46qAOQyQk14ZuTV9zdaYJOsHOTLdchj992LUBR1mO XpOZ6T3ArLrD2il5xv4BtvHDOCDMIvW3KLDo3MxQ1cR1dZwfDfXqiK71.BtfUtgGTfzwKKcyxMAC Z5w72uEusKHKb6SC19zP2cXE1Wy6qxUV4iUihmhdTOyrm95mUrKND6IppmxG_.pKBi9qbmIabzwH fN.I80VwhzfGPdMUo8WBE5PMsbcCOhmRe97QeBeWHH1ibC8FBbhuuz4ofuOzJzCEDrcvEb.nmNyc iMVc2__9T.EefTd4MGAMWc6VWEyhy_8MqaE27QMro.f.p1sXbulu_I6ESJ6TJz15sl0yB0aRcUrp CBsuX9nPFvqqGUJvbBLkRUk60BCIYx.DbaLmD9xPH8zB8kYSIIbh7B96x9XAoHrL0f9WUUfYvymP _3NzoYVjVm6T0iUqRr86FtZTWzWSM.q3xH1ZLP0IQriIa7CbusIOWJOoEIjthMw6Wrmfldn5eX.T 6MRQ6bXhh1gVesqyPGnkd81ThftcWWdyU5qe7uprzqnF9K_iB0KUC70geHy6roDUY9mQn3eDQ8Ij TuO6zk6VxDc9xiRJy4NBZmEGMYkFw9S7aqd4Sl8pRQUYKqX_ns_47_7xv6rqDwHtK0HA6v_uczEq 7GSdP0lsCOlGuuFWCJXljIRHX1V6zD3gnr8KIvvLC_avwcnixfxbhyMWTrgfwAm8ukFEQ6XI0l2L YdAGdxqlRb_Aar8orjp2sKbkJD1cpbs4ZlONUdh2TI765L25lI3EdXaatE2SCYUgz21e5BF2_.1n GldL9xRU8e3LfI1fXMngIA38hEWVqZ7jPwJY_GsaXQN9.392N1BykpTwgWH7pkL1NpjP09MkZ6av SbDnt6mBJVElOFHM0ArglBYj1J2.7zgeKwwhOW3tF.BPJ3EjiLvdo_DK8WwEn_35fI35k2OTcNAY 6S3ZDyU3gL6DLeCHnChzQEWuur3pnv4Wl_YEIMr6mYOQfhiYtTJWu7kPmNTGphl29jnuz9NIs1HF O73m59NRzdqOInt6FHicx6EUvH_mfWoDcC5m0.hjkx70UoYgq6Ta0tsXoQu_YlUvOlkox0vVISKr hsTNnTm1egP.olCArEm51BX9gligWXgwkEuiNEZS5U5YkvFP.hqBng2ljabDgJvaMOKXLTIZ5G7r egmWPCTjLvttW6xdKP8iInW29AZPGBaCRo6avQ0GYRS5WylZ3r350xKQf1EHDv3dvI1Ke3dD7aZw 5uMUKFIs0u4WOKfaw0941gvifchDAQxY7hD.f9aibsNIswtkZPs7670QRr_fGIXrM3TbZpMHT5nA 8u9ydNztTuhcAQnJ.1bX1ViqpFY0uk7wWGECCckg.39TTBzA2bTggxPe4zKEXyTfRPIazcadg491 r.jHSAmpKl3RGcTw8zznYR6aXeCBL_17C68PIq9Z8TxMQMro6RGlKDc.xI1oNMHRHmcKGOz.Iz9R IWKWvV2uI02EKt6Gh2r6GYLPxGBbSaF_NB_IqhEEj7fkr7L_Yf23c35rWC5jVS8.BcfqrRBh2BYj sFRwHySwsOaasMS2GbaFUpzh5RU8gIoQlvksKKz2nWoTEn51yEMQmJijrwga7L4kLswAmbK5AIHc gdoqam.Vydeg4ULWQbRAXHLeVnlMu_EN3tWjyMGrZ3y0F0uBE8xpLQ1Ylw80fV55ttUwyosmFgeC m X-Sonic-MF: X-Sonic-ID: 5b84c1d3-7548-4222-aeb9-79325bd06b9f Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Mon, 1 May 2023 19:57:18 +0000 Received: by hermes--production-bf1-5f9df5c5c4-fgkgh (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 7948446c661a58a6abcb434ead47fd04; Mon, 01 May 2023 19:57:13 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes? From: Mark Millard In-Reply-To: Date: Mon, 1 May 2023 12:57:01 -0700 Cc: FreeBSD Hackers , freebsd-arm Content-Transfer-Encoding: 7bit Message-Id: <8F883286-D713-4632-9575-5813E885D125@yahoo.com> References: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> <02DC03AE-E082-4FB5-AA0D-396F64CC23CB@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-2.42 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_SHORT(-0.92)[-0.918]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org] X-Rspamd-Queue-Id: 4Q9DWh13tTz4Pry X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N On Apr 30, 2023, at 18:57, Mark Millard wrote: > On Apr 30, 2023, at 17:44, Mateusz Guzik wrote: > >> can you redo zfs test with: >> sysctl vfs.zfs.per_txg_dirty_frees_percent=5 > > Sure. > > Result summary: Seems to have avoided the sustained periods > of low load average activity. Much better for the context. > > > Context: Original ZFS USB3 media. World or Kernel in use > had been built (non-debug style) using: > > -mcpu=cortex-a78C+flagm+nofp16fml > > > Steps for this test . . . > > # poudriere pkgclean -A -jmain-CA78C > . . . > > # sysctl vfs.zfs.per_txg_dirty_frees_percent=5 > vfs.zfs.per_txg_dirty_frees_percent: 30 -> 5 > > # grep USE_TMPFS= /usr/local/etc/poudriere.conf > # EXAMPLE: USE_TMPFS="wrkdir data" > USE_TMPFS="data" > #USE_TMPFS=all > > # poudriere bulk -jmain-CA78C -w -f ~/origins/CA78C-origins.txt > . . . > > At 15 minutes into the build: > 46 ports in 1st 15 minutes. Load average stayed reasonable > for the configuration. > > At 30 minutes into the build: > 102 ports in 1st 30 minutes. Load average still reasonable > for the configuration. > > Looks good compared to before. > > > I've no clue what optimal would be for the context, but > > vfs.zfs.per_txg_dirty_frees_percent=5 > > is vastly better for the context than the default 30 was. > > Thanks. > > > I'm going to stop the test and do the conversion to the > U2 960GB Optane media in the USB3 adaptor and then > compare USE_TMPFS=data vs. USE_TMPFS=all --but using your > vfs.zfs.per_txg_dirty_frees_percent=5 assignment. > Took a while to actually get around to stopping the test. It got 186 of the ports built in the 1st hour. (A from scratch build, starting with building pkg.) I finally have started the U2 960GB Optane based tests, currently USE_TMPFS=data . The initial activity looks like it might build about as many ports as the earlier USE_TMPFS=all test (for different media, vfs.zfs.per_txg_dirty_frees_percent being 30). . . . waiting . . . It got 222 of the ports built in the 1st hour, again starting with pkg. That compares to 262 for the earlier USE_TMPFS=all test. None of these ports form the first hour are large, long running port builds, none using large scale amounts of storage space for its builder. (As I build things, rust for example, uses 17GiBytes+ of file system space, more than half of the size of the RAM in the Windows Dev Kit 2023 for USE_TEMPFS=all just for file system content.) Now for a USE_TMPFS=all build test with vfs.zfs.per_txg_dirty_frees_percent being 5 . I may try letting that run to completion. (The configuration has 118 GiBytes of swap for paging activity.) === Mark Millard marklmi at yahoo.com From nobody Mon May 1 21:21:26 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9GP407Mbz491sF for ; Mon, 1 May 2023 21:21:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9GP22ywxz3Qq7 for ; Mon, 1 May 2023 21:21:42 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="ioo/gDBX"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682976100; bh=zux7QH1t8qvzq6qDMuP24l22VAwqouhbdhExbhPJ4DU=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=ioo/gDBXuLGuu/bKRlJazmbiYoHrQ70/MU50OC7w3/EureLkZ9/jbHhoFScEBb9wXFLmX2hMw+sSbzPKJvWJ9SDWC3qhEKMfhbI5rC4h/VmS3ORg4U9+0hTMXF/95TWb043Q5fuhxsFgX6fQPvDFCkaOUo4HRE/6BH9K7GoAgoJ8Y53cneQ0PiTHaciPNaC5cQIIyLCE1WIfP7dfF02gKRlIGoTT2oevlYXLL8UOy0q9BAZwuuVgRJjmeedNefZ3Puz05I1xfMKTgmsvg3S+biGiOCbpoz6GK+2FRAV67X9qmAq+1zHeh1Ok73wOiRWPvtgPvgn0DefoCE4YrlhM/Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682976100; bh=wTY/7LVnvXTDGO5RN8EVAKQspxL2Hv+jD6GsNRvFEaM=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ZwnjuzUs8vA6SKxSaqSv6ONH+YjB3z9nljn7rFYmsR2trA5yNydLiSJLBcxyC7ujbYys5tx5kqXuxLpuddoixyffHDPrmSOceUkXzBPVsUuw/MFEpeZDoqcWvTr6JcLLv1T0xOXIg/4lrJ09L4GBd0s/2kst6SnclofNLaZuXl432aotV4YolmkGKE61S9UBWuZUMQmeR9s7T8MJt5lFwMhR4+k6JuzMwqguwC31d5IETH3YtKzQpnTtnEICXzc/LUulraTdpx4a/dVIs1R6gIkFnoA6WQkL3sdOGoxmY99eNeQDTZm5NZNKFPtaHzsT9WXxOd73EwOzREs35EAtCg== X-YMail-OSG: wQqzf84VM1lRmRofstmjdy0Q0U._o88s3sqEZ__MzeTFIOm1KbrkZXt4AFhtV78 toQNh8VTc75JEjc8bpSeSWOi4lN6dAvyp8PyZQA7YoVUgB4YHM.bTxXfaqjvfEb_mdxqCw_Hwst7 PeQfdaWomGy2e1H3k1JOFzDARqFfsLpWIc5kxSN8Tg18.Z5GSSbLGhc33XgiLfU.t2tZQBe8fz8H IjAsmH33sBSCVGnwNSKdGzQD9KjaEK0jytdlv4tbdI2J.J.1O6GR66SGC3PS6k0Uv.GBN.7IQpDT DoHl9hO0E9gy8BGS66aEGK1SOoy0u3HxZ64PRmJUNiqtQyIDi3dZp3ltvTPN0Xx9u9weyRwQv.ng xR3GJ_Crx5p_GRGcUpNMf6.8MUxIgtBqWsZKNeQJdFvaVezilFm6C52KoatfQOzsTEyIxdQ6mr_E btTumEaTKWiD7x4XlBSYBRwznF_5tDzgRLhoDzVLKFGs0n3XydlKAVo2AI9EiU85K36BVDBDMbT8 Nur8V5Jx1WSOq5rLruCodIqVH_LbQXbxLVxlfEKRBtjY2BDra_pgm8APFOwgsG1nchSYZfYZKd65 egRTFTgqAv7QKLgyfU4hrwsz5hKCSewoGSG0ZsYfkOPA3Fzjkc5Grox1NwkHN7j7G8rbXYCMZLVC Oh_RaNdZI6pCaIKjVxOAUwHNFxnmYmxQTPzrDTbs5VNZXK.y0RPfMfSirCPBGs89jn_CGUwlF_1n SgMeOLU3iQQbIPZ6e24pwBx2A21UnEvDBVMGd1aL8akrKsmIOP5Buy2uDbGqYj9Pv7LBYALUTzRp hPQionw.6nXR41XsJkiZFlbLQxoc0LYL1d0O6xyj3Ok74k2P_hDmMynGjdkC1D9Ae_VRWYOtuGik X7Jj00QNkebhWImVN0zDrFPnfv5ralyK755trDn9gYKBf7b7a_X70MeqFcEZtAzo9JwmcWnc1ngU LgtlEhdNANFbCbS4R4IpMrP6aW6shY5lUxHLm5j727CfQ4FMay5mPpNLVO0OLHN34HZuJyNUjisN 0zKCsiBqzimsLncUE6PuI9y.FZrHQzhpm3kVplgUO6kUrBVTUdm1Ro5OjvUKEI1pQ4DnQRRUg0b3 BcGRMgQ68CTLK1wFoM8NlPGZwZrP_duJcXE6XA_aoZLxlBnKGJlvVPiLUeqqPHVLkshLpnVc5eHh djlhc2.xkVc4pflhtnwcmW2_FoUZ_1WG3L.M7Tr_YETVk3SzCCLixueuivQ4r8PMJCx4W_FM4enV d8xTNWaipcF8k8nBHuBmmJ1XTxXcQgB9JJqZeYUeq9Nm9OpOa5uTELsjmaL4QrZeEVFfdZgcf.Vj 5z0_fHQrCw9AABmnY0NO7c0h.BJgIcu1NQrhZ2OEj3p7A.iTvlTJX2pTi.J.HaxHWf.R7HwJYv26 2zKLqv_0_WDJ.8jYDsCPBDFGgscH0LnJmprbdCDd0Krcmh37fs59a87VMYUQsU49yuzm7_LkEwmQ yDSemiBXenHs9a63Q4BbB3u6R2qxA7pFpcyaWaFUgLM8oVK8W3UqJs3aG30TEAwaf5WDXdA.xHRi iipktYZ5sGxuexc1vgFqrZmYAdH0YcNArCe.6UjhpD_rlssSK76gvF7R11Dcy_b_KX2WIBbk4LSd 1uzZVvPIQAVQADuUAnHJ.VD0daWxi.JD0B.wKAOISFdqLk6GsWxfQdgOMVgks7XvlPfWI8uFLGqJ SQa3ZTbMD5sp9DNtlndGQWqqRNsirHhdL8VT.g6q88gdacExcKF6YtKvma.JEbmxkFV_uROxgkYd i3yWuft03B_VtxoMj20HE3u_ZqJi.irm6zm_A9yXyBXgSGVSu56eAiacxS0S1mxrnsgTd8DUxXES q0N6GFBINYXh45CjSIZYH_m90zubpM08xS.K10_rHk0Nva5AJKTuymK42Tk276vMAjPC41lB9Z.S 85B3FWcpcQvQuXGJ6HKhoKJJmps_rOUAsUZ8rVnkLGd0uqx1UaS59cBKvqWdeG0zOjhITT9YsDwp CeOVxuqjh8AiG.6IoFEfBSfepQbDqu.2IK86GwAuOG0WTZIrr9aQyYtMgu5bGJCNBHV8r.9Ogv1O lEx0ndeRYl1s7KdvKeMjiM5qyfuXaQUNVIzsnI_XRFITKbkj9vSyWtsaNwF6bM9kOq3wBd2wNEfW U8PNZuLaBHT4ZaYLLcy5ORSyd41FOdQNoIWfUOtoffgZy3mkURO2vcWIdeo.wMGgse9lANZ9abIf U6DzoaKLcenJ_5Tf0XMB5nadCQ.h1U.Ve1K39nVw4jiSiF330cwwo.bKvtX9Y2xbZZwo4RndO_hV 1J4k- X-Sonic-MF: X-Sonic-ID: 8f881a25-07ff-4f4f-8d0b-a3b62dbac332 Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Mon, 1 May 2023 21:21:40 +0000 Received: by hermes--production-gq1-546798879c-l2qgj (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 23455dc20bc6a2c6fc14c38aa33e2c08; Mon, 01 May 2023 21:21:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes? From: Mark Millard In-Reply-To: <8F883286-D713-4632-9575-5813E885D125@yahoo.com> Date: Mon, 1 May 2023 14:21:26 -0700 Cc: FreeBSD Hackers , freebsd-arm Content-Transfer-Encoding: 7bit Message-Id: References: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> <02DC03AE-E082-4FB5-AA0D-396F64CC23CB@yahoo.com> <8F883286-D713-4632-9575-5813E885D125@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-1.70 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-0.202]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org] X-Rspamd-Queue-Id: 4Q9GP22ywxz3Qq7 X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N On May 1, 2023, at 12:57, Mark Millard wrote: > On Apr 30, 2023, at 18:57, Mark Millard wrote: > >> On Apr 30, 2023, at 17:44, Mateusz Guzik wrote: >> >>> can you redo zfs test with: >>> sysctl vfs.zfs.per_txg_dirty_frees_percent=5 >> >> Sure. >> >> Result summary: Seems to have avoided the sustained periods >> of low load average activity. Much better for the context. >> >> >> Context: Original ZFS USB3 media. World or Kernel in use >> had been built (non-debug style) using: >> >> -mcpu=cortex-a78C+flagm+nofp16fml >> >> >> Steps for this test . . . >> >> # poudriere pkgclean -A -jmain-CA78C >> . . . >> >> # sysctl vfs.zfs.per_txg_dirty_frees_percent=5 >> vfs.zfs.per_txg_dirty_frees_percent: 30 -> 5 >> >> # grep USE_TMPFS= /usr/local/etc/poudriere.conf >> # EXAMPLE: USE_TMPFS="wrkdir data" >> USE_TMPFS="data" >> #USE_TMPFS=all >> >> # poudriere bulk -jmain-CA78C -w -f ~/origins/CA78C-origins.txt >> . . . >> >> At 15 minutes into the build: >> 46 ports in 1st 15 minutes. Load average stayed reasonable >> for the configuration. >> >> At 30 minutes into the build: >> 102 ports in 1st 30 minutes. Load average still reasonable >> for the configuration. >> >> Looks good compared to before. >> >> >> I've no clue what optimal would be for the context, but >> >> vfs.zfs.per_txg_dirty_frees_percent=5 >> >> is vastly better for the context than the default 30 was. >> >> Thanks. >> >> >> I'm going to stop the test and do the conversion to the >> U2 960GB Optane media in the USB3 adaptor and then >> compare USE_TMPFS=data vs. USE_TMPFS=all --but using your >> vfs.zfs.per_txg_dirty_frees_percent=5 assignment. >> > > Took a while to actually get around to stopping the test. > It got 186 of the ports built in the 1st hour. (A from > scratch build, starting with building pkg.) > > I finally have started the U2 960GB Optane based tests, > currently USE_TMPFS=data . The initial activity looks > like it might build about as many ports as the earlier > USE_TMPFS=all test (for different media, > vfs.zfs.per_txg_dirty_frees_percent being 30). > > . . . waiting . . . > > It got 222 of the ports built in the 1st hour, again > starting with pkg. That compares to 262 for the earlier > USE_TMPFS=all test. > > None of these ports form the first hour are large, long > running port builds, none using large scale amounts of > storage space for its builder. (As I build things, rust > for example, uses 17GiBytes+ of file system space, more > than half of the size of the RAM in the Windows Dev Kit > 2023 for USE_TEMPFS=all just for file system content.) > > Now for a USE_TMPFS=all build test with > vfs.zfs.per_txg_dirty_frees_percent being 5 . I may try > letting that run to completion. (The configuration has > 118 GiBytes of swap for paging activity.) USE_TMPFS=all with the U2 960GB Optane USB3 media: It built 270 ports in the first hour. I'm going to let it run to completion (assuming that it does in under, say, 16 hours total for the overall 480 ports). I'll see what it is like when llvm15, llvm16, rust, and possibly gcc13 or such will likely build in overlapping time frames. (I do not build LTO style and, for gcc* do do not even use a bootstrap style. My usage of compiler toolchains between toolchain builds is minor compared to the toolchain builds themselves for the most part and my time preferences for builds are fairly strongly biased to less wait time.) === Mark Millard marklmi at yahoo.com From nobody Tue May 2 06:55:17 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9W726fRDz48bjT for ; Tue, 2 May 2023 06:55:26 +0000 (UTC) (envelope-from freebsd-arm@c0decafe.de) Received: from mail.c0decafe.de (mail.c0decafe.de [IPv6:2a01:4f8:222:100a::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.c0decafe.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9W714jD4z4VHJ for ; Tue, 2 May 2023 06:55:25 +0000 (UTC) (envelope-from freebsd-arm@c0decafe.de) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=c0decafe.de header.s=c0decafe.de header.b="znG/ZAGI"; spf=pass (mx1.freebsd.org: domain of freebsd-arm@c0decafe.de designates 2a01:4f8:222:100a::2 as permitted sender) smtp.mailfrom=freebsd-arm@c0decafe.de; dmarc=pass (policy=none) header.from=c0decafe.de Received: from [172.17.30.254] (unknown [172.17.30.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.c0decafe.de (Postfix) with ESMTPSA id 25DA5C1491 for ; Tue, 2 May 2023 08:55:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=c0decafe.de; s=c0decafe.de; t=1683010518; bh=GJ6j3aFp/4SdKtGot51ydGLKsrhkCr2JN4MKoxNcPfs=; h=Date:To:From:Subject; b=znG/ZAGIuyzAhb8z3guXh9KHJqKjA0Ox35jAD3cXC87haMaEzlky/tK5HNRhsTqVL wo7SK5JWIp9nPNKIBJ5hxrQkkQzj4ckB3UIBUjhPe7Hg2Tif83fH6bsTGcLQhS/kqX qqesCF4WtIuF6SPa73l5a9+Ob3N8BAS5CN5+/SZE= Message-ID: <13b3f66b-45ab-ac63-f84a-419d5b376fa8@c0decafe.de> Date: Tue, 2 May 2023 08:55:17 +0200 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 To: freebsd-arm@FreeBSD.org Content-Language: en-US From: Daniel Subject: pid xxxx (progname), jid 0, uid 0, was killed: failed to reclaim memory Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[c0decafe.de,none]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[c0decafe.de:s=c0decafe.de]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[c0decafe.de:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; MLMMJ_DEST(0.00)[freebsd-arm@FreeBSD.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4Q9W714jD4z4VHJ X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Hi List, I noticed that on my aarch64 boards I recently get a lot of 'pid xxxx (progname), jid 0, uid 0, was killed: failed to reclaim memory' messages in syslog. This happens on a rockpro64 as well as a raspberry pi 3b, both running 13.2. None of the boards is near its memory capacity (more like less than 50% used). It started on the rockpro64 first, were i did a bit of fiddling, e.g. en/disable swap, replace zfs with ufs, etc. nothing helped in the end. I thought the board might be defective but now i start seeing the same thing on the raspi as well. Any ideas what this could be or how to debug this further? Thanks & best regards From nobody Tue May 2 07:25:21 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9Wnv103bz48d87 for ; Tue, 2 May 2023 07:25:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-54.consmr.mail.gq1.yahoo.com (sonic307-54.consmr.mail.gq1.yahoo.com [98.137.64.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9Wnt4Dzqz4XhT for ; Tue, 2 May 2023 07:25:38 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1683012335; bh=CN8wcZ/sNIHkcesWWuwQT00RWCKua7L8HAXtZbZzsW8=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=qUgvxPrMSbCdbLRfufEdIxer/sIsCssQPKW4U7a7/pkgHyJJRpwx7odib6mxzT9vyv/nzMLG5bea8aZSJnyWdjck94jjnlQRwAn35tva8ImfWesS78NGLWPQSyr0YoprAmL9m/2GxKhTq3whPJPIOuvYbbcd/AMCKSdGHqYylhFYZn4cb2WG0zm4dKVKSrsx3u/ptOelb0r6jjI6HzNPF88ZIjtd/1co82pKrW8JJ/XOkoFa5H1ORLAsOUmStJd1w5KSOiH7Pjxw8lYUCgZBeWQftCl6ychV6+z8+L4+TqFaa+bCLGcKCkQz917ilaVlCpesOHyCNZH6bxiRdYVzdw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1683012335; bh=EQJQ8d92bPcKoUQVmVJAC48+CuooMk7pvXlTJUx9dGn=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=S4aPheFK7g2BPcgX86nCP2wckq1Lqy37HfB6aVjKMm79H2pyr4RNIxb8XtdE0akkchsX/JrGSjgRUj8vfA65XVp05S21seKqlB1m4gyEZndnXL4+SzybYzzSBTLI6pURjFj68HtxdCx6ahOQ2fFvButwA9IiZ8jGgNaew3dharERQOqhkqiIW7ojp4V5cCFYiFTaFFJMS8tiZAm3D7RcvzQHPR+BYCCqMspJC36y78lpGO0VHRzmjEDgdKBe63oU/LlzEnFWYrxHxavbzj/RK4ehNTHSE5k/U60CKyGR2OxaGuZe9hk+mRLhPue2yRd0cnF1Q2xeTARdiPOaaip+0g== X-YMail-OSG: tBQ4MaUVM1nWDyajyLqv.iAJslDv2BUZBYpkqxfaDaiRofW2vsNK.fqfux4DiHB xiuwxiqDPt.kBvMHXCBh3HMsE_LgPQsYJkwvtD1bpOESqTN1mHRzppCcUn16JBciQ8dgVRFrBPtB 81kJb9DPwW48ZqpzT4QGIDs6YpU.GuSuNpReo8DWc4gqmaXs34pCEVGAqStOcSmytjAirgSJqykl cG4sjXp_NV3STOK4zLWRKWYNNsJu_HRcQM71AvUlgDnWjIbaGPoJVAf.NSr00xRaBQ.2sRTYk_rz QtJiuNHwF8owMAuYmKDp.jGIDfAOnbYz5mbnMSFrQA_9D6fmA1DajzxtQcFZctpzg3wWOAy6FSix za.zT6yOYweMJRj1Qx4y0.ofGcJ0B_PRCx_c.kRmaXAdjgv4czS_ArXIo1s2ea1HfdGPmNrSuKxA ZtNrF2851ahuT0xY6xWLEkbeMRSemTk5U_CEPmGUe1y39FWd6LmShv8HrmWCRj5GzZJCmGbWUkpW ufy2W6N_Or_fIpo7AWL3nzWK2Mfl6A6nZjlP2P2pvUNd3VJZRORziyrpxnsjo0WkqTm6jLX9WfTj z0smLWPrbL51nqyTSSM75UWDXymNSwUZO.D6gapoQStMDEvhmP3sSPSmSxvu2FupxuUkNdp4Ttyn 30Iflea8Vogrjf2LQYfUA5BDLuxDjnN6Ef6dCwyALCkDeXYQgcorWP2kZeMJMPEJ44qlnNi6aMXT .6pSyYTFbhUYo8Y1kDeIHoVC3hbpoF_UMhMFbEKRwmM8gjo_iy7FtPpcSujoKdmO2gw9D6ozsyN8 SpHlk0dwcmnMUfXu1uNjibAZiDHwK8uHTNoHTYZhdEpayrnjQ0IFuJByeV9xZDICiAnlh39ExoML gk3ItNcJxyETbEJ6bUdTAYsorG7PPus7gwrsMPpA9Yl2m94yTU9UE3pOXtAGhycomVdZV56Msr.j bjCrVXGVZH2j6wE7J17neQdOXs3C6rRhs09u9sQu86h6MIwcZ9e0aVD7pBxpxtqnWkJ1dNloCuIu bTzdLLm1ZLavGb5rFy5v1QRXlAtF4QjkHNGSx7BfWYdQERusH_sfpkaQCb1oIGn4clGdMVXyam6W .raFoCZZNBix1jIIZz0caWn5QTWkmlGW93rGWKIza_9cHZ0NpMTqwafoqu9AQjDnq5ozUDYgMD4y E37TxunFwqRYVW_pJSqPEe5MjrenVJPf04sapltt.oQXIjzc2IQyjpcT8DzEdIHm1.8evZEc4jwA XAjBX184Sr.s3t0HQWfho0pr7MoYdW75oR5RCR1z_6b9jkTy5aD6JTszdpqLvBPOKyWBOIOmd87E cab8S72hhFzjb8cByJ1Wq.6VyCCflgODwCIrFJf5kAT34NTogQJ1fDdsqiLjFlEEShJ9TN6phbLc 3A0TvpPMTPSt4TC4ZkbbB3A7fp4x0dVwWblqp_RqEk7I834B2PaVz47.yXmeTVSb5_9.7zp1S5Ka e0vlbB5kXV_NsdYhUfpbOreT6t.D.4tS0_dyAZVJ3tKCJWYBvQyhgVl6VZl3zOutZlC.J3dy.KIv IYASL6_tXGMqEIm5OeQAQ7M4ymWNQgvHaudIQIBLXgsQ9V_TaIF6q4eSSeyf4r6w.ezxPzcUCar_ 5k9.zYTQnHMAONn1p_VHkyJvV59nfPl2isV7SeJv3o5f23HUXGnCh.hKfDZ6Gxbk7YMZmw7AcEJo Vw.uI7Qgn0B8hvl1oroRJSG5YIH8gpyAGD_.ju1YXfihqdkBwhOtQz1m_KUFMHpjRMw0RLrj0bF9 93fTQHIrt1LbZTgQF46fZBbKvqlAUt_vrem3FnrG5HF4BX.BEgFUfupKlcbYqFIGgU7uBPONU7Lq WtMH52f5hIzP.Jrpok6UNMKNLGwtah5cdxmQnxZRZdo2vIZDFNzFumqCvEMmnLIVQjsX4kKl9ZLZ Tbh834LOBqs2Ydiuu.ngCibvEJe_Y5eXGr97YwOBtzNDvJVpejp8p61VYOxfi8pZ0dU6I.2gh7oU HnaSh6HdxYnKWaQK1UcXt.2_aEWxPRsuNiCBeF126suingJJOg4Mm3aX7HxYBmuIXLjxMZhs7aZh LZT4pCEvy8RdbklwHS8fzIZyZSxNARTOnH7QCjGODtvsBcmA4VvRf0..VBhi4Jj4opo0YNGVsPz0 I35usEyddjbhvi.2iDwgOSXtlGw7nMLyzzGeBxnsNMQmrBYqDhrLIhiFnU0BUV84X9ZFySxsUVpw QnhBVRvN6Sk8Jadb3clgXTh0HhPQRzuELrx2NuGUpKpBESa79BAcX70G2WYiSOdTJc3OGD.JyKpo pG7g- X-Sonic-MF: X-Sonic-ID: e6a79966-e981-45d7-bff2-dc90294b2ab9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Tue, 2 May 2023 07:25:35 +0000 Received: by hermes--production-gq1-546798879c-l2qgj (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID d712bd5b766f49f099bceab17e826b06; Tue, 02 May 2023 07:25:32 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: pid xxxx (progname), jid 0, uid 0, was killed: failed to reclaim memory From: Mark Millard In-Reply-To: <13b3f66b-45ab-ac63-f84a-419d5b376fa8@c0decafe.de> Date: Tue, 2 May 2023 00:25:21 -0700 Cc: "freebsd-arm@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <13b3f66b-45ab-ac63-f84a-419d5b376fa8@c0decafe.de> To: Daniel X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4Q9Wnt4Dzqz4XhT X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On May 1, 2023, at 23:55, Daniel wrote: > I noticed that on my aarch64 boards I recently get a lot of 'pid xxxx = (progname), jid 0, uid 0, was killed: failed to reclaim memory' messages = in syslog. >=20 > This happens on a rockpro64 as well as a raspberry pi 3b, both running = 13.2. None of the boards is near its memory capacity (more like less = than 50% used). Are you counting RAM+SWAP as "memory capacity"? Just RAM? The message is strictly about maintaining a certain amount of free RAM. It turns out swap does not automatically avoid the issue for all contexts. QUOTE from back in 2022-Dec for another context with the problem: (I've not reworked the wording to your context but the points will probably be clear anyway.) This is the FreeBSD kernel complaining about the configuration not well matching the RPi3B+ workload. In essence, it was unable to achieve it targeted minimum amount of free RAM in the sort of time frame (really: effort) it is configured for. Depending on what you do, the FreeBSD defaults do not work well for 1 GiByte of RAM. Swap space alone is insufficient because FreeBSD does not swap out processes that stay runnable. Just one process that stays runnable using a working set that is as large as the fits RAM for overall operation will lead to such "failed to reclaim memory" kills. But, if you are getting this, you will almost certainly need a non-trivial swap space anyway. I have a starting point to recommend, configuring some settings. As I've no detailed clue for your context, I'll just provide the general description. A) I recommend a swap space something like shown in the below (from gpart show output): =3D> 40 1953525088 da0 GPT (932G) 40 532480 1 efi (260M) 532520 2008 - free - (1.0M) 534528 7340032 2 freebsd-swap (3.5G) . . . 67643392 1740636160 5 freebsd-ufs (830G) 1808279552 145245576 - free - (69G) This size (3.5 GiBytes or so) is somewhat below were FreeBSD starts to complain about potential mistuning from a large swap space, given the 1 GiByte of RAM. (I boot the same boot media on a variety of machines and have other swap partitions to match up with RAM sizes. But I omitted showing them.) It is easy to have things like buildworld or building ports end up with individual processes that are temporarily bigger than the 1 GiByte RAM. Getting multiple cores going can also lead to not fitting and needing to page. I'll note that I normally use USB3 NVMe media that also works with USB2 ports. My alternate is USB3 SSD media that works with USB2 ports. I avoid spinning rust and microsd cards. This limits what I can usefully comment on for some aspects of configuration related to the alternatives. B) /boot/loader.conf content: # # Delay when persistent low free RAM leads to # Out Of Memory killing of processes: vm.pageout_oom_seq=3D120 # # For plunty of swap/paging space (will not # run out), avoid pageout delays leading to # Out Of Memory killing of processes: vm.pfault_oom_attempts=3D-1 # # For possibly insufficient swap/paging space # (might run out), increase the pageout delay # that leads to Out Of Memory killing of # processes (showing defaults at the time): #vm.pfault_oom_attempts=3D 3 #vm.pfault_oom_wait=3D 10 # (The multiplication is the total but there # are other potential tradoffs in the factors # multiplied, even for nearly the same total.) If use of vm.pfault_oom_attempts=3D-1 is going to be inappropriate, I do not have background with figuring out a good combination of settings for vm.pfault_oom_attempts and vm.pfault_oom_wait . I'll note that vm.pageout_oom_seq is not a time --more like how many insufficient tries to reclaim RAM happen in sequence before an OOM kill is started (effort). 120 is 10 times the default. While nothing disables such criteria, larger figures can be used if needed. (I've never had to but others have.) C) /etc/sysctl.conf content: # # Together this pair avoids swapping out the process kernel stacks. # This avoids one way for processes for interacting with the system # from ending up being hung-up. vm.swap_enabled=3D0 vm.swap_idle_enabled=3D0 D) I strictly avoid having tmpfs complete for RAM in this kind of context. tmpfs use just makes avoiding "failed to reclaim memory" more difficult to avoid. (As various folks have run into despite having vastly more RAM than an RPi3B+.) So my /usr/local/etc/poudriere.conf has: USE_TMPFS=3Dno There are examples, like building rust, where anything but "no" or "data" leads to huge 10 GiByte+ tmpfs spaces for poudriere's build activity. Not a good match to an RPi3B+ . That is it for the recommendations of a starting point configuration. With such measures, I've been able to have poudriere with -j4 but also using ALLOW_MAKE_JOBS=3D without using the likes of MAKE_JOBS_NUMBER limiting it. (So the load average could be around 16 a fair amount of the time but still not get "failed to reclaim memory" kills.) Note: I'm not claiming above that -j4 is the best setting to use from, say, a elapsed time point of view for my poudriere bulk activity. END QUOTE > It started on the rockpro64 first, were i did a bit of fiddling, e.g. = en/disable swap, replace zfs with ufs, etc. nothing helped in the end. I = thought the board might be defective but now i start seeing the same = thing on the raspi as well. >=20 > Any ideas what this could be or how to debug this further? >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue May 2 08:35:29 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9YLX6gHzz48hT1 for ; Tue, 2 May 2023 08:35:32 +0000 (UTC) (envelope-from freebsd-arm@c0decafe.de) Received: from mail.c0decafe.de (mail.c0decafe.de [IPv6:2a01:4f8:222:100a::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.c0decafe.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9YLX2hM8z3F4X for ; Tue, 2 May 2023 08:35:32 +0000 (UTC) (envelope-from freebsd-arm@c0decafe.de) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=c0decafe.de header.s=c0decafe.de header.b=eFBMH60j; spf=pass (mx1.freebsd.org: domain of freebsd-arm@c0decafe.de designates 2a01:4f8:222:100a::2 as permitted sender) smtp.mailfrom=freebsd-arm@c0decafe.de; dmarc=pass (policy=none) header.from=c0decafe.de Received: from [172.17.30.254] (unknown [172.17.30.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.c0decafe.de (Postfix) with ESMTPSA id 8F021C1B71 for ; Tue, 2 May 2023 10:35:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=c0decafe.de; s=c0decafe.de; t=1683016529; bh=XXkMT/R4zcrNioJ3aRpICsQIX5Ft7H61HRx8+HkQnyE=; h=Date:Subject:References:To:From:In-Reply-To; b=eFBMH60jhQv+sxEaH3bEuzhA+3rhTzVuukl+dKYFRjopPNaBTfswvIpIvD6N1kywh uNbhklBbV/J5FQgBjjJf/jEpSitn9cO15I243vZad2vuCWQVwCtJjlyseHLepdzAxT ryWo57yASqBzVDbMPqLv9PLYEc0LkY1Lsb9RVy04= Message-ID: <2d862f69-02f0-7356-3a75-d410b4a5a961@c0decafe.de> Date: Tue, 2 May 2023 10:35:29 +0200 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: pid xxxx (progname), jid 0, uid 0, was killed: failed to reclaim memory Content-Language: en-US References: To: freebsd-arm@FreeBSD.org From: Daniel In-Reply-To: X-Forwarded-Message-Id: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.992]; DMARC_POLICY_ALLOW(-0.50)[c0decafe.de,none]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_ALLOW(-0.20)[c0decafe.de:s=c0decafe.de]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[c0decafe.de:+]; MLMMJ_DEST(0.00)[freebsd-arm@FreeBSD.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4Q9YLX2hM8z3F4X X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On 5/2/23 09:25, Mark Millard wrote: > On May 1, 2023, at 23:55, Daniel wrote: > >> I noticed that on my aarch64 boards I recently get a lot of 'pid xxxx >> (progname), jid 0, uid 0, was killed: failed to reclaim memory' >> messages in syslog. >> >> This happens on a rockpro64 as well as a raspberry pi 3b, both >> running 13.2. None of the boards is near its memory capacity (more >> like less than 50% used). > Are you counting RAM+SWAP as "memory capacity"? Just RAM? with memory capacity i mean just RAM. Take this vmstat from the pi as an example: # vmstat  procs    memory    page                      disks faults cpu  r  b  w  avm  fre  flt  re  pi  po   fr   sr mm0 md0   in   sy cs us sy id  0  0  0 864M 511M 2.2K   0  15   0 2.4K   72   0   0 19730 1.8K 6.1K  4  3 93 yes, there is a memdisk (as unionfs overlay for the way too frequently dying and now ro sdcard) but its barley used: # df -h Filesystem         Size    Used   Avail Capacity  Mounted on [...] /dev/md0           496M    4.6M    451M     1%    /rwroot still i see processes being killed with the above message. on the rockpro64 i had an fairly huge swap (4G) on an nvme that never really got filled (~500megs maybe). I'll try your suggestions below, thanks! Do you know of any recent changes to memory mgmt, oom conditions that might trigger this? I've been running this setup (slapd, radiusd, smtpd) for quite some time on the pi now without any problems, before going to 13.2 Thanks! > The message is strictly about maintaining a certain amount of free > RAM. It turns out swap does not automatically avoid the issue for > all contexts. > > QUOTE from back in 2022-Dec for another context with the problem: > (I've not reworked the wording to your context but the points > will probably be clear anyway.) > > This is the FreeBSD kernel complaining about the configuration > not well matching the RPi3B+ workload. In essence, it was unable > to achieve it targeted minimum amount of free RAM in the sort of > time frame (really: effort) it is configured for. Depending on > what you do, the FreeBSD defaults do not work well for 1 GiByte > of RAM. Swap space alone is insufficient because FreeBSD does > not swap out processes that stay runnable. Just one process that > stays runnable using a working set that is as large as the fits > RAM for overall operation will lead to such "failed to reclaim > memory" kills. > > But, if you are getting this, you will almost certainly need > a non-trivial swap space anyway. > > I have a starting point to recommend, configuring some > settings. As I've no detailed clue for your context, > I'll just provide the general description. > > > A) I recommend a swap space something like shown in > the below (from gpart show output): > > => 40 1953525088 da0 GPT (932G) > 40 532480 1 efi (260M) > 532520 2008 - free - (1.0M) > 534528 7340032 2 freebsd-swap (3.5G) > . . . > 67643392 1740636160 5 freebsd-ufs (830G) > 1808279552 145245576 - free - (69G) > > This size (3.5 GiBytes or so) is somewhat below > were FreeBSD starts to complain about potential > mistuning from a large swap space, given the 1 > GiByte of RAM. (I boot the same boot media on a > variety of machines and have other swap partitions > to match up with RAM sizes. But I omitted showing > them.) > > It is easy to have things like buildworld or > building ports end up with individual processes > that are temporarily bigger than the 1 GiByte RAM. > Getting multiple cores going can also lead to > not fitting and needing to page. > > I'll note that I normally use USB3 NVMe media that > also works with USB2 ports. My alternate is USB3 > SSD media that works with USB2 ports. I avoid > spinning rust and microsd cards. This limits what > I can usefully comment on for some aspects of > configuration related to the alternatives. > > > B) /boot/loader.conf content: > > # > # Delay when persistent low free RAM leads to > # Out Of Memory killing of processes: > vm.pageout_oom_seq=120 > # > # For plunty of swap/paging space (will not > # run out), avoid pageout delays leading to > # Out Of Memory killing of processes: > vm.pfault_oom_attempts=-1 > # > # For possibly insufficient swap/paging space > # (might run out), increase the pageout delay > # that leads to Out Of Memory killing of > # processes (showing defaults at the time): > #vm.pfault_oom_attempts= 3 > #vm.pfault_oom_wait= 10 > # (The multiplication is the total but there > # are other potential tradoffs in the factors > # multiplied, even for nearly the same total.) > > If use of vm.pfault_oom_attempts=-1 is going to > be inappropriate, I do not have background with > figuring out a good combination of settings for > vm.pfault_oom_attempts and vm.pfault_oom_wait . > > I'll note that vm.pageout_oom_seq is not a time > --more like how many insufficient tries to > reclaim RAM happen in sequence before an OOM > kill is started (effort). 120 is 10 times the > default. While nothing disables such criteria, > larger figures can be used if needed. (I've > never had to but others have.) > > > C) /etc/sysctl.conf content: > > # > # Together this pair avoids swapping out the process kernel stacks. > # This avoids one way for processes for interacting with the system > # from ending up being hung-up. > vm.swap_enabled=0 > vm.swap_idle_enabled=0 > > > D) I strictly avoid having tmpfs complete for RAM > in this kind of context. tmpfs use just makes > avoiding "failed to reclaim memory" more difficult > to avoid. (As various folks have run into despite > having vastly more RAM than an RPi3B+.) So my > /usr/local/etc/poudriere.conf has: > > USE_TMPFS=no > > There are examples, like building rust, where > anything but "no" or "data" leads to huge 10 > GiByte+ tmpfs spaces for poudriere's build > activity. Not a good match to an RPi3B+ . > > > That is it for the recommendations of a starting > point configuration. > > With such measures, I've been able to have poudriere > with -j4 but also using ALLOW_MAKE_JOBS= without using > the likes of MAKE_JOBS_NUMBER limiting it. (So the > load average could be around 16 a fair amount of the > time but still not get "failed to reclaim memory" > kills.) > > Note: I'm not claiming above that -j4 is the best > setting to use from, say, a elapsed time point of > view for my poudriere bulk activity. > > END QUOTE > >> It started on the rockpro64 first, were i did a bit of fiddling, e.g. >> en/disable swap, replace zfs with ufs, etc. nothing helped in the >> end. I thought the board might be defective but now i start seeing >> the same thing on the raspi as well. >> >> Any ideas what this could be or how to debug this further? >> > > > > === > Mark Millard > marklmi at yahoo.com > From nobody Tue May 2 10:20:03 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9bgT6MW2z48nTJ for ; Tue, 2 May 2023 10:20:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-23.consmr.mail.gq1.yahoo.com (sonic304-23.consmr.mail.gq1.yahoo.com [98.137.68.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9bgT3Mhbz3PkX for ; Tue, 2 May 2023 10:20:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1683022819; bh=BBEJCJ6v4vURwPvre40ZqZxjmk5TcliON1bRJ2fh1UA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=dV2NAygXenQ1biYqlAHi/ciyoqckmUbW+P6v45ClyA7YIzHvhXhJdSnEhTYOCLc53Csnr3k/uat5EpVDMTWzzE6u/x0oldHRzMh2mDGPxdXh9BuEilFjmPfn/lJn07E+OVUQIuJsYRNPV0p08XspSjiyGcZVmP144NRTbMwkqAd1CesIaAsPktgf3RAgEH67o4bRf9FQw1y9naj9U9F6T5GLaLBXjxt512ZtnNONxZ5uLuD01KVG/SNSvhKIpQIfYQoQBtDlsMtzVSWvWhNelqfkecxQDfEsFOb0kSUiijgiu9me9owd67cYKX+8sI50yu6MGBmfbPUSu6unhRR+pQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1683022819; bh=sVoo2Wkx80m84tRUpiBVcX2B5sp4al+CjSE5k4/9n1I=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=uBuSwv8/8kpJc0fgbCHjyy8QYsyEBadykEYWN/6c0krdPz8HNy0oA8gAu21k2juvuH5GlReRvDbj2o9PIc83e8gucuzDbVY/kK1o2zdvHJm5FgldL8PIvvn99SUyeuKjiQZtS31y6Upk1mlKK3YHkJNn0ky0S7GjzHdY1rRabHLSimiqu3SXSO9eO6mbExYC1QVpuijd1qwwicZ8J4l/xr7Wonomw4aKEJm5JViAcV4UHCUXpDrwR8UOPzgkVr8RiwdNMWxjmtc2pmq0QOxgKnH8D1hjZ4tczyYDWjkq4evEjDlnfLA/d+qr4+WNJGa61xINAW+SdsEiY0j5LQe3kQ== X-YMail-OSG: XCkY7P0VM1n4cWZQF6rktSbnxE2gF6SbPhyi6TXKsE2hudXJiCMS7O0Mi7Wkndb xZMeYF.AluH72GUnVIzu1HGJ1DraPTi08VyopxA7tVH85k1jZelEe.SUCVlqLjPcls_hDjxQmBVv Ateju5_.LLI78TbXuWKHJbmXd09s4FA0HfJwOBMAnO5kxw_AmWTJQc7xk4nfDuazNLtLqzlr5vZy ejwRBPXjBpQWpn2JUJR0VwgTsM8zQjbXTuoSIJSrV.GC.R4FmU9s2crGt8.WkTjlrPTgmAzyKFoO oAceRLnmmGuUyhhTX2.faQAQP21LyS3Eu7wDsq9xspP18kDAOzw6lMCFibcTkqMCxC2E95Xc8Pun S568prl74JRbwl1w2vl8kT.Rc3fXj8NcFhnRB65VZSNsGCeSmf8sIjzvm.m4HlS_HtfcPc._w6Z2 Q8fdJ7S692E6JVYTN.32kpfOPPHdM38RlovLXFW1WpkJCv0WtWpkU.ZBYmmFBJpGIRVtWt0O4yPk z3uLwmqh3A39it.P90uOgwJxs3ehmzC5QfrcQmxJsLVKm4JCLwmfGmLWIwTB5MKu45iPTVeoiOyZ lzZS8I2JHbXy3Bb_WNTKZXezquwIRGHzdp9D.esKfKcwqAAQ0NCthNsCcPSS.pu7CVoNv.LHCwjq cj5jobOuUZOyqWCYTY76y.ABCga64YOKBpO_iIzT5m3jsooBfYnZ97LMqT.EZ4nOorKP9CFq1KUS qo5Ifyc6RQlSgAFYOf5DrnOMpFh7cCvIorT1Wm6of0T1PmRzncGN18HBCOAWwZTv6YYZcji7GBsH nuu1YHt1sOJAIwx7DJuV.AWfshAdXphYnLG8zU8ABRul7DTOrktGp5yn1f36DMPvjgaF3Xi9MNXf AWo4XtZu7QW6uBraghfjaWjusUw.pAoEbXdSnAeXS79r1qBhDX_FixHXgm1nLjSkt8hyBHUyMaY5 O5ULs1ADBN0S5aFTXKrjvhwK3zjGLWKKtVc_3EHmOunj0ne1.kTmcJTXNGkR2N7S941llgAHunWL mSwnPw8Nik3LJ2ov72jnn8YUEeoVC83DSTYoejKnQV4gzH6uaeMMMu1ABKG8vjgFyOvJkIQG3XJ1 Gjn5QztrIIHzlE26yINxrDSAjUqs8eacgk0e2YuOGy4isK5AVdJqWiKOiIsOURnssCZ_SZG8g2iq 43QZs86_Rrao_hdSXDH2igsVxSS2SYOMjMicjBwt5y2F85v8Y.vdzyJGtTx5UIEWRTk54S6P9jv6 ADTXNMSEsQvnbecXhW7MTEzcVfjL7DOcqZnxnBf9brlDt632K1xnroKMk3CoxXs.RFyhblVOwHOn mKN0zp_wtefF1B_q29nwjUMBFQmPRM8UQUg0rVJHcxSRZhPGSCgyKyRpa6T.Hz6YBbxuqZPy0eCl XhTvMCR2Bwq7C0eNCC8IS_Q5pg3K8PvEnweBsVPb_UkV_Xb8CBTkYeZelvL7Gbaa3zGxaTakKW6e _jIQKuP51ujhpPCq5DKfSrs3Y9FbeSit0JCZp8r16fDPXt3SooG3cnhcat_vkFtx24qavnoA0w.j Kxq1h7GJ.Om7WXoVM1AuYaMPTQ0d3pYG.yhIotA3rGjoc1FTwDJduKyULGh7dtMrIQ3XeNczfv3O _crS6FhXA6g2yA8s5Vxbi0tCbRqsbdZJptm_H0l8DvOr0OBhpPuEFdvXdLwFEVzewIQfuvFc03XL 9ahE3cA0pYx68zo7LCybawe1lXJlah0fmd86PysNPx3ev51yaE9NNGRGKSKeQEz3GEJ_cgVaizhN mHUg9Mz097OKeejmXIHmMUE.OZ3lH4nNeOQAVIHbf7kCjHBdmlISmXDnx18BJZSyoN9orFGqt.Fp LcYEsNnA7euo9OsYMLgkZ2kdj1jscndRa_DpBCA4rWFc9e40zpGKOJHmCIgp3nkYVJBmUfYVuo59 FdE5bsfyro2WHzJq5vD63Brl61oDSVzPUsS2GWz5jE.lS3Z9Hbg2.DF.ArOLfQ.uV_7Ff7tp4_61 JgGgUXvwLtFWT55D_VEubOrlMZJX3F3i5dR_9RkXVdsFHb2GfQVK1vEmjL2_OLVB2BUyk5bZku0k p5brDycWsQHMDd61et68H555HGhNr9nwO0_q.caVbUojpIrwgKjvZ5IIwaibB.d5Gn5.Y6AcIYiQ wYCzluD.VZJ3ypcCQeIQJH3YfsAo5_ZQcUmTxfvwpukFTkfHf_S9GGzYR529Y2.naE0FIY90rIll Mj2_q_I_Bw6qG5pXLPKCrIRJY20ITxVIpcdy1U1cXThUGv2sT_126nKK_CnlTn7L8f1d9158VQC. iHw-- X-Sonic-MF: X-Sonic-ID: 57b05a52-eda2-400a-80aa-c1d52d6d93f3 Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Tue, 2 May 2023 10:20:19 +0000 Received: by hermes--production-bf1-5f9df5c5c4-qlh82 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 1a60e3a5694e2f07320691134634f8a4; Tue, 02 May 2023 10:20:15 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: pid xxxx (progname), jid 0, uid 0, was killed: failed to reclaim memory From: Mark Millard In-Reply-To: <2d862f69-02f0-7356-3a75-d410b4a5a961@c0decafe.de> Date: Tue, 2 May 2023 03:20:03 -0700 Cc: "freebsd-arm@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <58DC4BAF-7FB2-4216-8F38-6D26305ABD78@yahoo.com> References: <2d862f69-02f0-7356-3a75-d410b4a5a961@c0decafe.de> To: Daniel X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4Q9bgT3Mhbz3PkX X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On May 2, 2023, at 01:35, Daniel wrote: > On 5/2/23 09:25, Mark Millard wrote: >=20 >> On May 1, 2023, at 23:55, Daniel wrote: >>=20 >>> I noticed that on my aarch64 boards I recently get a lot of 'pid = xxxx (progname), jid 0, uid 0, was killed: failed to reclaim memory' = messages in syslog. >>>=20 >>> This happens on a rockpro64 as well as a raspberry pi 3b, both = running 13.2. None of the boards is near its memory capacity (more like = less than 50% used). >> Are you counting RAM+SWAP as "memory capacity"? Just RAM? >=20 > with memory capacity i mean just RAM. Take this vmstat from the pi as = an example: >=20 >=20 > # vmstat > procs memory page disks faults cpu > r b w avm fre flt re pi po fr sr mm0 md0 in sy cs us = sy id > 0 0 0 864M 511M 2.2K 0 15 0 2.4K 72 0 0 19730 1.8K 6.1K = 4 3 93 When was that command done? : A) Somewhat or just before the process was killed? B) After the notice was displayed about the kill (or, at least, after the process was killed)? If (B), it is too late to see the memory usage conditions that lead to the kill: RAM already freed by the kill. One needs to be monitoring the memory usage pattern/sequence that leads up to the kill. If it is too early, the conditions that leads to the kill need not have happend yet. It may be more reliable to get an idea by monitoring free memory over time via, say, top, spanning from somewhat before the problem occurs to after the problem. "systat -vmstat" is another display that can be used for monitoring. A question can be: ZFS in use vs. not? > yes, there is a memdisk (as unionfs overlay for the way too frequently = dying and now ro sdcard) but its barley used: >=20 >=20 > # df -h > Filesystem Size Used Avail Capacity Mounted on > [...] > /dev/md0 496M 4.6M 451M 1% /rwroot Off the top of my head, I do not know if it is the Size vs. the Used above that that indicates the (virtual) memory space use better. > still i see processes being killed with the above message. >=20 >=20 > on the rockpro64 i had an fairly huge swap (4G) on an nvme that never = really got filled (~500megs maybe). Swap usage is not directly relevant. The kills can happen with no swap in use (despite swap space having been configured) based on one or more processes that stay runnable and that keep sufficiently large working sets active. Large swap spaces (that avoid warning about possible mistuning) are something like 3.6 or so times the RAM. This can be too small for tmpfs use in poudriere for the likes of poudriere's USE_TMPFS=3Dall : rust can use over 10 GiBytes of file space in its build. > I'll try your suggestions below, thanks! >=20 > Do you know of any recent changes to memory mgmt, oom conditions that = might trigger this? No. But I also have no good understanding of the complete workload on the systems that get the notice. > I've been running this setup (slapd, radiusd, smtpd) for quite some = time on the pi now without any problems, before going to 13.2 I'm not familiar with slapd, radiusd, or the like. > Thanks! >=20 >=20 >> The message is strictly about maintaining a certain amount of free >> RAM. It turns out swap does not automatically avoid the issue for >> all contexts. >>=20 >> QUOTE from back in 2022-Dec for another context with the problem: >> (I've not reworked the wording to your context but the points >> will probably be clear anyway.) >>=20 >> This is the FreeBSD kernel complaining about the configuration >> not well matching the RPi3B+ workload. In essence, it was unable >> to achieve it targeted minimum amount of free RAM in the sort of >> time frame (really: effort) it is configured for. Depending on >> what you do, the FreeBSD defaults do not work well for 1 GiByte >> of RAM. Swap space alone is insufficient because FreeBSD does >> not swap out processes that stay runnable. Just one process that >> stays runnable using a working set that is as large as the fits >> RAM for overall operation will lead to such "failed to reclaim >> memory" kills. >>=20 >> But, if you are getting this, you will almost certainly need >> a non-trivial swap space anyway. >>=20 >> I have a starting point to recommend, configuring some >> settings. As I've no detailed clue for your context, >> I'll just provide the general description. >>=20 >>=20 >> A) I recommend a swap space something like shown in >> the below (from gpart show output): >>=20 >> =3D> 40 1953525088 da0 GPT (932G) >> 40 532480 1 efi (260M) >> 532520 2008 - free - (1.0M) >> 534528 7340032 2 freebsd-swap (3.5G) >> . . . >> 67643392 1740636160 5 freebsd-ufs (830G) >> 1808279552 145245576 - free - (69G) >>=20 >> This size (3.5 GiBytes or so) is somewhat below >> were FreeBSD starts to complain about potential >> mistuning from a large swap space, given the 1 >> GiByte of RAM. (I boot the same boot media on a >> variety of machines and have other swap partitions >> to match up with RAM sizes. But I omitted showing >> them.) >>=20 >> It is easy to have things like buildworld or >> building ports end up with individual processes >> that are temporarily bigger than the 1 GiByte RAM. >> Getting multiple cores going can also lead to >> not fitting and needing to page. >>=20 >> I'll note that I normally use USB3 NVMe media that >> also works with USB2 ports. My alternate is USB3 >> SSD media that works with USB2 ports. I avoid >> spinning rust and microsd cards. This limits what >> I can usefully comment on for some aspects of >> configuration related to the alternatives. >>=20 >>=20 >> B) /boot/loader.conf content: >>=20 >> # >> # Delay when persistent low free RAM leads to >> # Out Of Memory killing of processes: >> vm.pageout_oom_seq=3D120 >> # >> # For plunty of swap/paging space (will not >> # run out), avoid pageout delays leading to >> # Out Of Memory killing of processes: >> vm.pfault_oom_attempts=3D-1 >> # >> # For possibly insufficient swap/paging space >> # (might run out), increase the pageout delay >> # that leads to Out Of Memory killing of >> # processes (showing defaults at the time): >> #vm.pfault_oom_attempts=3D 3 >> #vm.pfault_oom_wait=3D 10 >> # (The multiplication is the total but there >> # are other potential tradoffs in the factors >> # multiplied, even for nearly the same total.) >>=20 >> If use of vm.pfault_oom_attempts=3D-1 is going to >> be inappropriate, I do not have background with >> figuring out a good combination of settings for >> vm.pfault_oom_attempts and vm.pfault_oom_wait . >>=20 >> I'll note that vm.pageout_oom_seq is not a time >> --more like how many insufficient tries to >> reclaim RAM happen in sequence before an OOM >> kill is started (effort). 120 is 10 times the >> default. While nothing disables such criteria, >> larger figures can be used if needed. (I've >> never had to but others have.) >>=20 >>=20 >> C) /etc/sysctl.conf content: >>=20 >> # >> # Together this pair avoids swapping out the process kernel stacks. >> # This avoids one way for processes for interacting with the system >> # from ending up being hung-up. >> vm.swap_enabled=3D0 >> vm.swap_idle_enabled=3D0 >>=20 >>=20 >> D) I strictly avoid having tmpfs complete for RAM >> in this kind of context. tmpfs use just makes >> avoiding "failed to reclaim memory" more difficult >> to avoid. (As various folks have run into despite >> having vastly more RAM than an RPi3B+.) So my >> /usr/local/etc/poudriere.conf has: >>=20 >> USE_TMPFS=3Dno >>=20 >> There are examples, like building rust, where >> anything but "no" or "data" leads to huge 10 >> GiByte+ tmpfs spaces for poudriere's build >> activity. Not a good match to an RPi3B+ . >>=20 >>=20 >> That is it for the recommendations of a starting >> point configuration. >>=20 >> With such measures, I've been able to have poudriere >> with -j4 but also using ALLOW_MAKE_JOBS=3D without using >> the likes of MAKE_JOBS_NUMBER limiting it. (So the >> load average could be around 16 a fair amount of the >> time but still not get "failed to reclaim memory" >> kills.) >>=20 >> Note: I'm not claiming above that -j4 is the best >> setting to use from, say, a elapsed time point of >> view for my poudriere bulk activity. >>=20 >> END QUOTE >>=20 >>> It started on the rockpro64 first, were i did a bit of fiddling, = e.g. en/disable swap, replace zfs with ufs, etc. nothing helped in the = end. I thought the board might be defective but now i start seeing the = same thing on the raspi as well. >>>=20 >>> Any ideas what this could be or how to debug this further? >>>=20 >>=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue May 2 10:49:55 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9cKn5JhVz48qb6 for ; Tue, 2 May 2023 10:50:05 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q9cKm39Snz3hn3 for ; Tue, 2 May 2023 10:50:04 +0000 (UTC) (envelope-from lojas@arroway.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of lojas@arroway.org designates 173.199.118.77 as permitted sender) smtp.mailfrom=lojas@arroway.org; dmarc=none Received: from [10.1.6.52] (unknown [179.190.185.222]) by hobbes.arroway.org (Postfix) with ESMTPA id B0F0214D1C2 for ; Tue, 2 May 2023 07:49:57 -0300 (-03) Date: Tue, 02 May 2023 07:49:55 -0300 User-Agent: K-9 Mail for Android In-Reply-To: References: List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Nanopi R5S support and build guide To: freebsd-arm@FreeBSD.org From: Matheus Message-ID: X-Spamd-Result: default: False [-0.87 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_SPAM_MEDIUM(0.73)[0.728]; NEURAL_HAM_LONG(-0.40)[-0.399]; R_SPF_ALLOW(-0.20)[+a]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@FreeBSD.org]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[arroway.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Q9cKm39Snz3hn3 X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=B8ren Schmidt" wrote: >> On 30 Apr 2023, at 23=2E44, Matheus wrote: >>=20 >> Hi, >>=20 >> I am trying to have FreeBSD 14 running on this SBC=2E I could not find >any guides in how to build images for it=2E I found the >people=2Efreebsd=2Eorg/~sos/ site that has some images, and one for it, b= ut >that seldom boots my board, and when it did, there was no console over >serial or vga=2E >>=20 >> If anyone can give any hints=2E Unfortunately my dev skills are not >good=2E But I can test and help build once I figure out how :) > >Hi Matheus > >The image at > >https://people=2Efreebsd=2Eorg/~sos/ARM64/current-RK356X-images/nano5-sdc= ard=2Eimg=2Egz > >for the Nanopi R5S does indeed boot with both HDMI output and serial >console (1500000baud)=2E > >The boot loader (EDK2 in FDT mode) is very picky on SD card quality >though from experience, I works for me with Sandisk Ultra / Extreme >cards but not with Samsung and cheap noname SD cards YMMV=2E Hi S=C3=B8ren,=20 I had really issues on sd carda=2E I got it to boot once, but I was printi= ng characters on screen at one per second=2E So I rebooted and don't rememb= er why rewrote the card=2E I can't boot anymore=2E Tried different cards, S= anDisk ultra, no luck=2E=20 I can boot an 13=2E2 image from the guy at personalbsd though=2E But there= I have just one ethernet=2E=20 On 13=2E2 I cannot list the ethernets nics using pciconf -lv, including th= e one that works=2E Is this expected?=20 I have little understanding of the arch, so my progress is much slow=2E=20 I got some dmesg from OpenBSD people where the nics show in ifconfig=2E Bu= t I couldn't get mine to behave this way=2E I can install though, using USB= nic=2E=20 I will try to buy a new sd card from the good list you pointed=2E=20 Another thing, I got the feeling that when I dd'ed the image using the SD = card slot on the notebook it worked and when was through usb adapter did no= t=2E Does it make sense? Using Linux mint as host for this=2E=20 Thanks so much for the answer and help,=20 Matheus=20 >You can build a stock ARM64 generic kernel and most things will be >usable, however as Ganbold wrote the DTS files is not in there yet (and >not even in linux where our DTS files are fetched from)=2E >However the EDK2 boot loader provided (and used in above image) on=20 > >https://people=2Efreebsd=2Eorg/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI=2Eit= b=EF=BF=BC=09 >NANOPI-R5S_EFI >File =C2=B7 1,7 MB > >does hand over the =E2=80=9Cright=E2=80=9D DTB file if you want to experi= ment=2E > >If you need the used DTS file and build guidance let me know in private >mail=2E=2E=2E > >-- >S=C3=B8ren Schmidt >sos@deepcore=2Edk / sos@freebsd=2Eorg >"So much code to hack, so little time" --- "We will call you Cygnus, the God of balance you shall be=2E" From nobody Tue May 2 10:54:28 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9cR64f6Dz48qsH for ; Tue, 2 May 2023 10:54:42 +0000 (UTC) (envelope-from ganbold@gmail.com) Received: from mail-il1-x12e.google.com (mail-il1-x12e.google.com [IPv6:2607:f8b0:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9cR62Hs3z3jKW for ; Tue, 2 May 2023 10:54:42 +0000 (UTC) (envelope-from ganbold@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x12e.google.com with SMTP id e9e14a558f8ab-33131d7a26eso2114825ab.0 for ; Tue, 02 May 2023 03:54:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683024881; x=1685616881; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=2Rn2uq+EDPUBBeIaBhJXBGazolFa9w5CQNogUlIfKaA=; b=k2q7+qTjILUSxDdgYYWP7BW/nmyin1GldA/Ij5/pjq4FCQWIVjkVMfQl5l56YlsRk7 0dwc71fpkDwJ77D1ar6bJkJghtEuPKD0V++p91FXY1j8p5V7x8fHZ3Kva6QhPIL+6kuh EzFQs8O4xpsZKygR0EG/Ru2T9WLyXFa3b9iZ5NQETP2CElxvhqtMiPgbg2PONWy6EpXv AQ7U7KopfO0e7kYnvhQmKx9JCeY7OWwon0vJjobHujsLpfkTfSVZkSXHUtNE0xyQTuH/ phxZTUA9zpHOXq8WSAvQ07CWpgR4Tc2bkWjT+iAmIYhw8WdRYl07x7/pQGXYqGxb6E1n 8xQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683024881; x=1685616881; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2Rn2uq+EDPUBBeIaBhJXBGazolFa9w5CQNogUlIfKaA=; b=U1vzskuYy/uVgO+iSTUn4ENJr8Z1OM0Ws8dJE/OBK9mL848hYPE/GrI5JXhnBkNYrl Uz1ZHs22QMIrb7/OC4HWuRptdWFhFuJ1PfHkP+Gbv0aE9Gr3urE0dMLDdw4Tc6WkENIx 7mDclEj8OUo1K7F0+yqwdWhGducMscBZVv8dKnEDnMetx43iEOht4YZLt3d5qZPB0bE7 DwHN5vYCg+iVz1abJfiPIxmT3XKereseFIoB0C1rX0WLqH1PleQnxcnftOD9+4vT6lJI MWGqheGjrjJRS1xLY8pAH3ZtnBa8ym7z+vGGHuOl9iiG3npt+/38l8EYZhGBUuCUHvTe Jjlg== X-Gm-Message-State: AC+VfDwhyVl57ENJuuFkEoJSqkY5AXPObjjPQ9zz2D4YCPnf1U319C7u OLOOdqlljvIbrZkWPgtRfa+cTzDNPfRW18VcNf8= X-Google-Smtp-Source: ACHHUZ5m3BLkQ2aRSEAtuvtbijbPoQjxyr/T9oDF6FVWnwd7qDz6xbdmOjsDpEAHwzTfVROrJ1bfYm34IiDWInnQ9BE= X-Received: by 2002:a92:c90e:0:b0:32a:85f7:eaa6 with SMTP id t14-20020a92c90e000000b0032a85f7eaa6mr12021744ilp.6.1683024880687; Tue, 02 May 2023 03:54:40 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Ganbold Tsagaankhuu Date: Tue, 2 May 2023 18:54:28 +0800 Message-ID: Subject: Re: Nanopi R5S support and build guide To: Matheus Cc: freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="00000000000029603505fab3c46d" X-Rspamd-Queue-Id: 4Q9cR62Hs3z3jKW X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000029603505fab3c46d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, May 2, 2023 at 6:50=E2=80=AFPM Matheus wrote: > > > On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=B8ren Schmidt" < > soren.schmidt@gmail.com> wrote: > >> On 30 Apr 2023, at 23.44, Matheus wrote: > >> > >> Hi, > >> > >> I am trying to have FreeBSD 14 running on this SBC. I could not find > >any guides in how to build images for it. I found the > >people.freebsd.org/~sos/ site that has some images, and one for it, but > >that seldom boots my board, and when it did, there was no console over > >serial or vga. > >> > >> If anyone can give any hints. Unfortunately my dev skills are not > >good. But I can test and help build once I figure out how :) > > > >Hi Matheus > > > >The image at > > > > > https://people.freebsd.org/~sos/ARM64/current-RK356X-images/nano5-sdcard.= img.gz > > > >for the Nanopi R5S does indeed boot with both HDMI output and serial > >console (1500000baud). > > > >The boot loader (EDK2 in FDT mode) is very picky on SD card quality > >though from experience, I works for me with Sandisk Ultra / Extreme > >cards but not with Samsung and cheap noname SD cards YMMV. > > Hi S=C3=B8ren, > > I had really issues on sd carda. I got it to boot once, but I was printin= g > characters on screen at one per second. So I rebooted and don't remember > why rewrote the card. I can't boot anymore. Tried different cards, SanDis= k > ultra, no luck. > > I can boot an 13.2 image from the guy at personalbsd though. But there I > have just one ethernet. > On 13.2 I cannot list the ethernets nics using pciconf -lv, including the > one that works. Is this expected? Yes. Did you try https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Nano= Pi-R5S-20230402.img.xz ? This image should have support for pcie and all ethernet should work IIRC. Ganbold > > I have little understanding of the arch, so my progress is much slow. > I got some dmesg from OpenBSD people where the nics show in ifconfig. But > I couldn't get mine to behave this way. I can install though, using USB > nic. > I will try to buy a new sd card from the good list you pointed. > Another thing, I got the feeling that when I dd'ed the image using the SD > card slot on the notebook it worked and when was through usb adapter did > not. Does it make sense? Using Linux mint as host for this. > Thanks so much for the answer and help, > > Matheus > > > >You can build a stock ARM64 generic kernel and most things will be > >usable, however as Ganbold wrote the DTS files is not in there yet (and > >not even in linux where our DTS files are fetched from). > >However the EDK2 boot loader provided (and used in above image) on > > > >https://people.freebsd.org/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb=EF= =BF=BC > >NANOPI-R5S_EFI > >File =C2=B7 1,7 MB > > > >does hand over the =E2=80=9Cright=E2=80=9D DTB file if you want to exper= iment. > > > >If you need the used DTS file and build guidance let me know in private > >mail... > > > >-- > >S=C3=B8ren Schmidt > >sos@deepcore.dk / sos@freebsd.org > >"So much code to hack, so little time" > > --- > "We will call you Cygnus, > the God of balance you shall be." > > --00000000000029603505fab3c46d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Tue, May 2, 2023 at 6:50=E2=80=AFP= M Matheus <lojas@arroway.org>= ; wrote:


On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=B8ren Schmidt" <soren.schmidt@gma= il.com> wrote:
>> On 30 Apr 2023, at 23.44, Matheus <lojas@arroway.org> wrote:
>>
>> Hi,
>>
>> I am trying to have FreeBSD 14 running on this SBC. I could not fi= nd
>any guides in how to build images for it. I found the
>people.freebsd.org/~sos/ site that has some images, and one= for it, but
>that seldom boots my board, and when it did, there was no console over<= br> >serial or vga.
>>
>> If anyone can give any hints. Unfortunately my dev skills are not<= br> >good. But I can test and help build once I figure out how :)
>
>Hi Matheus
>
>The image at
>
>https://people.fr= eebsd.org/~sos/ARM64/current-RK356X-images/nano5-sdcard.img.gz
>
>for the Nanopi R5S does indeed boot with both HDMI output and serial >console (1500000baud).
>
>The boot loader (EDK2 in FDT mode) is very picky on SD card quality
>though from experience, I works for me with Sandisk Ultra / Extreme
>cards but not with Samsung and cheap noname SD cards YMMV.

Hi S=C3=B8ren,

I had really issues on sd carda. I got it to boot once, but I was printing = characters on screen at one per second. So I rebooted and don't remembe= r why rewrote the card. I can't boot anymore. Tried different cards, Sa= nDisk ultra, no luck.

I can boot an 13.2 image from the guy at personalbsd though. But there I ha= ve just one ethernet.
On 13.2 I cannot list the ethernets nics using pciconf -lv, including the o= ne that works. Is this expected?

This image should have support for pcie and all ethernet should = work IIRC.

Ganbold

=C2=A0=

I have little understanding of the arch, so my progress is much slow.
I got some dmesg from OpenBSD people where the nics show in ifconfig. But I= couldn't get mine to behave this way. I can install though, using USB = nic.
I will try to buy a new sd card from the good list you pointed.
Another thing, I got the feeling that when I dd'ed the image using the = SD card slot on the notebook it worked and when was through usb adapter did= not. Does it make sense? Using Linux mint as host for this.
Thanks so much for the answer and help,

Matheus


>You can build a stock ARM64 generic kernel and most things will be
>usable, however as Ganbold wrote the DTS files is not in there yet (and=
>not even in linux where our DTS files are fetched from).
>However the EDK2 boot loader provided (and used in above image) on
>
>https://people.freebsd.org/~= sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb=EF=BF=BC=C2=A0
>NANOPI-R5S_EFI
>File =C2=B7 1,7 MB
>
>does hand over the =E2=80=9Cright=E2=80=9D DTB file if you want to expe= riment.
>
>If you need the used DTS file and build guidance let me know in private=
>mail...
>
>--
>S=C3=B8ren Schmidt
>sos@deepcore.dk / sos@freebsd.org
>"So much code to hack, so little time"

---
"We will call you Cygnus,
the God of balance you shall be."

--00000000000029603505fab3c46d-- From nobody Tue May 2 12:48:39 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9fz01bfKz48xMw for ; Tue, 2 May 2023 12:49:00 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q9fyz1s0gz3xNp for ; Tue, 2 May 2023 12:48:59 +0000 (UTC) (envelope-from lojas@arroway.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of lojas@arroway.org designates 173.199.118.77 as permitted sender) smtp.mailfrom=lojas@arroway.org; dmarc=none Received: from [10.15.253.193] (unknown [191.247.23.31]) by hobbes.arroway.org (Postfix) with ESMTPA id 1D66E14D1C2 for ; Tue, 2 May 2023 09:48:57 -0300 (-03) Date: Tue, 02 May 2023 09:48:39 -0300 User-Agent: K-9 Mail for Android In-Reply-To: References: List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Nanopi R5S support and build guide To: freebsd-arm@freebsd.org From: Matheus Message-ID: <3ECD87D6-C4CB-4566-A251-4FEF8F8DF044@arroway.org> X-Spamd-Result: default: False [-1.04 / 15.00]; NEURAL_HAM_MEDIUM(-0.66)[-0.664]; NEURAL_HAM_SHORT(-0.58)[-0.577]; NEURAL_SPAM_LONG(0.40)[0.401]; R_SPF_ALLOW(-0.20)[+a:c]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DMARC_NA(0.00)[arroway.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Q9fyz1s0gz3xNp X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N On May 2, 2023 7:54:28 AM GMT-03:00, Ganbold Tsagaankhuu wrote: >On Tue, May 2, 2023 at 6:50=E2=80=AFPM Matheus wrot= e: > >> >> >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=B8ren Schmidt" < >> soren=2Eschmidt@gmail=2Ecom> wrote: >> >> On 30 Apr 2023, at 23=2E44, Matheus wrote: >> >> >> >> Hi, >> >> >> >> I am trying to have FreeBSD 14 running on this SBC=2E I could not >find >> >any guides in how to build images for it=2E I found the >> >people=2Efreebsd=2Eorg/~sos/ site that has some images, and one for it= , >but >> >that seldom boots my board, and when it did, there was no console >over >> >serial or vga=2E >> >> >> >> If anyone can give any hints=2E Unfortunately my dev skills are not >> >good=2E But I can test and help build once I figure out how :) >> > >> >Hi Matheus >> > >> >The image at >> > >> > >> >https://people=2Efreebsd=2Eorg/~sos/ARM64/current-RK356X-images/nano5-sdc= ard=2Eimg=2Egz >> > >> >for the Nanopi R5S does indeed boot with both HDMI output and serial >> >console (1500000baud)=2E >> > >> >The boot loader (EDK2 in FDT mode) is very picky on SD card quality >> >though from experience, I works for me with Sandisk Ultra / Extreme >> >cards but not with Samsung and cheap noname SD cards YMMV=2E >> >> Hi S=C3=B8ren, >> >> I had really issues on sd carda=2E I got it to boot once, but I was >printing >> characters on screen at one per second=2E So I rebooted and don't >remember >> why rewrote the card=2E I can't boot anymore=2E Tried different cards, >SanDisk >> ultra, no luck=2E >> >> I can boot an 13=2E2 image from the guy at personalbsd though=2E But >there I >> have just one ethernet=2E >> On 13=2E2 I cannot list the ethernets nics using pciconf -lv, including >the >> one that works=2E Is this expected? > > >Yes=2E Did you try >https://personalbsd=2Eorg/download/Business/FreeBSD-aarch64-14=2E0-CURREN= T-NanoPi-R5S-20230402=2Eimg=2Exz >? >This image should have support for pcie and all ethernet should work >IIRC=2E > >Ganbold Hi Ganbold,=20 Now you got me=2E I can't tell if this or the image S=C3=B8ren pointed got= me to boot and be super slow on the kernel load=2E I will try both later a= t home=2E=20 Thanks,=20 Matheus=20 >> >> I have little understanding of the arch, so my progress is much slow=2E >> I got some dmesg from OpenBSD people where the nics show in ifconfig=2E >But >> I couldn't get mine to behave this way=2E I can install though, using >USB >> nic=2E >> I will try to buy a new sd card from the good list you pointed=2E >> Another thing, I got the feeling that when I dd'ed the image using >the SD >> card slot on the notebook it worked and when was through usb adapter >did >> not=2E Does it make sense? Using Linux mint as host for this=2E >> Thanks so much for the answer and help, >> >> Matheus >> >> >> >You can build a stock ARM64 generic kernel and most things will be >> >usable, however as Ganbold wrote the DTS files is not in there yet >(and >> >not even in linux where our DTS files are fetched from)=2E >> >However the EDK2 boot loader provided (and used in above image) on >> > >> >>https://people=2Efreebsd=2Eorg/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI=2Ei= tb=EF=BF=BC >> >NANOPI-R5S_EFI >> >File =C2=B7 1,7 MB >> > >> >does hand over the =E2=80=9Cright=E2=80=9D DTB file if you want to exp= eriment=2E >> > >> >If you need the used DTS file and build guidance let me know in >private >> >mail=2E=2E=2E >> > >> >-- >> >S=C3=B8ren Schmidt >> >sos@deepcore=2Edk / sos@freebsd=2Eorg >> >"So much code to hack, so little time" >> >> --- >> "We will call you Cygnus, >> the God of balance you shall be=2E" >> >> --- "We will call you Cygnus, the God of balance you shall be=2E" From nobody Tue May 2 18:30:28 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9pY02fHBz49GmT for ; Tue, 2 May 2023 18:30:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9pY00kmGz3wfH for ; Tue, 2 May 2023 18:30:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683052228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hmcjT5O15L8pK9OtR2Jv3HYkKHpgoFNqyVdJHA8e+Z4=; b=dfSv3g9p2pzmKAHcbOauV/cfYWcK0d107VAMRJMSpCy+FlICL8q/M742TDLQUJs3OmyB9p 3xGKYd53WjWuRA8eLgEUTrfLHODdzLsC0M8TBaDMh+BEYaXUivXjmL8SQv+/brK4gfKj0D bDJJ+yTyW4r68pP2PMiejZTvyb06iAm/9rjqv1sfJiqftn97ua2077CDNW7gQhEVPZl89t KQxDEg7cGFRpK3/7ALBirzhyNazcQS/uvMAJWzwGv4BO36QlCCep6Wg5pKoMPv1QHTfwjZ li+fdORQxd+ykU6tYIGGwON/QQmh4pcwhXw5CNAmudXr7XugsLekV42fcikoww== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1683052228; a=rsa-sha256; cv=none; b=ikw3QUaDCHV5qV1L3G7rYF+E2ORCtTXR+BcVIRKYdm2xISqRuX/j70yyaPpKScWjp1muxw Ps4kuY+74xmNPCJjbHtZKw8vvk+j/KWT0Ij6wLZ5g4SpTNS1ZwWY0WEWLBVNZrRdifRWM/ yPF60CK1dRT3UfdOA/7E8K24vc3jaN3x46xuBFIFRDZyFx+NH4tF1HaC0S1RU4SLFYLIfx h9Byv6IxMKqOZi3JPle6OUWq0hlwtNcbIYTLF+1aQB8QLsctDqiH5u9W70KCaFFn6kzyF7 GREHyONzYpoHoFAnS++j+WotHU0i6XM6So3C9n/nLIBrgJA89otgF07fUGJVZA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Q9pXz6yGSzyFX for ; Tue, 2 May 2023 18:30:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 342IURto040650 for ; Tue, 2 May 2023 18:30:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 342IURUs040649 for freebsd-arm@FreeBSD.org; Tue, 2 May 2023 18:30:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: freebsd-arm@FreeBSD.org Subject: [Bug 259438] 12.3-BETA1 SD image does not boot on Raspberry Pi 3B+ Date: Tue, 02 May 2023 18:30:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: arm X-Bugzilla-Version: 12.3-RELEASE X-Bugzilla-Keywords: needs-patch, needs-qa X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: karels@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-arm@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? mfc-stable13? mfc-stable12? X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D259438 Mike Karels changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Open |Closed Resolution|--- |Overcome By Events --- Comment #8 from Mike Karels --- Closing, as change has come in from upstream and is in all current releases. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed May 3 10:34:59 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QBDZr6vszz49DQJ for ; Wed, 3 May 2023 11:03:32 +0000 (UTC) (envelope-from dsl@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QBDZr6P25z3jCV; Wed, 3 May 2023 11:03:32 +0000 (UTC) (envelope-from dsl@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683111812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UM7ISAe2X+6BBR2ulf+b/7lzolNSvAFiWah2tt5wQGY=; b=lPql3ye38tz91jW+SzfxpBVautGrOggk6BGYxBYycxcOWDRNpXZK+9GXW5imiwp4YEPP8T VXp73gOMuNUV1x28caYMEmAw4VHlcUjCCSqH75Exl11SrOZeaGr3nyShFrC1rbfr5L380q ttUPrJgARxXAAyv2sKpLJ3ynHjy2V4xLjz040HDSYKAvRJb8T7ochVOhisl6w4o8FwFBvl jRXbRfVlZt2D48zyOOh29rJjL4dfB7vq/LZ6I4v01hE/9xikw+5ymNETOyfC7oJHqi5o4p zQPXLbRAc+7FrMQVZLYCyYpe+Y/Vn81bVjZ/NDPUPsUJ4SJPCNZENKMrZjPE/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1683111812; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UM7ISAe2X+6BBR2ulf+b/7lzolNSvAFiWah2tt5wQGY=; b=fZ1oM6fN8O4zPOFtFY/EoG6lELXsLNvlGvq+RaNxusMzwO+OGhicwEjceCJruolX9LwbxZ sWTuMe5d9+CT+EHhZWpYHAANVkLu/cttAR+e76CF/gJ9BKjJifVMpMeJlkL/S/TOonhFqz mNoTAhKHmPYrN1fiCxgoBeJ83+QuDOnSrAhRfMhmw+JADRdueUv4ahWL1K6xGc0+4oq5mG P3fr0amxKuQjLLPZGm9VaC2HgWDAEE8bUvZ7zd2t5H0PSK+woGp35eRDvER5D00nUIa5Hq uKsqVYW0aEWjrCuPa1+qpJhlLkB/00y3BFsGS6PxlHw1B3nOLLD12jQNHl7C5A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1683111812; a=rsa-sha256; cv=none; b=eMA1XyuR/uu3lmusLoPvUR5QrO0btwUi93GjE94s4a7bPU92rJE1mvzUMFwklMm/OdmIIT JX9FRbVltL+UcEX81hsIs/C7VHDYPL0CX59E3Yac2CebKDw6Yf/OjE6F+GUaYEVHQiyEs9 B18NCAJIMwXKkTE82lR7L2DOohrCidM8iEzpoLSGSE02X7iyobUFGqIYJYMS/J74qlnoz2 C3tmg/4XRzkfLKUrG+N756sWIiczS0vzfC1cTCuonaPCYa3bv9v8MO3kNQmtxH4iObygRW dxKhJIqOOwnkqzCitRpbslQLZRxXK6HOp3ltF3Mjqp6cRwJlInNMigSwNzivmA== Received: from localhost (unknown [193.164.254.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: dsl) by smtp.freebsd.org (Postfix) with ESMTPSA id 4QBDZr2F8rzfvr; Wed, 3 May 2023 11:03:32 +0000 (UTC) (envelope-from dsl@freebsd.org) References: <86ildyucuv.fsf@peasant.tower.home> <86zg78s2bj.fsf@peasant.tower.home> <08ff62fd-89db-d429-9ef3-04f213118f83@freebsd.org> User-agent: mu4e 1.6.10; emacs 28.2 From: Dmitry Salychev To: Mitchell Horne Cc: freebsd-arm@freebsd.org Subject: Re: About PHYS_TO_DMAP Date: Wed, 03 May 2023 12:34:59 +0200 In-reply-to: <08ff62fd-89db-d429-9ef3-04f213118f83@freebsd.org> Message-ID: <86r0rxllrw.fsf@peasant.tower.home> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain X-ThisMailContainsUnwantedMimeParts: N Mitchell Horne writes: > On 4/16/23 04:06, Dmitry Salychev wrote: >> Mitchell Horne writes: >> >>> On 4/14/23 04:31, Dmitry Salychev wrote: >>>> Hi, >>>> I'm struggling to understand which KVA will be returned by >>>> PHYS_TO_DMAP >>>> on arm64. For example, if I'll create a DMA tag this way: >>>> bus_dma_tag_create( >>>> bus_get_dma_tag(dev), >>>> sc->buf_align, 0, /* alignment, boundary */ >>>> DMAP_MAX_PHYSADDR, /* low restricted addr */ >>>> DMAP_MIN_PHYSADDR, /* high restricted addr */ >>> >>> I think you are confused about the purpose of lowaddr and >>> highaddr. They specify the window of bus-space addresses that are >>> *inaccessible* *to the device* for DMA. It does not prevent you from >>> providing a buffer backed by physical memory within this range, but in >>> this case bounce pages will be used as an intermediate to perform the >>> DMA transaction. >>> >>> Most commonly, if the device can only address a 32-bit range, then the >>> values lowaddr=BUS_SPACE_MAXADDR_32BIT and highaddr=BUS_SPACE_MAXADDR >>> will be given. If the entire bus range is accessible to the device for >>> DMA, a zero-sized window will be given: lowaddr=BUS_SPACE_MAXADDR, >>> highaddr=BUS_SPACE_MAXADDR. >>> >>> This is all described in adequate detail in busdma(9), but it is still >>> not easily understood without a lot of code study, in my experience. >>> >> According to busdma(9), the window contains all addresses *greater >> than* >> lowaddr and *less than or equal to* highaddr, i.e. your example with the >> 32-bit range looks like: >> allowed prohibited >> ------------------------------(xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx> >> lowaddr highaddr >> BUS_SPACE_MAXADDR_32BIT BUS_SPACE_MAXADDR >> But my example looks a bit different: >> prohibited allowed prohibited >> xxxxxxxxxxxxxxxx]----------------------------(xxxxxxxxxxxxxxxxxxx> >> highaddr lowaddr >> DMAP_MIN_PHYSADDR DMAP_MAX_PHYSADDR >> > > Okay I see, you are using MAX as lowaddr and MIN as highaddr. I read > it backwards the first time. > > However, I looked at the code and I'm not sure that it will handle the > case where lowaddr > highaddr correctly. Just a warning to you. > >> I've found the only example of such DMA tag configuration in >> src/sys/dev/vnic/nicvf_queues.c at line 379 (rbdr_buff_dmat). >> > I think this example is misleading, if not incorrect. All DMA > transactions for this tag will have a destination (physical memory > address) that falls within the [DMAP_MIN_PHYSADDR,DMAP_MAX_PHYSADDR] > range. So, the specified boundaries are not meaningful, and don't > represent a real constraint of the device. At least, this is my > understanding of it. I don't know the bus_dma implementation details yet unfortunately and thought that DMA tag created with [DMAP_MIN_PHYSADDR,DMAP_MAX_PHYSADDR] window is required to calculate KVA by PHYS_TO_DMAP properly. However, PHYS_TO_DMAP gave me an address to access frame annotation even with DMA tag created without a restriction window at all, i.e. BUS_SPACE_MAXADDR used for both lowaddr and highaddr. I've opened a review https://reviews.freebsd.org/D39946 if you'll be interested in details. > >> >>>> NULL, NULL, /* filter, filterarg */ >>>> BUF_SIZE, 1, /* maxsize, nsegments */ >>>> BUF_SIZE, 0, /* maxsegsize, flags */ >>>> NULL, NULL, /* lockfunc, lockarg */ >>>> &dmat); >>>> in order to restrict any physical addresses but a window defined by >>>> DMAP_MIN_PHYSADDR and DMAP_MAX_PHYSADDR. Later on when I'll be >>>> mapping my mbuf (BUF_SIZE) with >>>> bus_dmamap_load_mbuf_sg(dmat, dmap, >>>> m, &segs, &nsegs, BUS_DMA_NOWAIT); >>>> I expect that >>>> m->m_data == PHYS_TO_DMAP(segs[0].ds_addr) >>> >>> Why do you expect or need this to be the case? >>> >>> busdma is not responsible for setting or modifying m_data, it comes >>> from wherever you allocated the mbuf. Calling >>> bus_dmamap_load_mbuf_sg() will prepare a DMA mapping where the m_data >>> buffer is used as the source/destination for the DMA transaction, but >>> busdma does not allocate the buffer itself. >>> >> I don't need this to be the case exactly, but I'd like to be able to >> access a "frame annotation" (64 bytes long) which is located exactly at >> the start of m_data buffer having that physical address is provided, i.e. >> fa = (struct dpaa2_fa *) mbuf->m_data; >> /* ... fa populated ... */ >> /* ... DMA transaction of the Tx frame (together with fa) ... */ >> /* ... Tx confirmation from HW (bus address of the frame >> only) ...*/ >> fa = (struct dpaa2_fa *) PHYS_TO_DMAP(paddr); >> > > The short answer is that you are using PHYS_TO_DMAP correctly. Every > real paddr should have a corresponding VA in the DMAP that is valid > for read/write access to that memory. However, it is unusual to see > DMAP usage in driver code. > > It is hard for me to say exactly what you should be doing without more > information. > > In general, the m_data pointer remains valid after the DMA > transaction. If your updated frame annotation is located at the > beginning of this buffer, then you want to use the same m_data pointer > to access it. > > I guess the challenge is that the TX confirmation occurs in a separate > context (interrupt handler?) where the mbuf reference is lost? I > believe the usual practice is to maintain some kind of ring or queue > of transmitted mbuf+dma_map_t pairs, which can be handled together in > the TX interrupt handler (unload DMA map, free and/or reassign mbuf). This is correct. However, HW (firmware in my case) provides a frame descriptor which carries physical address of the Rx/Tx buffer only. This is why I needed a way to access the buffer itself from the kernel address space. > > Of course, you must use appropriate sync calls before accessing the > buffer again on the host, probably: bus_dmamap_sync(tag, map, > BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE). I'm sure you know this. > >>>> but it isn't true. Could somebody explain what exactly is returned >>>> by >>>> PHYS_TO_DMAP in this case and whether it's possible to translate >>>> physical address to KVA as fast as possible (O(1) ideally). >>>> >>> >>> PHYS_TO_DMAP is always a linear calculation of: physaddr + DMAP_MIN_ADDRESS. >>> >>> I do not think PHYS_TO_DMAP is in use at all in this example, or >>> anywhere within busdma really. >>> >> Is there any other way to obtain KVA of a buffer mapped for DMA >> transaction by the physical address? I've been crawling source code for >> sometime already, but DMAP is the only thing I managed to find. >> >>> Regards, >>>> Dmitry >>>> -- >>>> Open source software/hardware enthusiast >>>> hackaday.io/dsl | github.com/mcusim | patreon.com/salychev >>>> >> Regards, >> Dmitry -- Open source software/hardware enthusiast hackaday.io/dsl | github.com/mcusim | patreon.com/salychev From nobody Thu May 4 13:55:41 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QBwYt0MPMz498np for ; Thu, 4 May 2023 14:05:06 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4QBwYr3XTqz3Dk4 for ; Thu, 4 May 2023 14:05:04 +0000 (UTC) (envelope-from lojas@arroway.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of lojas@arroway.org designates 173.199.118.77 as permitted sender) smtp.mailfrom=lojas@arroway.org; dmarc=none Received: from [10.30.24.128] (unknown [179.240.19.90]) by hobbes.arroway.org (Postfix) with ESMTPA id D8AAF14DBA3 for ; Thu, 4 May 2023 10:55:58 -0300 (-03) Date: Thu, 04 May 2023 10:55:41 -0300 User-Agent: K-9 Mail for Android In-Reply-To: References: List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----EEICAKAOLJQP5UV08T1OF10ZH2LB7K" Content-Transfer-Encoding: 7bit Subject: Re: Nanopi R5S support and build guide To: freebsd-arm@freebsd.org From: Matheus Message-ID: X-Spamd-Result: default: False [1.87 / 15.00]; R_SUSPICIOUS_URL(5.00)[people.freebsd.org]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; NEURAL_HAM_LONG(-0.93)[-0.931]; R_SPF_ALLOW(-0.20)[+a:c]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DMARC_NA(0.00)[arroway.org]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCPT_COUNT_ONE(0.00)[1]; GREYLIST(0.00)[pass,body]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4QBwYr3XTqz3Dk4 X-Spamd-Bar: + X-ThisMailContainsUnwantedMimeParts: N ------EEICAKAOLJQP5UV08T1OF10ZH2LB7K Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > On Tue, May 2, 2023 at 6:50=C3=A2=C2=80=C2=AFPM Matheus wrote: > >> >> >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=83=C2=B8ren Schmidt" < >> soren=2Eschmidt@gmail=2Ecom> wrote: >> >> On 30 Apr 2023, at 23=2E44, Matheus wrote: >> >> >> >> Hi, >> >> >> >> I am trying to have FreeBSD 14 running on this SBC=2E I could not fi= nd >> >any guides in how to build images for it=2E I found the >> >people=2Efreebsd=2Eorg/~sos/ site that has some images, and one for it= , but >> >that seldom boots my board, and when it did, there was no console over >> >serial or vga=2E >> >> >> >> If anyone can give any hints=2E Unfortunately my dev skills are not >> >good=2E But I can test and help build once I figure out how :) >> > >> >Hi Matheus >> > >> >The image at >> > >> > >> https://people=2Efreebsd=2Eorg/~sos/ARM64/current-RK356X-images/nano5-s= dcard=2Eimg=2Egz >> > >> >for the Nanopi R5S does indeed boot with both HDMI output and serial >> >console (1500000baud)=2E >> > >> >The boot loader (EDK2 in FDT mode) is very picky on SD card quality >> >though from experience, I works for me with Sandisk Ultra / Extreme >> >cards but not with Samsung and cheap noname SD cards YMMV=2E >> >> Hi S=C3=83=C2=B8ren, >> >> I had really issues on sd carda=2E I got it to boot once, but I was >> printing >> characters on screen at one per second=2E So I rebooted and don't remem= ber >> why rewrote the card=2E I can't boot anymore=2E Tried different cards, >> SanDisk >> ultra, no luck=2E >> >> I can boot an 13=2E2 image from the guy at personalbsd though=2E But th= ere I >> have just one ethernet=2E >> On 13=2E2 I cannot list the ethernets nics using pciconf -lv, including >> the >> one that works=2E Is this expected? > > > Yes=2E Did you try > https://personalbsd=2Eorg/download/Business/FreeBSD-aarch64-14=2E0-CURRE= NT-NanoPi-R5S-20230402=2Eimg=2Exz > ? > This image should have support for pcie and all ethernet should work IIR= C=2E > > Ganbold Hi Ganbold, I tried it and other 2 images and no success, including the image pointed by Soren=2E Unfortunately I just got to boot one image from 14 but I overwritten the sd card and can't remember which sd card and image :( The only image I can make it boot is from 13=2E2R, and I got it installed fine (using an EFI image from personalbsd from March 23)=2E Will wait for the next round of images for 14=2E Thanks, matheus >> >> I have little understanding of the arch, so my progress is much slow=2E >> I got some dmesg from OpenBSD people where the nics show in ifconfig=2E >> But >> I couldn't get mine to behave this way=2E I can install though, using U= SB >> nic=2E >> I will try to buy a new sd card from the good list you pointed=2E >> Another thing, I got the feeling that when I dd'ed the image using the >> SD >> card slot on the notebook it worked and when was through usb adapter di= d >> not=2E Does it make sense? Using Linux mint as host for this=2E >> Thanks so much for the answer and help, >> >> Matheus >> >> >> >You can build a stock ARM64 generic kernel and most things will be >> >usable, however as Ganbold wrote the DTS files is not in there yet (an= d >> >not even in linux where our DTS files are fetched from)=2E >> >However the EDK2 boot loader provided (and used in above image) on >> > >> >https://people=2Efreebsd=2Eorg/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI= =2Eitb=C3=AF=C2=BF=C2=BC >> >NANOPI-R5S_EFI >> >File =C3=82=C2=B7 1,7 MB >> > >> >does hand over the =C3=A2=C2=80=C2=9Cright=C3=A2=C2=80=C2=9D DTB file = if you want to experiment=2E >> > >> >If you need the used DTS file and build guidance let me know in privat= e >> >mail=2E=2E=2E >> > >> >-- >> >S=C3=83=C2=B8ren Schmidt >> >sos@deepcore=2Edk / sos@freebsd=2Eorg >> >"So much code to hack, so little time" >> >> --- >> "We will call you Cygnus, >> the God of balance you shall be=2E" >> >> > --=20 "We will call you Cygnus, the God of balance you shall be=2E" --- "We will call you Cygnus, the God of balance you shall be=2E" ------EEICAKAOLJQP5UV08T1OF10ZH2LB7K Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable > On Tue, May 2, 2023 at 6:50=C3=A2=C2=80=C2=AFPM Matheus <lojas@arro= way=2Eorg> wrote:
>
>>
>>
>> On May 1, = 2023 1:47:12 PM GMT-03:00, "S=C3=83=C2=B8ren Schmidt" <
>> sore= n=2Eschmidt@gmail=2Ecom> wrote:
>> >> On 30 Apr 2023, at = 23=2E44, Matheus <lojas@arroway=2Eorg> wrote:
>> >>>> >> Hi,
>> >>
>> >> I am tryin= g to have FreeBSD 14 running on this SBC=2E I could not find
>> &g= t;any guides in how to build images for it=2E I found the
>> >p= eople=2Efreebsd=2Eorg/~sos/ site that has some images, and one for it, but<= br>>> >that seldom boots my board, and when it did, there was no c= onsole over
>> >serial or vga=2E
>> >>
>&g= t; >> If anyone can give any hints=2E Unfortunately my dev skills are= not
>> >good=2E But I can test and help build once I figure ou= t how :)
>> >
>> >Hi Matheus
>> >
&g= t;> >The image at
>> >
>> >
>> https://people=2Efreebsd=2Eorg/~sos/ARM64/current-RK356= X-images/nano5-sdcard=2Eimg=2Egz
>> >
>> >for t= he Nanopi R5S does indeed boot with both HDMI output and serial
>>= >console (1500000baud)=2E
>> >
>> >The boot loa= der (EDK2 in FDT mode) is very picky on SD card quality
>> >tho= ugh from experience, I works for me with Sandisk Ultra / Extreme
>>= ; >cards but not with Samsung and cheap noname SD cards YMMV=2E
>&= gt;
>> Hi S=C3=83=C2=B8ren,
>>
>> I had really i= ssues on sd carda=2E I got it to boot once, but I was
>> printing<= br>>> characters on screen at one per second=2E So I rebooted and don= 't remember
>> why rewrote the card=2E I can't boot anymore=2E Tri= ed different cards,
>> SanDisk
>> ultra, no luck=2E
&g= t;>
>> I can boot an 13=2E2 image from the guy at personalbsd t= hough=2E But there I
>> have just one ethernet=2E
>> On 1= 3=2E2 I cannot list the ethernets nics using pciconf -lv, including
>= > the
>> one that works=2E Is this expected?
>
>> Yes=2E Did you try
> https://personalbsd=2Eorg/download/Business/FreeBSD-aarch64-14=2E0-CURRENT= -NanoPi-R5S-20230402=2Eimg=2Exz
> ?
> This image should hav= e support for pcie and all ethernet should work IIRC=2E
>
> Gan= bold

Hi Ganbold,

I tried it and other 2 images and no success= , including the image pointed
by Soren=2E Unfortunately I just got to bo= ot one image from 14 but I
overwritten the sd card and can't remember wh= ich sd card and image :(

The only image I can make it boot is from 1= 3=2E2R, and I got it installed
fine (using an EFI image from personalbsd= from March 23)=2E

Will wait for the next round of images for 14=2E<= br>
Thanks,

matheus

>>
>> I have little und= erstanding of the arch, so my progress is much slow=2E
>> I got so= me dmesg from OpenBSD people where the nics show in ifconfig=2E
>>= But
>> I couldn't get mine to behave this way=2E I can install th= ough, using USB
>> nic=2E
>> I will try to buy a new sd c= ard from the good list you pointed=2E
>> Another thing, I got the = feeling that when I dd'ed the image using the
>> SD
>> ca= rd slot on the notebook it worked and when was through usb adapter did
&= gt;> not=2E Does it make sense? Using Linux mint as host for this=2E
= >> Thanks so much for the answer and help,
>>
>> Ma= theus
>>
>>
>> >You can build a stock ARM64 g= eneric kernel and most things will be
>> >usable, however as Ga= nbold wrote the DTS files is not in there yet (and
>> >not even= in linux where our DTS files are fetched from)=2E
>> >However = the EDK2 boot loader provided (and used in above image) on
>> >=
>> >https://people=2Efreebsd=2Eorg/~sos/ARM64/EDK2-RK356X/NANO= PI-R5S_EFI=2Eitb=C3=AF=C2=BF=C2=BC
>> >NANOPI-R5S_EFI
>&g= t; >File =C3=82=C2=B7 1,7 MB
>> >
>> >does hand = over the =C3=A2=C2=80=C2=9Cright=C3=A2=C2=80=C2=9D DTB file if you want to = experiment=2E
>> >
>> >If you need the used DTS fil= e and build guidance let me know in private
>> >mail=2E=2E=2E>> >
>> >--
>> >S=C3=83=C2=B8ren Schmidt=
>> >sos@deepcore=2Edk / sos@freebsd=2Eorg
>> >"So = much code to hack, so little time"
>>
>> ---
>> = "We will call you Cygnus,
>> the God of balance you shall be=2E">>
>>
>


--
"We will call you Cygnus,<= br>the God of balance you shall be=2E"
"We will call you Cygnus,
the = God of balance you shall be=2E" ------EEICAKAOLJQP5UV08T1OF10ZH2LB7K-- From nobody Fri May 5 05:23:15 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QCJxZ1DVyz49cZR for ; Fri, 5 May 2023 05:23:30 +0000 (UTC) (envelope-from ganbold@gmail.com) Received: from mail-il1-x12e.google.com (mail-il1-x12e.google.com [IPv6:2607:f8b0:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QCJxY5DZvz3xTX for ; Fri, 5 May 2023 05:23:29 +0000 (UTC) (envelope-from ganbold@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x12e.google.com with SMTP id e9e14a558f8ab-3315ccc1ce0so9979235ab.3 for ; Thu, 04 May 2023 22:23:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683264208; x=1685856208; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=rcxGseopz4Kqr3iOYzk9XaOUDTjx+nyvEU5SlDi8EKA=; b=WuWSprnAl+BO2NDXeOD9u2HYLSVDYqFmUQbQYfRJPYLV6BimeG0TeAynnnw1Snb6lc Td2k2R/1sgP6TAsPnBc2tCBV13rJYjocg9hU8Pz3iE0XA3CUjKSGVY6Gq5CyY0N6VuBf 3x7Gb1mTNfQw66JrL0ysdp/PE3VXuQhBWntRbJ9K6az7Qwk03dM/TYO66QePiWvxRSZs L/gjhS9idGGQ7h7CQzXSQDBX3Bkt/kIW59ommIFkB5Mse33/7Q2A+Leq5nkxD7y5HRPQ 32uRCcQoB5TaXnV48TekA8L6+aq1w814PgZ9lvMr9Ahe0l0EbbHHHo3SSs1ixN8qPy1u Q4mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683264208; x=1685856208; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=rcxGseopz4Kqr3iOYzk9XaOUDTjx+nyvEU5SlDi8EKA=; b=L2S1jVLNuHjvEDt3X/dxGwnRqTQX8XNmOKemi074EnuDiB684323Hk5odzWesqaAQw c9pclrTt+n+VjOSRlxeO12brBMuzGespw2tXDa8bKieicOjnddG26uuCxw1Gbug8tqEV 2kMIOBYKn7CN2hobHoUrlwAzopGOZqOgT8BdedbkspXAkz/oqL/bUeX7buZH+r7e0tph wPg+AHCl49Vw1W/6X2FuFElo1cHOO0c05pBy4fomUesuOTWUkaMBjqvVPh/ksK7NWePU yuX7FaQOgCPhNgGbgNqX73+xSjeq9FazHVHrVwzfLVdUcltWe1uhZQar9hG0OzEQCDEV VUbw== X-Gm-Message-State: AC+VfDzd16q0W5RlKhyNgMwy2bXfNZhoR2yKi4+FHmdDM5KNgQpCuHT9 +U+trmmmPdxGc1igS+BtS0kFcEFa7Jujh0cN2/i92+7y0csvTw== X-Google-Smtp-Source: ACHHUZ4cuos4uwGJx/wUDNtJQUgF8YwHtsKuhwdZ4Eq3jin75b5IUFmLRInkiUK1JTmRUQkhEPGjlzcIUj6EmSBTh/A= X-Received: by 2002:a92:90c:0:b0:329:bba2:781a with SMTP id y12-20020a92090c000000b00329bba2781amr204829ilg.0.1683264207919; Thu, 04 May 2023 22:23:27 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Ganbold Tsagaankhuu Date: Fri, 5 May 2023 13:23:15 +0800 Message-ID: Subject: Re: Nanopi R5S support and build guide To: Matheus Cc: freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="0000000000002d21cd05faeb7d17" X-Rspamd-Queue-Id: 4QCJxY5DZvz3xTX X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --0000000000002d21cd05faeb7d17 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, May 4, 2023 at 10:05=E2=80=AFPM Matheus wrote: > > On Tue, May 2, 2023 at 6:50=C3=A2=E2=82=AC=C2=AFPM Matheus wrote: > > > >> > >> > >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=83=C2=B8ren Schmidt" < > >> soren.schmidt@gmail.com> wrote: > >> >> On 30 Apr 2023, at 23.44, Matheus wrote: > >> >> > >> >> Hi, > >> >> > >> >> I am trying to have FreeBSD 14 running on this SBC. I could not fin= d > >> >any guides in how to build images for it. I found the > >> >people.freebsd.org/~sos/ site that has some images, and one for it, > but > >> >that seldom boots my board, and when it did, there was no console ove= r > >> >serial or vga. > >> >> > >> >> If anyone can give any hints. Unfortunately my dev skills are not > >> >good. But I can test and help build once I figure out how :) > >> > > >> >Hi Matheus > >> > > >> >The image at > >> > > >> > > >> > https://people.freebsd.org/~sos/ARM64/current-RK356X-images/nano5-sdcard.= img.gz > >> > > >> >for the Nanopi R5S does indeed boot with both HDMI output and serial > >> >console (1500000baud). > >> > > >> >The boot loader (EDK2 in FDT mode) is very picky on SD card quality > >> >though from experience, I works for me with Sandisk Ultra / Extreme > >> >cards but not with Samsung and cheap noname SD cards YMMV. > >> > >> Hi S=C3=83=C2=B8ren, > >> > >> I had really issues on sd carda. I got it to boot once, but I was > >> printing > >> characters on screen at one per second. So I rebooted and don't rememb= er > >> why rewrote the card. I can't boot anymore. Tried different cards, > >> SanDisk > >> ultra, no luck. > >> > >> I can boot an 13.2 image from the guy at personalbsd though. But there= I > >> have just one ethernet. > >> On 13.2 I cannot list the ethernets nics using pciconf -lv, including > >> the > >> one that works. Is this expected? > > > > > > Yes. Did you try > > > https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Na= noPi-R5S-20230402.img.xz > > ? > > This image should have support for pcie and all ethernet should work > IIRC. > > > > Ganbold > > Hi Ganbold, > > I tried it and other 2 images and no success, including the image pointed > by Soren. Unfortunately I just got to boot one image from 14 but I > overwritten the sd card and can't remember which sd card and image :( > > The only image I can make it boot is from 13.2R, and I got it installed > fine (using an EFI image from personalbsd from March 23). > > Will wait for the next round of images for 14. > Above image works for me: root@NanoPi-R5S:~ # ifconfig re0: flags=3D8803 metric 0 mtu 1500 options=3D201b ether 62:73:64:e2:d4:87 inet 192.168.111.2 netmask 0xffffff00 broadcast 192.168.111.255 media: Ethernet autoselect status: no carrier nd6 options=3D29 re1: flags=3D8843 metric 0 mtu 1500 options=3D201b ether 62:73:64:9a:5e:55 inet 192.168.2.2 netmask 0xffffff00 broadcast 192.168.2.255 media: Ethernet autoselect (1000baseT ) status: active nd6 options=3D29 eq0: flags=3D8843 metric 0 mtu 1500 options=3D80008 ether 22:07:01:bb:41:76 inet 192.168.1.202 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet autoselect (1000baseT ) status: active nd6 options=3D29 lo0: flags=3D8049 metric 0 mtu 16384 options=3D680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=3D21 root@NanoPi-R5S:~ # uname -an FreeBSD NanoPi-R5S 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n261950-4aeb939ecf8b-dirty: Sun Apr 2 17:42:01 MSK 2023 root@honeycomb.local:/usr/crochet/work/obj/usr/crochet/src-current-14.0/arm= 64.aarch64/sys/EXPERT arm64 root@NanoPi-R5S:~ # kldstat Id Refs Address Size Name 1 8 0xffff000000000000 10cb350 kernel 2 1 0xffff0000010cd000 a9508 if_re.ko 3 1 0xffff0000c9e00000 24000 fdescfs.ko 4 1 0xffff0000c9e24000 22000 mac_ntpd.ko root@NanoPi-R5S:~ # Ganbold > > Thanks, > > matheus > > >> > >> I have little understanding of the arch, so my progress is much slow. > >> I got some dmesg from OpenBSD people where the nics show in ifconfig. > >> But > >> I couldn't get mine to behave this way. I can install though, using US= B > >> nic. > >> I will try to buy a new sd card from the good list you pointed. > >> Another thing, I got the feeling that when I dd'ed the image using the > >> SD > >> card slot on the notebook it worked and when was through usb adapter d= id > >> not. Does it make sense? Using Linux mint as host for this. > >> Thanks so much for the answer and help, > >> > >> Matheus > >> > >> > >> >You can build a stock ARM64 generic kernel and most things will be > >> >usable, however as Ganbold wrote the DTS files is not in there yet (a= nd > >> >not even in linux where our DTS files are fetched from). > >> >However the EDK2 boot loader provided (and used in above image) on > >> > > >> >https://people.freebsd.org/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb= =C3=AF > =C2=BF=C2=BC > >> >NANOPI-R5S_EFI > >> >File =C3=82=C2=B7 1,7 MB > >> > > >> >does hand over the =C3=A2=E2=82=AC=C5=93right=C3=A2=E2=82=AC DTB file= if you want to experiment. > >> > > >> >If you need the used DTS file and build guidance let me know in priva= te > >> >mail... > >> > > >> >-- > >> >S=C3=83=C2=B8ren Schmidt > >> >sos@deepcore.dk / sos@freebsd.org > >> >"So much code to hack, so little time" > >> > >> --- > >> "We will call you Cygnus, > >> the God of balance you shall be." > >> > >> > > > > > -- > "We will call you Cygnus, > the God of balance you shall be." > ------------------------------ > "We will call you Cygnus, > the God of balance you shall be." --0000000000002d21cd05faeb7d17 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, May 4, 2023 at 10:05=E2=80=AF= PM Matheus <lojas@arroway.org&g= t; wrote:
> O= n Tue, May 2, 2023 at 6:50=C3=A2=E2=82=AC=C2=AFPM Matheus <lojas@arroway.org> wrote:<= br>>
>>
>>
>> On May 1, 2023 1:47:12 PM GMT-0= 3:00, "S=C3=83=C2=B8ren Schmidt" <
>> soren.schmidt@gmail.com&g= t; wrote:
>> >> On 30 Apr 2023, at 23.44, Matheus <lojas@arroway.org> = wrote:
>> >>
>> >> Hi,
>> >>>> >> I am trying to have FreeBSD 14 running on this SBC. I c= ould not find
>> >any guides in how to build images for it. I f= ound the
>> >people.freebsd.org/~sos/ site that has some images, and one= for it, but
>> >that seldom boots my board, and when it did, t= here was no console over
>> >serial or vga.
>> >>= ;
>> >> If anyone can give any hints. Unfortunately my dev s= kills are not
>> >good. But I can test and help build once I fi= gure out how :)
>> >
>> >Hi Matheus
>> >= ;
>> >The image at
>> >
>> >
>>= ; https://people.freebsd.org/~sos/ARM64/= current-RK356X-images/nano5-sdcard.img.gz
>> >
>> = >for the Nanopi R5S does indeed boot with both HDMI output and serial>> >console (1500000baud).
>> >
>> >The b= oot loader (EDK2 in FDT mode) is very picky on SD card quality
>> = >though from experience, I works for me with Sandisk Ultra / Extreme
= >> >cards but not with Samsung and cheap noname SD cards YMMV.
= >>
>> Hi S=C3=83=C2=B8ren,
>>
>> I had rea= lly issues on sd carda. I got it to boot once, but I was
>> printi= ng
>> characters on screen at one per second. So I rebooted and do= n't remember
>> why rewrote the card. I can't boot anymore= . Tried different cards,
>> SanDisk
>> ultra, no luck.>>
>> I can boot an 13.2 image from the guy at personalbsd = though. But there I
>> have just one ethernet.
>> On 13.2= I cannot list the ethernets nics using pciconf -lv, including
>> = the
>> one that works. Is this expected?
>
>
> Y= es. Did you try
> https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Na= noPi-R5S-20230402.img.xz
> ?
> This image should have suppo= rt for pcie and all ethernet should work IIRC.
>
> Ganbold
<= br>Hi Ganbold,

I tried it and other 2 images and no success, includi= ng the image pointed
by Soren. Unfortunately I just got to boot one imag= e from 14 but I
overwritten the sd card and can't remember which sd = card and image :(

The only image I can make it boot is from 13.2R, a= nd I got it installed
fine (using an EFI image from personalbsd from Mar= ch 23).

Will wait for the next round of images for 14.

Above image works for me:

r= oot@NanoPi-R5S:~ # ifconfig
re0: flags=3D8803<UP,BROADCAST,SIMPLEX,MU= LTICAST> metric 0 mtu 1500
=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D201b= <RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 ether 62:73:64:e2:d4:87
=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 1= 92.168.111.2 netmask 0xffffff00 broadcast 192.168.111.255
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 media: Ethernet autoselect
=C2=A0 =C2=A0 =C2=A0 =C2=A0 sta= tus: no carrier
=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D29<PERFORMN= UD,IFDISABLED,AUTO_LINKLOCAL>
re1: flags=3D8843<UP,BROADCAST,RUNNI= NG,SIMPLEX,MULTICAST> metric 0 mtu 1500
=C2=A0 =C2=A0 =C2=A0 =C2=A0 o= ptions=3D201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 ether 62:73:64:9a:5e:55
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 inet 192.168.2.2 netmask 0xffffff00 broadcast 192.168.2.255
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000baseT <full-dup= lex>)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 status: active
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
= eq0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mt= u 1500
=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D80008<VLAN_MTU,LINKSTATE= >
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 22:07:01:bb:41:76
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 inet 192.168.1.202 netmask 0xffffff00 broadcast 192.168.1= .255
=C2=A0 =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000baseT &= lt;full-duplex>)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 status: active
=C2=A0= =C2=A0 =C2=A0 =C2=A0 nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLO= CAL>
lo0: flags=3D8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 = mtu 16384
=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D680003<RXCSUM,TXCSUM,= LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet6 = ::1 prefixlen 128
=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet6 fe80::1%lo0 prefixle= n 64 scopeid 0x4
=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 127.0.0.1 netmask 0xff= 000000
=C2=A0 =C2=A0 =C2=A0 =C2=A0 groups: lo
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 nd6 options=3D21<PERFORMNUD,AUTO_LINKLOCAL>
root@NanoPi-R5S= :~ # uname -an
FreeBSD NanoPi-R5S 14.0-CURRENT FreeBSD 14.0-CURRENT #0 m= ain-n261950-4aeb939ecf8b-dirty: Sun Apr =C2=A02 17:42:01 MSK 2023 =C2=A0 = =C2=A0 root@honeycomb.local:/usr/crochet/work/obj/usr/crochet/src-current-1= 4.0/arm64.aarch64/sys/EXPERT arm64
root@NanoPi-R5S:~ # kldstat
Id Ref= s Address =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Size Name<= br>=C2=A01 =C2=A0 =C2=A08 0xffff000000000000 =C2=A010cb350 kernel
=C2=A0= 2 =C2=A0 =C2=A01 0xffff0000010cd000 =C2=A0 =C2=A0a9508 if_re.ko
=C2=A03 = =C2=A0 =C2=A01 0xffff0000c9e00000 =C2=A0 =C2=A024000 fdescfs.ko
=C2=A04 = =C2=A0 =C2=A01 0xffff0000c9e24000 =C2=A0 =C2=A022000 mac_ntpd.ko
root@Na= noPi-R5S:~ #

Ganbold

= =C2=A0

Thanks,
matheus

>>
>> I have little understanding of th= e arch, so my progress is much slow.
>> I got some dmesg from Open= BSD people where the nics show in ifconfig.
>> But
>> I c= ouldn't get mine to behave this way. I can install though, using USB>> nic.
>> I will try to buy a new sd card from the good li= st you pointed.
>> Another thing, I got the feeling that when I dd= 'ed the image using the
>> SD
>> card slot on the not= ebook it worked and when was through usb adapter did
>> not. Does = it make sense? Using Linux mint as host for this.
>> Thanks so muc= h for the answer and help,
>>
>> Matheus
>>
&= gt;>
>> >You can build a stock ARM64 generic kernel and most= things will be
>> >usable, however as Ganbold wrote the DTS fi= les is not in there yet (and
>> >not even in linux where our DT= S files are fetched from).
>> >However the EDK2 boot loader pro= vided (and used in above image) on
>> >
>> >https://people.freebsd.org/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb= =C3=AF=C2=BF=C2=BC
>> >NANOPI-R5S_EFI
>> >File = =C3=82=C2=B7 1,7 MB
>> >
>> >does hand over the =C3= =A2=E2=82=AC=C5=93right=C3=A2=E2=82=AC DTB file if you want to experiment.=
>> >
>> >If you need the used DTS file and build g= uidance let me know in private
>> >mail...
>> >
= >> >--
>> >S=C3=83=C2=B8ren Schmidt
>> >sos@deepcore.dk / sos@freebsd.org
&= gt;> >"So much code to hack, so little time"
>>>> ---
>> "We will call you Cygnus,
>> the Go= d of balance you shall be."
>>
>>
>

--
"We will call you Cygnus,
the God of balance you shall be.&= quot;
"We will call you Cygnus,
the God of balance you shall be.= "
--0000000000002d21cd05faeb7d17-- From nobody Fri May 5 22:03:10 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QCl7P58GQz49rlF for ; Fri, 5 May 2023 22:03:29 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4QCl7N4jFJz3jsV for ; Fri, 5 May 2023 22:03:28 +0000 (UTC) (envelope-from lojas@arroway.org) Authentication-Results: mx1.freebsd.org; none Received: from [10.9.192.149] (179-240-31-180.3g.claro.net.br [179.240.31.180]) by hobbes.arroway.org (Postfix) with ESMTPA id 7E7A114D2B0; Fri, 5 May 2023 19:03:27 -0300 (-03) Date: Fri, 05 May 2023 19:03:10 -0300 User-Agent: K-9 Mail for Android In-Reply-To: References: List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Nanopi R5S support and build guide To: freebsd-arm@freebsd.org,Ganbold Tsagaankhuu From: Matheus Message-ID: <7F68ABB3-8D7A-40EA-9713-DAA0E63E1A6A@arroway.org> X-Rspamd-Queue-Id: 4QCl7N4jFJz3jsV X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On May 5, 2023 2:23:15 AM GMT-03:00, Ganbold Tsagaankhuu wrote: >On Thu, May 4, 2023 at 10:05=E2=80=AFPM Matheus wro= te: > >> > On Tue, May 2, 2023 at 6:50=C3=A2=E2=82=AC=C2=AFPM Matheus wrote: >> > >> >> >> >> >> >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=83=C2=B8ren Schmidt" < >> >> soren=2Eschmidt@gmail=2Ecom> wrote: >> >> >> On 30 Apr 2023, at 23=2E44, Matheus wrote: >> >> >> >> >> >> Hi, >> >> >> >> >> >> I am trying to have FreeBSD 14 running on this SBC=2E I could not >find >> >> >any guides in how to build images for it=2E I found the >> >> >people=2Efreebsd=2Eorg/~sos/ site that has some images, and one for >it, >> but >> >> >that seldom boots my board, and when it did, there was no console >over >> >> >serial or vga=2E >> >> >> >> >> >> If anyone can give any hints=2E Unfortunately my dev skills are >not >> >> >good=2E But I can test and help build once I figure out how :) >> >> > >> >> >Hi Matheus >> >> > >> >> >The image at >> >> > >> >> > >> >> >> >https://people=2Efreebsd=2Eorg/~sos/ARM64/current-RK356X-images/nano5-sdc= ard=2Eimg=2Egz >> >> > >> >> >for the Nanopi R5S does indeed boot with both HDMI output and >serial >> >> >console (1500000baud)=2E >> >> > >> >> >The boot loader (EDK2 in FDT mode) is very picky on SD card >quality >> >> >though from experience, I works for me with Sandisk Ultra / >Extreme >> >> >cards but not with Samsung and cheap noname SD cards YMMV=2E >> >> >> >> Hi S=C3=83=C2=B8ren, >> >> >> >> I had really issues on sd carda=2E I got it to boot once, but I was >> >> printing >> >> characters on screen at one per second=2E So I rebooted and don't >remember >> >> why rewrote the card=2E I can't boot anymore=2E Tried different card= s, >> >> SanDisk >> >> ultra, no luck=2E >> >> >> >> I can boot an 13=2E2 image from the guy at personalbsd though=2E But >there I >> >> have just one ethernet=2E >> >> On 13=2E2 I cannot list the ethernets nics using pciconf -lv, >including >> >> the >> >> one that works=2E Is this expected? >> > >> > >> > Yes=2E Did you try >> > >> >https://personalbsd=2Eorg/download/Business/FreeBSD-aarch64-14=2E0-CURREN= T-NanoPi-R5S-20230402=2Eimg=2Exz >> > ? >> > This image should have support for pcie and all ethernet should >work >> IIRC=2E >> > >> > Ganbold >> >> Hi Ganbold, >> >> I tried it and other 2 images and no success, including the image >pointed >> by Soren=2E Unfortunately I just got to boot one image from 14 but I >> overwritten the sd card and can't remember which sd card and image :( >> >> The only image I can make it boot is from 13=2E2R, and I got it >installed >> fine (using an EFI image from personalbsd from March 23)=2E >> >> Will wait for the next round of images for 14=2E >> > >Above image works for me: > >root@NanoPi-R5S:~ # ifconfig >re0: flags=3D8803 metric 0 mtu 1500 > options=3D201b > ether 62:73:64:e2:d4:87 > inet 192=2E168=2E111=2E2 netmask 0xffffff00 broadcast 192=2E168= =2E111=2E255 > media: Ethernet autoselect > status: no carrier > nd6 options=3D29 >re1: flags=3D8843 metric 0 mtu >1500 > options=3D201b > ether 62:73:64:9a:5e:55 > inet 192=2E168=2E2=2E2 netmask 0xffffff00 broadcast 192=2E168=2E2= =2E255 > media: Ethernet autoselect (1000baseT ) > status: active > nd6 options=3D29 >eq0: flags=3D8843 metric 0 mtu >1500 > options=3D80008 > ether 22:07:01:bb:41:76 > inet 192=2E168=2E1=2E202 netmask 0xffffff00 broadcast 192=2E168= =2E1=2E255 > media: Ethernet autoselect (1000baseT ) > status: active > nd6 options=3D29 >lo0: flags=3D8049 metric 0 mtu 16384 > options=3D680003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 > inet 127=2E0=2E0=2E1 netmask 0xff000000 > groups: lo > nd6 options=3D21 >root@NanoPi-R5S:~ # uname -an >FreeBSD NanoPi-R5S 14=2E0-CURRENT FreeBSD 14=2E0-CURRENT #0 >main-n261950-4aeb939ecf8b-dirty: Sun Apr 2 17:42:01 MSK 2023 >root@honeycomb=2Elocal:/usr/crochet/work/obj/usr/crochet/src-current-14= =2E0/arm64=2Eaarch64/sys/EXPERT >arm64 >root@NanoPi-R5S:~ # kldstat >Id Refs Address Size Name > 1 8 0xffff000000000000 10cb350 kernel > 2 1 0xffff0000010cd000 a9508 if_re=2Eko > 3 1 0xffff0000c9e00000 24000 fdescfs=2Eko > 4 1 0xffff0000c9e24000 22000 mac_ntpd=2Eko >root@NanoPi-R5S:~ # > >Ganbold Hi Ganbold and others,=20 I gave another try today and I got to see some stuff on the serial console= but the serial terminal itself is not usable=2E It prints unrecognizable c= haracters, but I could recognize the red beastie logo=2E=20 On the hdmi screen I have no output=2E Can you confirm the serial speed li= ne?=20 It was 115200 when I saw all that, and then tried 1500000 and despite it w= as printing something that looks like the boot, just weird characters appea= red=2E=20 Thanks again,=20 Matheus=20 >> >> Thanks, >> >> matheus >> >> >> >> >> I have little understanding of the arch, so my progress is much >slow=2E >> >> I got some dmesg from OpenBSD people where the nics show in >ifconfig=2E >> >> But >> >> I couldn't get mine to behave this way=2E I can install though, >using USB >> >> nic=2E >> >> I will try to buy a new sd card from the good list you pointed=2E >> >> Another thing, I got the feeling that when I dd'ed the image using >the >> >> SD >> >> card slot on the notebook it worked and when was through usb >adapter did >> >> not=2E Does it make sense? Using Linux mint as host for this=2E >> >> Thanks so much for the answer and help, >> >> >> >> Matheus >> >> >> >> >> >> >You can build a stock ARM64 generic kernel and most things will >be >> >> >usable, however as Ganbold wrote the DTS files is not in there >yet (and >> >> >not even in linux where our DTS files are fetched from)=2E >> >> >However the EDK2 boot loader provided (and used in above image) >on >> >> > >> >> >>https://people=2Efreebsd=2Eorg/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI=2Ei= tb=C3=AF >> =C2=BF=C2=BC >> >> >NANOPI-R5S_EFI >> >> >File =C3=82=C2=B7 1,7 MB >> >> > >> >> >does hand over the =C3=A2=E2=82=AC=C5=93right=C3=A2=E2=82=AC DTB fi= le if you want to experiment=2E >> >> > >> >> >If you need the used DTS file and build guidance let me know in >private >> >> >mail=2E=2E=2E >> >> > >> >> >-- >> >> >S=C3=83=C2=B8ren Schmidt >> >> >sos@deepcore=2Edk / sos@freebsd=2Eorg >> >> >"So much code to hack, so little time" >> >> >> >> --- >> >> "We will call you Cygnus, >> >> the God of balance you shall be=2E" >> >> >> >> >> > >> >> >> -- >> "We will call you Cygnus, >> the God of balance you shall be=2E" >> ------------------------------ >> "We will call you Cygnus, >> the God of balance you shall be=2E" --- "We will call you Cygnus, the God of balance you shall be=2E" From nobody Sat May 6 02:03:47 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QCrSy4Zpsz4B54L for ; Sat, 6 May 2023 02:04:02 +0000 (UTC) (envelope-from ganbold@gmail.com) Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QCrSy0bxFz4BLl for ; Sat, 6 May 2023 02:04:02 +0000 (UTC) (envelope-from ganbold@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-33119abae99so40079265ab.0 for ; Fri, 05 May 2023 19:04:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683338640; x=1685930640; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=iBC+lnJgcqCjGh805IV7nDRiw49sEGeivqY1zsQkacM=; b=HwtQBTXfS6gAmwDB1X1O4NRNBJnjSU/Ig3t89R3b6k9HH5ldQ6vob4EkIXWm1HcPLM V7rwdmVIXBaAvJQqGgfVyT/MlmCyfXXD1WV7sUlEeoWdp5FgrHuzUNOFgVJrqMXBHBsu C/bO3MQAYkce4EXVa5WVn4hZw0+1rQYLS7JQu8nh5SmUQFuTMdTqRGXMguVOSekh4QN8 zQY/2j72ux8BPHKVE6NLd/miSSH0sOEyuAigCGT+pASf0mR2AYF8ou88MXsY2cQEBeeN Er2Dm3cTEtR3CUvVEU0RzHXTFVv+fX9tAVJVmARHCD3if6S9tTlsnHMjSv8WWuXRAuUB /fRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683338640; x=1685930640; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=iBC+lnJgcqCjGh805IV7nDRiw49sEGeivqY1zsQkacM=; b=hvuP7/GMt7HXRXZXGO3loluLWeKN0RbjPr7ounVNLYusitd9NbMZQFN3m3Q6qIKYUN 7mJ6qBw7aUP6n5aH47324F4jChqS+CtdkjNZN/U3q41rO92hjT6S6dnmQrfCn46iYUsB SoHFxfwzyc20OfdeUCCOzif5mlkE4WRMZ+cYEeCZlHgadmBQ3tYr2Yn4xOlO8hFqNo42 dRfR2HcWn9lZBpYrEGH+mmDxa/TqJIpuCMFckWcYh56N05pOpZh4FGpVoF+T+zdKzIhE BG36ep1pz4vT1DshNY+zQKWtISX+DUtoxZQOKOdQc7xCBerzUIKuR4EPLZsvC2+FM5ek veqw== X-Gm-Message-State: AC+VfDy3JeOh9xktOQYl6En3qZfLl8PqMh2Fbgd74mlueS6xMt1CX592 WHIRDbcb2uQ/mSbCwQZLGWp9T82OSaIFIxhGqs5qfAf7wbIrbw== X-Google-Smtp-Source: ACHHUZ4Ng/o9NoGpchcLfG8dW9ivv5mStJypeQeiJsrYw0+P5STPC5prJ9T8GUa7Sh9dPHpGkokpPk0VBgqzHUghmm0= X-Received: by 2002:a05:6602:4141:b0:763:5548:c53a with SMTP id bv1-20020a056602414100b007635548c53amr2947860iob.8.1683338640274; Fri, 05 May 2023 19:04:00 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <7F68ABB3-8D7A-40EA-9713-DAA0E63E1A6A@arroway.org> In-Reply-To: <7F68ABB3-8D7A-40EA-9713-DAA0E63E1A6A@arroway.org> From: Ganbold Tsagaankhuu Date: Sat, 6 May 2023 10:03:47 +0800 Message-ID: Subject: Re: Nanopi R5S support and build guide To: Matheus Cc: freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="000000000000b0b8d505fafcd1d8" X-Rspamd-Queue-Id: 4QCrSy0bxFz4BLl X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000b0b8d505fafcd1d8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, May 6, 2023 at 6:03=E2=80=AFAM Matheus wrote: > > > On May 5, 2023 2:23:15 AM GMT-03:00, Ganbold Tsagaankhuu < > ganbold@gmail.com> wrote: > >On Thu, May 4, 2023 at 10:05=E2=80=AFPM Matheus wrot= e: > > > >> > On Tue, May 2, 2023 at 6:50=C3=A2=E2=82=AC=C2=AFPM Matheus wrote: > >> > > >> >> > >> >> > >> >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=83=C2=B8ren Schmidt" < > >> >> soren.schmidt@gmail.com> wrote: > >> >> >> On 30 Apr 2023, at 23.44, Matheus wrote: > >> >> >> > >> >> >> Hi, > >> >> >> > >> >> >> I am trying to have FreeBSD 14 running on this SBC. I could not > >find > >> >> >any guides in how to build images for it. I found the > >> >> >people.freebsd.org/~sos/ site that has some images, and one for > >it, > >> but > >> >> >that seldom boots my board, and when it did, there was no console > >over > >> >> >serial or vga. > >> >> >> > >> >> >> If anyone can give any hints. Unfortunately my dev skills are > >not > >> >> >good. But I can test and help build once I figure out how :) > >> >> > > >> >> >Hi Matheus > >> >> > > >> >> >The image at > >> >> > > >> >> > > >> >> > >> > > > https://people.freebsd.org/~sos/ARM64/current-RK356X-images/nano5-sdcard.= img.gz > >> >> > > >> >> >for the Nanopi R5S does indeed boot with both HDMI output and > >serial > >> >> >console (1500000baud). > >> >> > > >> >> >The boot loader (EDK2 in FDT mode) is very picky on SD card > >quality > >> >> >though from experience, I works for me with Sandisk Ultra / > >Extreme > >> >> >cards but not with Samsung and cheap noname SD cards YMMV. > >> >> > >> >> Hi S=C3=83=C2=B8ren, > >> >> > >> >> I had really issues on sd carda. I got it to boot once, but I was > >> >> printing > >> >> characters on screen at one per second. So I rebooted and don't > >remember > >> >> why rewrote the card. I can't boot anymore. Tried different cards, > >> >> SanDisk > >> >> ultra, no luck. > >> >> > >> >> I can boot an 13.2 image from the guy at personalbsd though. But > >there I > >> >> have just one ethernet. > >> >> On 13.2 I cannot list the ethernets nics using pciconf -lv, > >including > >> >> the > >> >> one that works. Is this expected? > >> > > >> > > >> > Yes. Did you try > >> > > >> > > > https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Na= noPi-R5S-20230402.img.xz > >> > ? > >> > This image should have support for pcie and all ethernet should > >work > >> IIRC. > >> > > >> > Ganbold > >> > >> Hi Ganbold, > >> > >> I tried it and other 2 images and no success, including the image > >pointed > >> by Soren. Unfortunately I just got to boot one image from 14 but I > >> overwritten the sd card and can't remember which sd card and image :( > >> > >> The only image I can make it boot is from 13.2R, and I got it > >installed > >> fine (using an EFI image from personalbsd from March 23). > >> > >> Will wait for the next round of images for 14. > >> > > > >Above image works for me: > > > >root@NanoPi-R5S:~ # ifconfig > >re0: flags=3D8803 metric 0 mtu 1500 > > options=3D201b > > ether 62:73:64:e2:d4:87 > > inet 192.168.111.2 netmask 0xffffff00 broadcast 192.168.111.255 > > media: Ethernet autoselect > > status: no carrier > > nd6 options=3D29 > >re1: flags=3D8843 metric 0 mtu > >1500 > > options=3D201b > > ether 62:73:64:9a:5e:55 > > inet 192.168.2.2 netmask 0xffffff00 broadcast 192.168.2.255 > > media: Ethernet autoselect (1000baseT ) > > status: active > > nd6 options=3D29 > >eq0: flags=3D8843 metric 0 mtu > >1500 > > options=3D80008 > > ether 22:07:01:bb:41:76 > > inet 192.168.1.202 netmask 0xffffff00 broadcast 192.168.1.255 > > media: Ethernet autoselect (1000baseT ) > > status: active > > nd6 options=3D29 > >lo0: flags=3D8049 metric 0 mtu 16384 > > options=3D680003 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 > > inet 127.0.0.1 netmask 0xff000000 > > groups: lo > > nd6 options=3D21 > >root@NanoPi-R5S:~ # uname -an > >FreeBSD NanoPi-R5S 14.0-CURRENT FreeBSD 14.0-CURRENT #0 > >main-n261950-4aeb939ecf8b-dirty: Sun Apr 2 17:42:01 MSK 2023 > >root@honeycomb.local > :/usr/crochet/work/obj/usr/crochet/src-current-14.0/arm64.aarch64/sys/EXP= ERT > >arm64 > >root@NanoPi-R5S:~ # kldstat > >Id Refs Address Size Name > > 1 8 0xffff000000000000 10cb350 kernel > > 2 1 0xffff0000010cd000 a9508 if_re.ko > > 3 1 0xffff0000c9e00000 24000 fdescfs.ko > > 4 1 0xffff0000c9e24000 22000 mac_ntpd.ko > >root@NanoPi-R5S:~ # > > > >Ganbold > > Hi Ganbold and others, > > I gave another try today and I got to see some stuff on the serial consol= e > but the serial terminal itself is not usable. It prints unrecognizable > characters, but I could recognize the red beastie logo. > On the hdmi screen I have no output. Can you confirm the serial speed > line? > It was 115200 when I saw all that, and then tried 1500000 and despite it > was printing something that looks like the boot, just weird characters > appeared. With https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Nano= Pi-R5S-20230402.img.xz image I use 115200. Ganbold > > > Thanks again, > > Matheus > > > >> > >> Thanks, > >> > >> matheus > >> > >> >> > >> >> I have little understanding of the arch, so my progress is much > >slow. > >> >> I got some dmesg from OpenBSD people where the nics show in > >ifconfig. > >> >> But > >> >> I couldn't get mine to behave this way. I can install though, > >using USB > >> >> nic. > >> >> I will try to buy a new sd card from the good list you pointed. > >> >> Another thing, I got the feeling that when I dd'ed the image using > >the > >> >> SD > >> >> card slot on the notebook it worked and when was through usb > >adapter did > >> >> not. Does it make sense? Using Linux mint as host for this. > >> >> Thanks so much for the answer and help, > >> >> > >> >> Matheus > >> >> > >> >> > >> >> >You can build a stock ARM64 generic kernel and most things will > >be > >> >> >usable, however as Ganbold wrote the DTS files is not in there > >yet (and > >> >> >not even in linux where our DTS files are fetched from). > >> >> >However the EDK2 boot loader provided (and used in above image) > >on > >> >> > > >> >> > >>https://people.freebsd.org/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb=C3= =AF > >> =C2=BF=C2=BC > >> >> >NANOPI-R5S_EFI > >> >> >File =C3=82=C2=B7 1,7 MB > >> >> > > >> >> >does hand over the =C3=A2=E2=82=AC=C5=93right=C3=A2=E2=82=AC DTB f= ile if you want to experiment. > >> >> > > >> >> >If you need the used DTS file and build guidance let me know in > >private > >> >> >mail... > >> >> > > >> >> >-- > >> >> >S=C3=83=C2=B8ren Schmidt > >> >> >sos@deepcore.dk / sos@freebsd.org > >> >> >"So much code to hack, so little time" > >> >> > >> >> --- > >> >> "We will call you Cygnus, > >> >> the God of balance you shall be." > >> >> > >> >> > >> > > >> > >> > >> -- > >> "We will call you Cygnus, > >> the God of balance you shall be." > >> ------------------------------ > >> "We will call you Cygnus, > >> the God of balance you shall be." > > --- > "We will call you Cygnus, > the God of balance you shall be." > --000000000000b0b8d505fafcd1d8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sat, May 6, 2023 at 6:03=E2=80=AFA= M Matheus <lojas@arroway.org>= ; wrote:


On May 5, 2023 2:23:15 AM GMT-03:00, Ganbold Tsagaankhuu <ganbold@gmail.com> wrote: >On Thu, May 4, 2023 at 10:05=E2=80=AFPM Matheus <lojas@arroway.org> wrote:
>
>> > On Tue, May 2, 2023 at 6:50=C3=A2=E2=82=AC=C2=AFPM Matheus &l= t;lojas@arroway.org<= /a>> wrote:
>> >
>> >>
>> >>
>> >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=83=C2=B8r= en Schmidt" <
>> >> soren.schmidt@gmail.com> wrote:
>> >> >> On 30 Apr 2023, at 23.44, Matheus <lojas@arroway.org> wrot= e:
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> I am trying to have FreeBSD 14 running on this S= BC. I could not
>find
>> >> >any guides in how to build images for it. I found the=
>> >> >people.freebsd.org/~sos/ site that has some= images, and one for
>it,
>> but
>> >> >that seldom boots my board, and when it did, there wa= s no console
>over
>> >> >serial or vga.
>> >> >>
>> >> >> If anyone can give any hints. Unfortunately my d= ev skills are
>not
>> >> >good. But I can test and help build once I figure out= how :)
>> >> >
>> >> >Hi Matheus
>> >> >
>> >> >The image at
>> >> >
>> >> >
>> >>
>>
>https://people.fr= eebsd.org/~sos/ARM64/current-RK356X-images/nano5-sdcard.img.gz
>> >> >
>> >> >for the Nanopi R5S does indeed boot with both HDMI ou= tput and
>serial
>> >> >console (1500000baud).
>> >> >
>> >> >The boot loader (EDK2 in FDT mode) is very picky on S= D card
>quality
>> >> >though from experience, I works for me with Sandisk U= ltra /
>Extreme
>> >> >cards but not with Samsung and cheap noname SD cards = YMMV.
>> >>
>> >> Hi S=C3=83=C2=B8ren,
>> >>
>> >> I had really issues on sd carda. I got it to boot once, b= ut I was
>> >> printing
>> >> characters on screen at one per second. So I rebooted and= don't
>remember
>> >> why rewrote the card. I can't boot anymore. Tried dif= ferent cards,
>> >> SanDisk
>> >> ultra, no luck.
>> >>
>> >> I can boot an 13.2 image from the guy at personalbsd thou= gh. But
>there I
>> >> have just one ethernet.
>> >> On 13.2 I cannot list the ethernets nics using pciconf -l= v,
>including
>> >> the
>> >> one that works. Is this expected?
>> >
>> >
>> > Yes. Did you try
>> >
>>
>https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Nan= oPi-R5S-20230402.img.xz
>> > ?
>> > This image should have support for pcie and all ethernet shou= ld
>work
>> IIRC.
>> >
>> > Ganbold
>>
>> Hi Ganbold,
>>
>> I tried it and other 2 images and no success, including the image<= br> >pointed
>> by Soren. Unfortunately I just got to boot one image from 14 but I=
>> overwritten the sd card and can't remember which sd card and i= mage :(
>>
>> The only image I can make it boot is from 13.2R, and I got it
>installed
>> fine (using an EFI image from personalbsd from March 23).
>>
>> Will wait for the next round of images for 14.
>>
>
>Above image works for me:
>
>root@NanoPi-R5S:~ # ifconfig
>re0: flags=3D8803<UP,BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 15= 00
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D201b<RXCSUM,TXCSUM,VLAN_MTU,VL= AN_HWTAGGING,WOL_MAGIC>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 62:73:64:e2:d4:87
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 192.168.111.2 netmask 0xffffff00 broad= cast 192.168.111.255
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 status: no carrier
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D29<PERFORMNUD,IFDISABLED,A= UTO_LINKLOCAL>
>re1: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric = 0 mtu
>1500
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D201b<RXCSUM,TXCSUM,VLAN_MTU,VL= AN_HWTAGGING,WOL_MAGIC>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 62:73:64:9a:5e:55
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 192.168.2.2 netmask 0xffffff00 broadca= st 192.168.2.255
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000baseT <f= ull-duplex>)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 status: active
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D29<PERFORMNUD,IFDISABLED,A= UTO_LINKLOCAL>
>eq0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric = 0 mtu
>1500
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D80008<VLAN_MTU,LINKSTATE> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 22:07:01:bb:41:76
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 192.168.1.202 netmask 0xffffff00 broad= cast 192.168.1.255
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000baseT <f= ull-duplex>)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 status: active
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D29<PERFORMNUD,IFDISABLED,A= UTO_LINKLOCAL>
>lo0: flags=3D8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 163= 84
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D680003<RXCSUM,TXCSUM,LINKSTATE= ,RXCSUM_IPV6,TXCSUM_IPV6>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet6 ::1 prefixlen 128
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4<= br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 127.0.0.1 netmask 0xff000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 groups: lo
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D21<PERFORMNUD,AUTO_LINKLOC= AL>
>root@NanoPi-R5S:~ # uname -an
>FreeBSD NanoPi-R5S 14.0-CURRENT FreeBSD 14.0-CURRENT #0
>main-n261950-4aeb939ecf8b-dirty: Sun Apr=C2=A0 2 17:42:01 MSK 2023
>root@honeycomb.local:/usr/crochet/work/obj/usr/crochet/src-current-14.0= /arm64.aarch64/sys/EXPERT
>arm64
>root@NanoPi-R5S:~ # kldstat
>Id Refs Address=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = Size Name
> 1=C2=A0 =C2=A0 8 0xffff000000000000=C2=A0 10cb350 kernel
> 2=C2=A0 =C2=A0 1 0xffff0000010cd000=C2=A0 =C2=A0 a9508 if_re.ko
> 3=C2=A0 =C2=A0 1 0xffff0000c9e00000=C2=A0 =C2=A0 24000 fdescfs.ko
> 4=C2=A0 =C2=A0 1 0xffff0000c9e24000=C2=A0 =C2=A0 22000 mac_ntpd.ko
>root@NanoPi-R5S:~ #
>
>Ganbold

Hi Ganbold and others,

I gave another try today and I got to see some stuff on the serial console = but the serial terminal itself is not usable. It prints unrecognizable char= acters, but I could recognize the red beastie logo.
On the hdmi screen I have no output. Can you confirm the serial speed line?=
It was 115200 when I saw all that, and then tried 1500000 and despite it wa= s printing something that looks like the boot, just weird characters appear= ed.


Ganbold

=C2=A0


Thanks again,

Matheus


>>
>> Thanks,
>>
>> matheus
>>
>> >>
>> >> I have little understanding of the arch, so my progress i= s much
>slow.
>> >> I got some dmesg from OpenBSD people where the nics show = in
>ifconfig.
>> >> But
>> >> I couldn't get mine to behave this way. I can install= though,
>using USB
>> >> nic.
>> >> I will try to buy a new sd card from the good list you po= inted.
>> >> Another thing, I got the feeling that when I dd'ed th= e image using
>the
>> >> SD
>> >> card slot on the notebook it worked and when was through = usb
>adapter did
>> >> not. Does it make sense? Using Linux mint as host for thi= s.
>> >> Thanks so much for the answer and help,
>> >>
>> >> Matheus
>> >>
>> >>
>> >> >You can build a stock ARM64 generic kernel and most t= hings will
>be
>> >> >usable, however as Ganbold wrote the DTS files is not= in there
>yet (and
>> >> >not even in linux where our DTS files are fetched fro= m).
>> >> >However the EDK2 boot loader provided (and used in ab= ove image)
>on
>> >> >
>> >>
>>https://people.fre= ebsd.org/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb=C3=AF
>> =C2=BF=C2=BC
>> >> >NANOPI-R5S_EFI
>> >> >File =C3=82=C2=B7 1,7 MB
>> >> >
>> >> >does hand over the =C3=A2=E2=82=AC=C5=93right=C3=A2= =E2=82=AC DTB file if you want to experiment.
>> >> >
>> >> >If you need the used DTS file and build guidance let = me know in
>private
>> >> >mail...
>> >> >
>> >> >--
>> >> >S=C3=83=C2=B8ren Schmidt
>> >> >= sos@deepcore.dk / = sos@freebsd.org
>> >> >"So much code to hack, so little time"
>> >>
>> >> ---
>> >> "We will call you Cygnus,
>> >> the God of balance you shall be."
>> >>
>> >>
>> >
>>
>>
>> --
>> "We will call you Cygnus,
>> the God of balance you shall be."
>> ------------------------------
>> "We will call you Cygnus,
>> the God of balance you shall be."

---
"We will call you Cygnus,
the God of balance you shall be."
--000000000000b0b8d505fafcd1d8-- From nobody Sat May 6 05:55:38 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QCxcL0dqxz495XP for ; Sat, 6 May 2023 05:55:46 +0000 (UTC) (envelope-from lojas@arroway.org) Received: from hobbes.arroway.org (hobbes.arroway.org [173.199.118.77]) by mx1.freebsd.org (Postfix) with ESMTP id 4QCxcK5gtWz3FgN for ; Sat, 6 May 2023 05:55:45 +0000 (UTC) (envelope-from lojas@arroway.org) Authentication-Results: mx1.freebsd.org; none Received: from [10.28.86.191] (179-240-9-46.3g.claro.net.br [179.240.9.46]) by hobbes.arroway.org (Postfix) with ESMTPA id 4240514D29A; Sat, 6 May 2023 02:55:44 -0300 (-03) Date: Sat, 06 May 2023 02:55:38 -0300 User-Agent: K-9 Mail for Android In-Reply-To: References: <7F68ABB3-8D7A-40EA-9713-DAA0E63E1A6A@arroway.org> List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Nanopi R5S support and build guide To: freebsd-arm@freebsd.org,Ganbold Tsagaankhuu From: Matheus Message-ID: X-Rspamd-Queue-Id: 4QCxcK5gtWz3FgN X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:173.199.116.0/22, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On May 5, 2023 11:03:47 PM GMT-03:00, Ganbold Tsagaankhuu wrote: >On Sat, May 6, 2023 at 6:03=E2=80=AFAM Matheus wrot= e: > >> >> >> On May 5, 2023 2:23:15 AM GMT-03:00, Ganbold Tsagaankhuu < >> ganbold@gmail=2Ecom> wrote: >> >On Thu, May 4, 2023 at 10:05=E2=80=AFPM Matheus = wrote: >> > >> >> > On Tue, May 2, 2023 at 6:50=C3=A2=E2=82=AC=C2=AFPM Matheus >wrote: >> >> > >> >> >> >> >> >> >> >> >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=83=C2=B8ren Schmidt" < >> >> >> soren=2Eschmidt@gmail=2Ecom> wrote: >> >> >> >> On 30 Apr 2023, at 23=2E44, Matheus wrot= e: >> >> >> >> >> >> >> >> Hi, >> >> >> >> >> >> >> >> I am trying to have FreeBSD 14 running on this SBC=2E I could >not >> >find >> >> >> >any guides in how to build images for it=2E I found the >> >> >> >people=2Efreebsd=2Eorg/~sos/ site that has some images, and one >for >> >it, >> >> but >> >> >> >that seldom boots my board, and when it did, there was no >console >> >over >> >> >> >serial or vga=2E >> >> >> >> >> >> >> >> If anyone can give any hints=2E Unfortunately my dev skills >are >> >not >> >> >> >good=2E But I can test and help build once I figure out how :) >> >> >> > >> >> >> >Hi Matheus >> >> >> > >> >> >> >The image at >> >> >> > >> >> >> > >> >> >> >> >> >> > >> >https://people=2Efreebsd=2Eorg/~sos/ARM64/current-RK356X-images/nano5-sdc= ard=2Eimg=2Egz >> >> >> > >> >> >> >for the Nanopi R5S does indeed boot with both HDMI output and >> >serial >> >> >> >console (1500000baud)=2E >> >> >> > >> >> >> >The boot loader (EDK2 in FDT mode) is very picky on SD card >> >quality >> >> >> >though from experience, I works for me with Sandisk Ultra / >> >Extreme >> >> >> >cards but not with Samsung and cheap noname SD cards YMMV=2E >> >> >> >> >> >> Hi S=C3=83=C2=B8ren, >> >> >> >> >> >> I had really issues on sd carda=2E I got it to boot once, but I >was >> >> >> printing >> >> >> characters on screen at one per second=2E So I rebooted and don't >> >remember >> >> >> why rewrote the card=2E I can't boot anymore=2E Tried different >cards, >> >> >> SanDisk >> >> >> ultra, no luck=2E >> >> >> >> >> >> I can boot an 13=2E2 image from the guy at personalbsd though=2E >But >> >there I >> >> >> have just one ethernet=2E >> >> >> On 13=2E2 I cannot list the ethernets nics using pciconf -lv, >> >including >> >> >> the >> >> >> one that works=2E Is this expected? >> >> > >> >> > >> >> > Yes=2E Did you try >> >> > >> >> >> > >> >https://personalbsd=2Eorg/download/Business/FreeBSD-aarch64-14=2E0-CURREN= T-NanoPi-R5S-20230402=2Eimg=2Exz >> >> > ? >> >> > This image should have support for pcie and all ethernet should >> >work >> >> IIRC=2E >> >> > >> >> > Ganbold >> >> >> >> Hi Ganbold, >> >> >> >> I tried it and other 2 images and no success, including the image >> >pointed >> >> by Soren=2E Unfortunately I just got to boot one image from 14 but I >> >> overwritten the sd card and can't remember which sd card and image >:( >> >> >> >> The only image I can make it boot is from 13=2E2R, and I got it >> >installed >> >> fine (using an EFI image from personalbsd from March 23)=2E >> >> >> >> Will wait for the next round of images for 14=2E >> >> >> > >> >Above image works for me: >> > >> >root@NanoPi-R5S:~ # ifconfig >> >re0: flags=3D8803 metric 0 mtu 1500 >> > =20 >options=3D201b >> > ether 62:73:64:e2:d4:87 >> > inet 192=2E168=2E111=2E2 netmask 0xffffff00 broadcast >192=2E168=2E111=2E255 >> > media: Ethernet autoselect >> > status: no carrier >> > nd6 options=3D29 >> >re1: flags=3D8843 metric 0 mtu >> >1500 >> > =20 >options=3D201b >> > ether 62:73:64:9a:5e:55 >> > inet 192=2E168=2E2=2E2 netmask 0xffffff00 broadcast 192=2E168= =2E2=2E255 >> > media: Ethernet autoselect (1000baseT ) >> > status: active >> > nd6 options=3D29 >> >eq0: flags=3D8843 metric 0 mtu >> >1500 >> > options=3D80008 >> > ether 22:07:01:bb:41:76 >> > inet 192=2E168=2E1=2E202 netmask 0xffffff00 broadcast >192=2E168=2E1=2E255 >> > media: Ethernet autoselect (1000baseT ) >> > status: active >> > nd6 options=3D29 >> >lo0: flags=3D8049 metric 0 mtu 16384 >> > =20 >options=3D680003 >> > inet6 ::1 prefixlen 128 >> > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 >> > inet 127=2E0=2E0=2E1 netmask 0xff000000 >> > groups: lo >> > nd6 options=3D21 >> >root@NanoPi-R5S:~ # uname -an >> >FreeBSD NanoPi-R5S 14=2E0-CURRENT FreeBSD 14=2E0-CURRENT #0 >> >main-n261950-4aeb939ecf8b-dirty: Sun Apr 2 17:42:01 MSK 2023 >> >root@honeycomb=2Elocal >> >:/usr/crochet/work/obj/usr/crochet/src-current-14=2E0/arm64=2Eaarch64/sys= /EXPERT >> >arm64 >> >root@NanoPi-R5S:~ # kldstat >> >Id Refs Address Size Name >> > 1 8 0xffff000000000000 10cb350 kernel >> > 2 1 0xffff0000010cd000 a9508 if_re=2Eko >> > 3 1 0xffff0000c9e00000 24000 fdescfs=2Eko >> > 4 1 0xffff0000c9e24000 22000 mac_ntpd=2Eko >> >root@NanoPi-R5S:~ # >> > >> >Ganbold >> >> Hi Ganbold and others, >> >> I gave another try today and I got to see some stuff on the serial >console >> but the serial terminal itself is not usable=2E It prints >unrecognizable >> characters, but I could recognize the red beastie logo=2E >> On the hdmi screen I have no output=2E Can you confirm the serial speed >> line? >> It was 115200 when I saw all that, and then tried 1500000 and despite >it >> was printing something that looks like the boot, just weird >characters >> appeared=2E > > >With >https://personalbsd=2Eorg/download/Business/FreeBSD-aarch64-14=2E0-CURREN= T-NanoPi-R5S-20230402=2Eimg=2Exz >image >I use 115200=2E > >Ganbold Thanks=2E Tried earlier but no luck=2E=20 Could you send me a dmesg? I think much of it is in the acpi/dtb issue=2E = I would like to see how your dmesg shows=2E Does it have regular hdmi output? As there is in 13=2E2? Thanks,=20 Matheus >> >> >> Thanks again, >> >> Matheus >> >> >> >> >> >> Thanks, >> >> >> >> matheus >> >> >> >> >> >> >> >> I have little understanding of the arch, so my progress is much >> >slow=2E >> >> >> I got some dmesg from OpenBSD people where the nics show in >> >ifconfig=2E >> >> >> But >> >> >> I couldn't get mine to behave this way=2E I can install though, >> >using USB >> >> >> nic=2E >> >> >> I will try to buy a new sd card from the good list you pointed=2E >> >> >> Another thing, I got the feeling that when I dd'ed the image >using >> >the >> >> >> SD >> >> >> card slot on the notebook it worked and when was through usb >> >adapter did >> >> >> not=2E Does it make sense? Using Linux mint as host for this=2E >> >> >> Thanks so much for the answer and help, >> >> >> >> >> >> Matheus >> >> >> >> >> >> >> >> >> >You can build a stock ARM64 generic kernel and most things >will >> >be >> >> >> >usable, however as Ganbold wrote the DTS files is not in there >> >yet (and >> >> >> >not even in linux where our DTS files are fetched from)=2E >> >> >> >However the EDK2 boot loader provided (and used in above >image) >> >on >> >> >> > >> >> >> >> >>>https://people=2Efreebsd=2Eorg/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI=2E= itb=C3=AF >> >> =C2=BF=C2=BC >> >> >> >NANOPI-R5S_EFI >> >> >> >File =C3=82=C2=B7 1,7 MB >> >> >> > >> >> >> >does hand over the =C3=A2=E2=82=AC=C5=93right=C3=A2=E2=82=AC DTB= file if you want to >experiment=2E >> >> >> > >> >> >> >If you need the used DTS file and build guidance let me know >in >> >private >> >> >> >mail=2E=2E=2E >> >> >> > >> >> >> >-- >> >> >> >S=C3=83=C2=B8ren Schmidt >> >> >> >sos@deepcore=2Edk / sos@freebsd=2Eorg >> >> >> >"So much code to hack, so little time" >> >> >> >> >> >> --- >> >> >> "We will call you Cygnus, >> >> >> the God of balance you shall be=2E" >> >> >> >> >> >> >> >> > >> >> >> >> >> >> -- >> >> "We will call you Cygnus, >> >> the God of balance you shall be=2E" >> >> ------------------------------ >> >> "We will call you Cygnus, >> >> the God of balance you shall be=2E" >> >> --- >> "We will call you Cygnus, >> the God of balance you shall be=2E" >> --- "We will call you Cygnus, the God of balance you shall be=2E" From nobody Sat May 6 06:00:51 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QCxkS6Plcz495lG for ; Sat, 6 May 2023 06:01:04 +0000 (UTC) (envelope-from ganbold@gmail.com) Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QCxkS4GLCz3GDR for ; Sat, 6 May 2023 06:01:04 +0000 (UTC) (envelope-from ganbold@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-333eb36e453so2678265ab.3 for ; Fri, 05 May 2023 23:01:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683352863; x=1685944863; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=sgKMcxJpMZeaQ/5VAzrT/HxXBNxZ4s/gjlYXpw/2srM=; b=Duuillkx7MdA2AYH91KE/sVoiHEC+q4DairjsdVNhl263mz6uRy1vyATEZlzOeQTcm 9CHtpcXIG3saEhQVWrS/EwqdVOd/mr5XrkaPh0A7pNoC4GYEMD9mBJwM5VfOJx2Bms4j tKDZeWGdGC/oTA6uXjGYVXN19omMwY6a5utkboh642uUnDDX79C8CAv8iMUfIYDTVUCd cl8pAKHb9CeYXOEWytPvHUM6MZkLBpy/Ooc/5DwMfRuQgf2l/vrSHIuURTOQDUeddncS KCWl14Wis80P2tDU8gNI0lMJ0ermPJj8W3CQa1Xzk0Qwa0j2b47YJvmvihmmKO15lSTG qzlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683352863; x=1685944863; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=sgKMcxJpMZeaQ/5VAzrT/HxXBNxZ4s/gjlYXpw/2srM=; b=V4lzAgXgVkzCaY9u0wqsqeLijuGljcJxCCI3b7zaV2Ka6HGmFrOnNoIvxjzNLecXwC Pun5rrUIc2SA5xSSGG0V8H8Pcw+Bu2SYmoJ6SeIESTWfL74RKduThv5NgLOksrHy+4yw KmbJxLsyOu7GFBZhenbD2PIqDFg73wWurccLnUoYDaD7Vpcc66/zeZcCSTGZPZ61fTIT pf9GoWt3dciVckDAdUwxwp3PVTDQo+67teL2vMwPyJXxRf8S6Y3FL1haqnCeox5hbe9i tBxvn4x2OJkt2Fk0VYUjo6YWDVRR6Y6Fj1/jOfROC9wGLQjFx0wbssV6+l1G7b2vbtcx CKaA== X-Gm-Message-State: AC+VfDwFw1Ng2XrDgBesPoIQpygzgmp53nEcuR6mxnyT3e0UFVaUzrpi TIySplf8hzAeLg5oXEWeq/f5Pz0Asg8tOKNXfsbHJnfWMYmvcw== X-Google-Smtp-Source: ACHHUZ64gdIIuvN86m0GRLFBaQT4Ii9Um5q+RWx+DnfOLgVRmUNiYe+Sac1L0Mu5M+TMbHMltPzDKCjRgQ7tKALNjTQ= X-Received: by 2002:a92:d30d:0:b0:331:50c4:12b2 with SMTP id x13-20020a92d30d000000b0033150c412b2mr2630081ila.16.1683352863465; Fri, 05 May 2023 23:01:03 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <7F68ABB3-8D7A-40EA-9713-DAA0E63E1A6A@arroway.org> In-Reply-To: From: Ganbold Tsagaankhuu Date: Sat, 6 May 2023 14:00:51 +0800 Message-ID: Subject: Re: Nanopi R5S support and build guide To: Matheus Cc: freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="00000000000075663405fb0021d8" X-Rspamd-Queue-Id: 4QCxkS4GLCz3GDR X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000075663405fb0021d8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, May 6, 2023 at 1:55=E2=80=AFPM Matheus wrote: > > > On May 5, 2023 11:03:47 PM GMT-03:00, Ganbold Tsagaankhuu < > ganbold@gmail.com> wrote: > >On Sat, May 6, 2023 at 6:03=E2=80=AFAM Matheus wrote= : > > > >> > >> > >> On May 5, 2023 2:23:15 AM GMT-03:00, Ganbold Tsagaankhuu < > >> ganbold@gmail.com> wrote: > >> >On Thu, May 4, 2023 at 10:05=E2=80=AFPM Matheus w= rote: > >> > > >> >> > On Tue, May 2, 2023 at 6:50=C3=A2=E2=82=AC=C2=AFPM Matheus > >wrote: > >> >> > > >> >> >> > >> >> >> > >> >> >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3=83=C2=B8ren Schmidt" = < > >> >> >> soren.schmidt@gmail.com> wrote: > >> >> >> >> On 30 Apr 2023, at 23.44, Matheus wrote: > >> >> >> >> > >> >> >> >> Hi, > >> >> >> >> > >> >> >> >> I am trying to have FreeBSD 14 running on this SBC. I could > >not > >> >find > >> >> >> >any guides in how to build images for it. I found the > >> >> >> >people.freebsd.org/~sos/ site that has some images, and one > >for > >> >it, > >> >> but > >> >> >> >that seldom boots my board, and when it did, there was no > >console > >> >over > >> >> >> >serial or vga. > >> >> >> >> > >> >> >> >> If anyone can give any hints. Unfortunately my dev skills > >are > >> >not > >> >> >> >good. But I can test and help build once I figure out how :) > >> >> >> > > >> >> >> >Hi Matheus > >> >> >> > > >> >> >> >The image at > >> >> >> > > >> >> >> > > >> >> >> > >> >> > >> > > >> > > > https://people.freebsd.org/~sos/ARM64/current-RK356X-images/nano5-sdcard.= img.gz > >> >> >> > > >> >> >> >for the Nanopi R5S does indeed boot with both HDMI output and > >> >serial > >> >> >> >console (1500000baud). > >> >> >> > > >> >> >> >The boot loader (EDK2 in FDT mode) is very picky on SD card > >> >quality > >> >> >> >though from experience, I works for me with Sandisk Ultra / > >> >Extreme > >> >> >> >cards but not with Samsung and cheap noname SD cards YMMV. > >> >> >> > >> >> >> Hi S=C3=83=C2=B8ren, > >> >> >> > >> >> >> I had really issues on sd carda. I got it to boot once, but I > >was > >> >> >> printing > >> >> >> characters on screen at one per second. So I rebooted and don't > >> >remember > >> >> >> why rewrote the card. I can't boot anymore. Tried different > >cards, > >> >> >> SanDisk > >> >> >> ultra, no luck. > >> >> >> > >> >> >> I can boot an 13.2 image from the guy at personalbsd though. > >But > >> >there I > >> >> >> have just one ethernet. > >> >> >> On 13.2 I cannot list the ethernets nics using pciconf -lv, > >> >including > >> >> >> the > >> >> >> one that works. Is this expected? > >> >> > > >> >> > > >> >> > Yes. Did you try > >> >> > > >> >> > >> > > >> > > > https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Na= noPi-R5S-20230402.img.xz > >> >> > ? > >> >> > This image should have support for pcie and all ethernet should > >> >work > >> >> IIRC. > >> >> > > >> >> > Ganbold > >> >> > >> >> Hi Ganbold, > >> >> > >> >> I tried it and other 2 images and no success, including the image > >> >pointed > >> >> by Soren. Unfortunately I just got to boot one image from 14 but I > >> >> overwritten the sd card and can't remember which sd card and image > >:( > >> >> > >> >> The only image I can make it boot is from 13.2R, and I got it > >> >installed > >> >> fine (using an EFI image from personalbsd from March 23). > >> >> > >> >> Will wait for the next round of images for 14. > >> >> > >> > > >> >Above image works for me: > >> > > >> >root@NanoPi-R5S:~ # ifconfig > >> >re0: flags=3D8803 metric 0 mtu 1500 > >> > > >options=3D201b > >> > ether 62:73:64:e2:d4:87 > >> > inet 192.168.111.2 netmask 0xffffff00 broadcast > >192.168.111.255 > >> > media: Ethernet autoselect > >> > status: no carrier > >> > nd6 options=3D29 > >> >re1: flags=3D8843 metric 0 mt= u > >> >1500 > >> > > >options=3D201b > >> > ether 62:73:64:9a:5e:55 > >> > inet 192.168.2.2 netmask 0xffffff00 broadcast 192.168.2.255 > >> > media: Ethernet autoselect (1000baseT ) > >> > status: active > >> > nd6 options=3D29 > >> >eq0: flags=3D8843 metric 0 mt= u > >> >1500 > >> > options=3D80008 > >> > ether 22:07:01:bb:41:76 > >> > inet 192.168.1.202 netmask 0xffffff00 broadcast > >192.168.1.255 > >> > media: Ethernet autoselect (1000baseT ) > >> > status: active > >> > nd6 options=3D29 > >> >lo0: flags=3D8049 metric 0 mtu 16384 > >> > > >options=3D680003 > >> > inet6 ::1 prefixlen 128 > >> > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 > >> > inet 127.0.0.1 netmask 0xff000000 > >> > groups: lo > >> > nd6 options=3D21 > >> >root@NanoPi-R5S:~ # uname -an > >> >FreeBSD NanoPi-R5S 14.0-CURRENT FreeBSD 14.0-CURRENT #0 > >> >main-n261950-4aeb939ecf8b-dirty: Sun Apr 2 17:42:01 MSK 2023 > >> >root@honeycomb.local > >> > > >:/usr/crochet/work/obj/usr/crochet/src-current-14.0/arm64.aarch64/sys/EX= PERT > >> >arm64 > >> >root@NanoPi-R5S:~ # kldstat > >> >Id Refs Address Size Name > >> > 1 8 0xffff000000000000 10cb350 kernel > >> > 2 1 0xffff0000010cd000 a9508 if_re.ko > >> > 3 1 0xffff0000c9e00000 24000 fdescfs.ko > >> > 4 1 0xffff0000c9e24000 22000 mac_ntpd.ko > >> >root@NanoPi-R5S:~ # > >> > > >> >Ganbold > >> > >> Hi Ganbold and others, > >> > >> I gave another try today and I got to see some stuff on the serial > >console > >> but the serial terminal itself is not usable. It prints > >unrecognizable > >> characters, but I could recognize the red beastie logo. > >> On the hdmi screen I have no output. Can you confirm the serial speed > >> line? > >> It was 115200 when I saw all that, and then tried 1500000 and despite > >it > >> was printing something that looks like the boot, just weird > >characters > >> appeared. > > > > > >With > > > https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Na= noPi-R5S-20230402.img.xz > >image > >I use 115200. > > > >Ganbold > > Thanks. Tried earlier but no luck. > > Could you send me a dmesg? I think much of it is in the acpi/dtb issue. I > would like to see how your dmesg shows. > Here it is: https://dmesgd.nycbug.org/index.cgi?do=3Dview&id=3D7129 Ganbold > > Does it have regular hdmi output? As there is in 13.2? > > Thanks, > > Matheus > > >> > >> > >> Thanks again, > >> > >> Matheus > >> > >> > >> >> > >> >> Thanks, > >> >> > >> >> matheus > >> >> > >> >> >> > >> >> >> I have little understanding of the arch, so my progress is much > >> >slow. > >> >> >> I got some dmesg from OpenBSD people where the nics show in > >> >ifconfig. > >> >> >> But > >> >> >> I couldn't get mine to behave this way. I can install though, > >> >using USB > >> >> >> nic. > >> >> >> I will try to buy a new sd card from the good list you pointed. > >> >> >> Another thing, I got the feeling that when I dd'ed the image > >using > >> >the > >> >> >> SD > >> >> >> card slot on the notebook it worked and when was through usb > >> >adapter did > >> >> >> not. Does it make sense? Using Linux mint as host for this. > >> >> >> Thanks so much for the answer and help, > >> >> >> > >> >> >> Matheus > >> >> >> > >> >> >> > >> >> >> >You can build a stock ARM64 generic kernel and most things > >will > >> >be > >> >> >> >usable, however as Ganbold wrote the DTS files is not in there > >> >yet (and > >> >> >> >not even in linux where our DTS files are fetched from). > >> >> >> >However the EDK2 boot loader provided (and used in above > >image) > >> >on > >> >> >> > > >> >> >> > >> > >>>https://people.freebsd.org/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb= =C3=AF > >> >> =C2=BF=C2=BC > >> >> >> >NANOPI-R5S_EFI > >> >> >> >File =C3=82=C2=B7 1,7 MB > >> >> >> > > >> >> >> >does hand over the =C3=A2=E2=82=AC=C5=93right=C3=A2=E2=82=AC DT= B file if you want to > >experiment. > >> >> >> > > >> >> >> >If you need the used DTS file and build guidance let me know > >in > >> >private > >> >> >> >mail... > >> >> >> > > >> >> >> >-- > >> >> >> >S=C3=83=C2=B8ren Schmidt > >> >> >> >sos@deepcore.dk / sos@freebsd.org > >> >> >> >"So much code to hack, so little time" > >> >> >> > >> >> >> --- > >> >> >> "We will call you Cygnus, > >> >> >> the God of balance you shall be." > >> >> >> > >> >> >> > >> >> > > >> >> > >> >> > >> >> -- > >> >> "We will call you Cygnus, > >> >> the God of balance you shall be." > >> >> ------------------------------ > >> >> "We will call you Cygnus, > >> >> the God of balance you shall be." > >> > >> --- > >> "We will call you Cygnus, > >> the God of balance you shall be." > >> > > --- > "We will call you Cygnus, > the God of balance you shall be." > --00000000000075663405fb0021d8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sat, May 6, 2023 at 1:55=E2=80=AFP= M Matheus <lojas@arroway.org>= ; wrote:


On May 5, 2023 11:03:47 PM GMT-03:00, Ganbold Tsagaankhuu <ganbold@gmail.com> wrote:<= br> >On Sat, May 6, 2023 at 6:03=E2=80=AFAM Matheus <lojas@arroway.org> wrote:
>
>>
>>
>> On May 5, 2023 2:23:15 AM GMT-03:00, Ganbold Tsagaankhuu <
>> ganbold@gma= il.com> wrote:
>> >On Thu, May 4, 2023 at 10:05=E2=80=AFPM Matheus <lojas@arroway.org> wrot= e:
>> >
>> >> > On Tue, May 2, 2023 at 6:50=C3=A2=E2=82=AC=C2=AFPM M= atheus <lojas@arr= oway.org>
>wrote:
>> >> >
>> >> >>
>> >> >>
>> >> >> On May 1, 2023 1:47:12 PM GMT-03:00, "S=C3= =83=C2=B8ren Schmidt" <
>> >> >> soren.schmidt@gmail.com> wrote:
>> >> >> >> On 30 Apr 2023, at 23.44, Matheus <<= a href=3D"mailto:lojas@arroway.org" target=3D"_blank">lojas@arroway.org= > wrote:
>> >> >> >>
>> >> >> >> Hi,
>> >> >> >>
>> >> >> >> I am trying to have FreeBSD 14 running = on this SBC. I could
>not
>> >find
>> >> >> >any guides in how to build images for it. I = found the
>> >> >> >people.freebsd.org/~sos/ site that= has some images, and one
>for
>> >it,
>> >> but
>> >> >> >that seldom boots my board, and when it did,= there was no
>console
>> >over
>> >> >> >serial or vga.
>> >> >> >>
>> >> >> >> If anyone can give any hints. Unfortuna= tely my dev skills
>are
>> >not
>> >> >> >good. But I can test and help build once I f= igure out how :)
>> >> >> >
>> >> >> >Hi Matheus
>> >> >> >
>> >> >> >The image at
>> >> >> >
>> >> >> >
>> >> >>
>> >>
>> >
>>
>https://people.fr= eebsd.org/~sos/ARM64/current-RK356X-images/nano5-sdcard.img.gz
>> >> >> >
>> >> >> >for the Nanopi R5S does indeed boot with bot= h HDMI output and
>> >serial
>> >> >> >console (1500000baud).
>> >> >> >
>> >> >> >The boot loader (EDK2 in FDT mode) is very p= icky on SD card
>> >quality
>> >> >> >though from experience, I works for me with = Sandisk Ultra /
>> >Extreme
>> >> >> >cards but not with Samsung and cheap noname = SD cards YMMV.
>> >> >>
>> >> >> Hi S=C3=83=C2=B8ren,
>> >> >>
>> >> >> I had really issues on sd carda. I got it to boo= t once, but I
>was
>> >> >> printing
>> >> >> characters on screen at one per second. So I reb= ooted and don't
>> >remember
>> >> >> why rewrote the card. I can't boot anymore. = Tried different
>cards,
>> >> >> SanDisk
>> >> >> ultra, no luck.
>> >> >>
>> >> >> I can boot an 13.2 image from the guy at persona= lbsd though.
>But
>> >there I
>> >> >> have just one ethernet.
>> >> >> On 13.2 I cannot list the ethernets nics using p= ciconf -lv,
>> >including
>> >> >> the
>> >> >> one that works. Is this expected?
>> >> >
>> >> >
>> >> > Yes. Did you try
>> >> >
>> >>
>> >
>>
>https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Nan= oPi-R5S-20230402.img.xz
>> >> > ?
>> >> > This image should have support for pcie and all ethe= rnet should
>> >work
>> >> IIRC.
>> >> >
>> >> > Ganbold
>> >>
>> >> Hi Ganbold,
>> >>
>> >> I tried it and other 2 images and no success, including t= he image
>> >pointed
>> >> by Soren. Unfortunately I just got to boot one image from= 14 but I
>> >> overwritten the sd card and can't remember which sd c= ard and image
>:(
>> >>
>> >> The only image I can make it boot is from 13.2R, and I go= t it
>> >installed
>> >> fine (using an EFI image from personalbsd from March 23).=
>> >>
>> >> Will wait for the next round of images for 14.
>> >>
>> >
>> >Above image works for me:
>> >
>> >root@NanoPi-R5S:~ # ifconfig
>> >re0: flags=3D8803<UP,BROADCAST,SIMPLEX,MULTICAST> metric= 0 mtu 1500
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0
>options=3D201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 62:73:64:e2:d4:87
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 192.168.111.2 netmask 0xfffff= f00 broadcast
>192.168.111.255
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 status: no carrier
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D29<PERFORMNUD,IFD= ISABLED,AUTO_LINKLOCAL>
>> >re1: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>= ; metric 0 mtu
>> >1500
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0
>options=3D201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 62:73:64:9a:5e:55
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 192.168.2.2 netmask 0xffffff0= 0 broadcast 192.168.2.255
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000ba= seT <full-duplex>)
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 status: active
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D29<PERFORMNUD,IFD= ISABLED,AUTO_LINKLOCAL>
>> >eq0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>= ; metric 0 mtu
>> >1500
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 options=3D80008<VLAN_MTU,LINKST= ATE>
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 ether 22:07:01:bb:41:76
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 192.168.1.202 netmask 0xfffff= f00 broadcast
>192.168.1.255
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 media: Ethernet autoselect (1000ba= seT <full-duplex>)
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 status: active
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D29<PERFORMNUD,IFD= ISABLED,AUTO_LINKLOCAL>
>> >lo0: flags=3D8049<UP,LOOPBACK,RUNNING,MULTICAST> metric = 0 mtu 16384
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0
>options=3D680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>=
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet6 ::1 prefixlen 128
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet6 fe80::1%lo0 prefixlen 64 sco= peid 0x4
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 inet 127.0.0.1 netmask 0xff000000<= br> >> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 groups: lo
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 nd6 options=3D21<PERFORMNUD,AUT= O_LINKLOCAL>
>> >root@NanoPi-R5S:~ # uname -an
>> >FreeBSD NanoPi-R5S 14.0-CURRENT FreeBSD 14.0-CURRENT #0
>> >main-n261950-4aeb939ecf8b-dirty: Sun Apr=C2=A0 2 17:42:01 MSK = 2023
>> >root@honeycomb.local
>>
>:/usr/crochet/work/obj/usr/crochet/src-current-14.0/arm64.aarch64/sys/E= XPERT
>> >arm64
>> >root@NanoPi-R5S:~ # kldstat
>> >Id Refs Address=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Size Name
>> > 1=C2=A0 =C2=A0 8 0xffff000000000000=C2=A0 10cb350 kernel
>> > 2=C2=A0 =C2=A0 1 0xffff0000010cd000=C2=A0 =C2=A0 a9508 if_re.= ko
>> > 3=C2=A0 =C2=A0 1 0xffff0000c9e00000=C2=A0 =C2=A0 24000 fdescf= s.ko
>> > 4=C2=A0 =C2=A0 1 0xffff0000c9e24000=C2=A0 =C2=A0 22000 mac_nt= pd.ko
>> >root@NanoPi-R5S:~ #
>> >
>> >Ganbold
>>
>> Hi Ganbold and others,
>>
>> I gave another try today and I got to see some stuff on the serial=
>console
>> but the serial terminal itself is not usable. It prints
>unrecognizable
>> characters, but I could recognize the red beastie logo.
>> On the hdmi screen I have no output. Can you confirm the serial sp= eed
>> line?
>> It was 115200 when I saw all that, and then tried 1500000 and desp= ite
>it
>> was printing something that looks like the boot, just weird
>characters
>> appeared.
>
>
>With
>https://personalbsd.org/download/Business/FreeBSD-aarch64-14.0-CURRENT-Nan= oPi-R5S-20230402.img.xz
>image
>I use 115200.
>
>Ganbold

Thanks. Tried earlier but no luck.

Could you send me a dmesg? I think much of it is in the acpi/dtb issue. I w= ould like to see how your dmesg shows.

= Here it is:


Ganbold

<= /div>
=C2=A0

Does it have regular hdmi output? As there is in 13.2?

Thanks,

Matheus

>>
>>
>> Thanks again,
>>
>> Matheus
>>
>>
>> >>
>> >> Thanks,
>> >>
>> >> matheus
>> >>
>> >> >>
>> >> >> I have little understanding of the arch, so my p= rogress is much
>> >slow.
>> >> >> I got some dmesg from OpenBSD people where the n= ics show in
>> >ifconfig.
>> >> >> But
>> >> >> I couldn't get mine to behave this way. I ca= n install though,
>> >using USB
>> >> >> nic.
>> >> >> I will try to buy a new sd card from the good li= st you pointed.
>> >> >> Another thing, I got the feeling that when I dd&= #39;ed the image
>using
>> >the
>> >> >> SD
>> >> >> card slot on the notebook it worked and when was= through usb
>> >adapter did
>> >> >> not. Does it make sense? Using Linux mint as hos= t for this.
>> >> >> Thanks so much for the answer and help,
>> >> >>
>> >> >> Matheus
>> >> >>
>> >> >>
>> >> >> >You can build a stock ARM64 generic kernel a= nd most things
>will
>> >be
>> >> >> >usable, however as Ganbold wrote the DTS fil= es is not in there
>> >yet (and
>> >> >> >not even in linux where our DTS files are fe= tched from).
>> >> >> >However the EDK2 boot loader provided (and u= sed in above
>image)
>> >on
>> >> >> >
>> >> >>
>>
>>>https://people= .freebsd.org/~sos/ARM64/EDK2-RK356X/NANOPI-R5S_EFI.itb=C3=AF
>> >> =C2=BF=C2=BC
>> >> >> >NANOPI-R5S_EFI
>> >> >> >File =C3=82=C2=B7 1,7 MB
>> >> >> >
>> >> >> >does hand over the =C3=A2=E2=82=AC=C5=93righ= t=C3=A2=E2=82=AC DTB file if you want to
>experiment.
>> >> >> >
>> >> >> >If you need the used DTS file and build guid= ance let me know
>in
>> >private
>> >> >> >mail...
>> >> >> >
>> >> >> >--
>> >> >> >S=C3=83=C2=B8ren Schmidt
>> >> >> >sos@deepcore.dk / sos@freebsd.org
>> >> >> >"So much code to hack, so little time&q= uot;
>> >> >>
>> >> >> ---
>> >> >> "We will call you Cygnus,
>> >> >> the God of balance you shall be."
>> >> >>
>> >> >>
>> >> >
>> >>
>> >>
>> >> --
>> >> "We will call you Cygnus,
>> >> the God of balance you shall be."
>> >> ------------------------------
>> >> "We will call you Cygnus,
>> >> the God of balance you shall be."
>>
>> ---
>> "We will call you Cygnus,
>> the God of balance you shall be."
>>

---
"We will call you Cygnus,
the God of balance you shall be."
--00000000000075663405fb0021d8--
3D"preview.png"