From owner-freebsd-arm@freebsd.org Sun Sep 23 23:20:04 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03DC0109CFDB for ; Sun, 23 Sep 2018 23:20:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-15.consmr.mail.bf2.yahoo.com (sonic310-15.consmr.mail.bf2.yahoo.com [74.6.135.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 95786872B0 for ; Sun, 23 Sep 2018 23:20:03 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: egZBz8kVM1kWzsI4SjydlXtTkybsIpFZnpZau_BQtG2uvVyqCzK3BUoabMBbDxA o6m8kjuPEiWn1ijvlbFu.Q5JFKJhFlNmgjjxr72CaDSbWLAvKUrjqlx6yaQqP0tC2ZWDFGVotSnm f5.zfBQgYDWgUyGSjlZaz95BrpOr2j60snf7fSl6qpWO1iI.7TJa9wsA9JQ0caDUZhmiNCoZZKMz tSi1EDFjVGh.pQ.hXp2cYdOGvaADRwLcw3nmWG08xuiNxKE41A2plKJoHFQfKNohZSqEEWOQov2e x7EXLSg3hiIG2Juq_HVYgkwcJHBVtDE2Ck9W4o7fJyAsP4vA7Y_5vnj.PIMRbeV01dB8qOVYce2q EMShLm3bUBGsG_rffEokecK322pr5o7BVYKk5rmvjO6Vyc0GtOrdQtmzWiXoHtF8aio8WAqCsE_r 3n2AqTw5Eq2aMVBhFxUCM5LKl97LFLwR5Oq6sAKiAk5096_74Su2hVPF2DVGW98DK.yyX6iTMnnc pkgsv4mlJyjU9KGkw65MydEXRH6eaTSGZ3IerLoUkwXL_6mODLCrwuvwNEpkNj1Mg33Q95vr2z3G PI21E3sB47SDogJJpIb0pb_WXIaXz.hjhodCqfopWK6rJAuCcfMJJMxaNJg4SNDET0b9ZMoHc0DV yiOPGS8EBl9nqRXUoKe_a0CUlPH.2u_DkiyVZyfrJOuH65qyuoeLkpWAOF89TnRToK346dvbGnCS oV7SLXVaG_bUDlQ39hbEMmQSsNgJFsTlHBNf6G6Yi2ajcsNTB5ctzWNEHH7zYezBgtIIcp.t3HVj mM4hc8GDmp9uHnLmkiieYXZV_r26GnODMdmqknsnfh9OEEE7kvCc7UVQej.hWFwVDnMu8Ocsy_Vd x3hNBlcbrb7pqWUqFujt_pYjoNSIGQFCN.6nrB0QekcAs3YX8PwaH1FY10DTT_0ifXo0u3vRU.L3 imVfUJMkgakjxWa8JsliuN78Wn3icyjQtQObbJwgx60kJM6YlCJt37kUualp90UXexOZq29CPdDX JhVUiqErhR2g- Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.bf2.yahoo.com with HTTP; Sun, 23 Sep 2018 23:19:57 +0000 Received: from ip70-189-131-151.lv.lv.cox.net (EHLO [192.168.0.105]) ([70.189.131.151]) by smtp413.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 2fbebe06943e6c6ff8d2a31290691be3; Sun, 23 Sep 2018 23:19:53 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: For comparison, head -r338860 based -j4 buildworld buildkernel time and swap usage on Pine64+ 2GB Message-Id: <94BB0AF9-6B81-430F-9897-26382B201F85@yahoo.com> Date: Sun, 23 Sep 2018 16:19:50 -0700 Cc: freebsd-arm To: bob prohaska X-Mailer: Apple Mail (2.3445.9.1) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Sep 2018 23:20:04 -0000 As I remember Bob Prohaska's fastest -j4 buildworld buildkernel full-builds on his rpi3 in his recent experiments were somewhat over 22 hours and had fairly extensive swapping. This report allows comparison to a particular Pine64+ 2GB based -j4 buildworld buildkernel, although I give details Bob P. did not. I also was not running any monitoring other than a modified top, not logged to a file. /usr/obj/cortexA53_clang_alt did not exist when this started, so this is for a from-scratch build. (The below avoids some >>> prefix line text.) QUOTE Script started on Sun Sep 23 00:28:14 2018 Command: env __MAKE_CONF=3D/root/src.configs/make.conf SRCCONF=3D/dev/null= = SRC_ENV_CONF=3D/root/src.configs/src.conf.cortexA53-clang-bootstrap.aarch6= 4-host WITH_META_MODE=3Dyes = MAKEOBJDIRPREFIX=3D/usr/obj/cortexA53_clang_alt/arm64.aarch64 make -j4 = buildworld buildkernel [Creating objdir = /usr/obj/cortexA53_clang_alt/arm64.aarch64/usr/src/arm64.aarch64...] --- buildworld --- make[1]: "/usr/src/Makefile.inc1" line 341: SYSTEM_COMPILER: Determined = that CC=3Dcc matches the source tree. Not bootstrapping a = cross-compiler. make[1]: "/usr/src/Makefile.inc1" line 346: SYSTEM_LINKER: Determined = that LD=3Dld matches the source tree. Not bootstrapping a cross-linker. --- buildworld_prologue --- -------------------------------------------------------------- World build started on Sun Sep 23 00:28:16 PDT 2018 -------------------------------------------------------------- . . . -------------------------------------------------------------- Kernel build for GENERIC-NODBG completed on Sun Sep 23 14:11:05 PDT = 2018 -------------------------------------------------------------- Command exit status: 0 Script done on Sun Sep 23 14:11:05 2018 END QUOTE So somewhat under 13 hours 45 minutes when the bootstrap avoided the cross-compiler and cross-linker builds when I'm building -r338860 via selecting: #WITH_CROSS_COMPILER=3D WITH_SYSTEM_COMPILER=3D WITH_SYSTEM_LINKER=3D # #CPUTYPE=3Dsoft WITH_LIBCPLUSPLUS=3D #WITH_LLD_BOOTSTRAP=3D WITHOUT_BINUTILS_BOOTSTRAP=3D WITH_ELFTOOLCHAIN_BOOTSTRAP=3D #WITH_CLANG_BOOTSTRAP=3D WITH_CLANG=3D WITH_CLANG_IS_CC=3D WITH_CLANG_FULL=3D WITH_CLANG_EXTRAS=3D WITH_LLD=3D WITH_LLD_IS_LD=3D WITHOUT_BINUTILS=3D WITH_LLDB=3D # WITH_BOOT=3D WITHOUT_LIB32=3D # WITHOUT_GCC_BOOTSTRAP=3D WITHOUT_GCC=3D WITHOUT_GCC_IS_CC=3D WITHOUT_GNUCXX=3D MALLOC_PRODUCTION=3D # WITH_REPRODUCIBLE_BUILD=3D WITH_DEBUG_FILES=3D When I build for cortex-A53's I use: CFLAGS.clang+=3D -mcpu=3Dcortex-a53 CXXFLAGS.clang+=3D -mcpu=3Dcortex-a53 CPPFLAGS.clang+=3D -mcpu=3Dcortex-a53 ACFLAGS.arm64cpuid.S+=3D -mcpu=3Dcortex-a53+crypto ACFLAGS.aesv8-armx.S+=3D -mcpu=3Dcortex-a53+crypto ACFLAGS.ghashv8-armx.S+=3D -mcpu=3Dcortex-a53+crypto The environment that did the build had been built that way as well. Similarly for the GENERIC-NODBG used: # more /usr/src/sys/arm64/conf/GENERIC-NODBG # # GENERIC -- Custom configuration for the arm64/aarch64 # include "GENERIC" ident GENERIC-NODBG makeoptions DEBUG=3D-g # Build kernel with gdb(1) = debug symbols options ALT_BREAK_TO_DEBUGGER options KDB # Enable kernel debugger support # For minimum debugger support (stable branch) use: #options KDB_TRACE # Print a stack trace for a = panic options DDB # Enable the kernel debugger # Extra stuff: #options VERBOSE_SYSINIT # Enable verbose sysinit = messages #options BOOTVERBOSE=3D1 #options BOOTHOWTO=3DRB_VERBOSE #options KTR #options KTR_MASK=3DKTR_TRAP ##options KTR_CPUMASK=3D0xF #options KTR_VERBOSE # Disable any extra checking for. . . nooptions DEADLKRES # Enable the deadlock resolver nooptions INVARIANTS # Enable calls of extra sanity = checking nooptions INVARIANT_SUPPORT # Extra sanity checks of = internal structures, required by INVARIANTS nooptions WITNESS # Enable checks to detect = deadlocks and cycles nooptions WITNESS_SKIPSPIN # Don't run witness on spinlocks = for speed nooptions DIAGNOSTIC nooptions MALLOC_DEBUG_MAXZONES # Separate malloc(9) zones nooptions BUF_TRACKING nooptions FULL_BUF_TRACKING I've modified top in my active environments to report: Maximum Observed Mem Active (reported: 1501M) Maximum Observed Swap Used (reported: 198M) Minimum Observed Mem Free (reported: 33M) After the buildworld buildkernel completed top was showing: Mem: 79M Active, 1501M MaxObsActive, 1352M Inact, 1056K Laundry, 347M = Wired, 201M Buf, 194M Free, 33M MinObsFree Swap: 3584M Total, 19M Used, 198M MaxObsUsed, 3565M Free I had enabled using and used an e.MMC in DDR52 mode on a microsd adapter, using TRIM on the UFS file system (with the DISCARD bit set). I also had to enable DISCARD. "Enable(d)" here means source code changes in both cases. No other types of storage devices were connected so swap was on the same device as the UFS file system partition. After the build: # df -m Filesystem 1M-blocks Used Avail Capacity Mounted on /dev/label/PINE64P2Groot 109101 50910 49462 51% / devfs 0 0 0 100% /dev /dev/label/PINE642GAboot 63 8 55 13% /boot/efi I have in /boot/loader.conf : vfs.ffs.dotrimcons=3D1 and in /etc/sysctl.conf : vm.pageout_oom_seq=3D120 vfs.ffs.dotrimcons=3D1 (But vfs.ffs.dotrimcons defaults to 1 now as I remember. As I understand vm.pageout_oom_seq still defaults to 12. 120 might not be sufficient for a rpi3 to buildworld buildkernel in some contexts, or even for the Pine64+ 2GB if the storage I/O is sufficiently problematical for latencies that occur.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)