From owner-freebsd-stable@freebsd.org Mon Feb 12 05:03:04 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2031AF17EAD for ; Mon, 12 Feb 2018 05:03:04 +0000 (UTC) (envelope-from ask@develooper.com) Received: from mbox1.develooper.com (mbox1.develooper.com [207.171.7.178]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B8B067EA89 for ; Mon, 12 Feb 2018 05:03:03 +0000 (UTC) (envelope-from ask@develooper.com) Received: from mbox1.develooper.com (mbox1.develooper.com [127.0.0.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mbox1.develooper.com (Postfix) with ESMTPS id E4B27175D1B for ; Sun, 11 Feb 2018 20:56:26 -0800 (PST) Received: (qmail 4424 invoked from network); 12 Feb 2018 04:56:26 -0000 Received: from c-67-188-112-34.hsd1.ca.comcast.net (HELO ?10.0.200.100?) (ask@mail.dev@67.188.112.34) by smtp.develooper.com with ESMTPA; 12 Feb 2018 04:56:26 -0000 From: =?utf-8?Q?Ask_Bj=C3=B8rn_Hansen?= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: FreeBSD on 64MB memory Message-Id: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> Date: Sun, 11 Feb 2018 20:56:02 -0800 To: freebsd-stable@freebsd.org X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 05:03:04 -0000 Hi, I have an old Soekris system with 64MB memory that I upgraded from 10.3 = to 11.1 recently. Since then it=E2=80=99s started hanging every few = days. Today I happened to have a =E2=80=9Ctop=E2=80=9D instance running on the = serial console. The system is minimally responsive to the network (ICMP = and CARP are working, but no services). =46rom the top output it=E2=80=99s not clear what resource it=E2=80=99s = out of. There=E2=80=99s no swap configured, but that what it looks like = it=E2=80=99s trying to do?=20 The =E2=80=98pf purge=E2=80=99 process is suspicious. There are no pf = rules configured on the system (it should be all disabled). Any suggestions? (Other than =E2=80=9Cseriously =E2=80=A6 64MB = memory?!=E2=80=9D). Ask last pid: 2228; load averages: 0.63, 0.65, 0.70 up 0+21:13:56 = 04:50:47 36 processes: 2 running, 33 sleeping, 1 waiting CPU: 0.1% user, 0.0% nice, 11.3% system, 2.4% interrupt, 86.2% idle Mem: 1616K Active, 10M Inact, 28M Wired, 3099K Buf, 1768K Free Swap: PID UID THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 11 0 1 155 ki31 0K 8K RUN 16.8H 75.53% idle 0 0 7 -16 - 0K 64K swapin 155:10 12.00% kernel 16 0 3 -16 - 0K 24K psleep 10:04 8.15% = pagedaemon 12 0 14 -64 - 0K 112K WAIT 27:21 2.80% intr 6 0 1 -16 - 0K 8K pftm 8:56 0.73% pf = purge 1027 0 1 20 0 7272K 1140K RUN 6:04 0.52% top 7 0 1 -16 - 0K 8K - 2:57 0.18% = rand_harvestq 21 0 1 16 - 0K 8K syncer 0:22 0.05% syncer 22 0 1 21 - 0K 8K vlruwt 0:07 0.01% vnlru 19 0 1 20 - 0K 8K psleep 0:08 0.01% = bufdaemon 788 0 1 20 0 5920K 1752K select 0:04 0.01% syslogd 20 0 1 20 - 0K 8K - 0:07 0.01% = bufspacedaemon 911 0 1 20 0 8816K 8844K kmem a 43:41 0.00% ntpd 996 0 1 20 0 13628K 4620K kmem a 0:06 0.00% sshd 985 0 1 20 0 5952K 584K kmem a 0:07 0.00% cron 709 0 1 20 0 7300K 3364K kmem a 0:01 0.00% devd 2227 0 1 24 0 5952K 1232K kmem a 0:00 0.00% cron=20= From owner-freebsd-stable@freebsd.org Mon Feb 12 05:19:38 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3898AF19295 for ; Mon, 12 Feb 2018 05:19:38 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B7E657F818 for ; Mon, 12 Feb 2018 05:19:37 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w1C5JIfO023431 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 12 Feb 2018 06:19:19 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: ask@develooper.com Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w1C5JDTL021293 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 12 Feb 2018 12:19:13 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: FreeBSD on 64MB memory To: =?UTF-8?Q?Ask_Bj=c3=b8rn_Hansen?= , freebsd-stable@freebsd.org References: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> From: Eugene Grosbein Message-ID: <5A8123CE.9050609@grosbein.net> Date: Mon, 12 Feb 2018 12:19:10 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 05:19:38 -0000 12.02.2018 11:56, Ask Bjørn Hansen пишет: > Hi, > > I have an old Soekris system with 64MB memory that I upgraded from 10.3 to 11.1 recently. Since then it’s started hanging every few days. > > Today I happened to have a “top” instance running on the serial console. The system is minimally responsive to the network (ICMP and CARP are working, but no services). > >>From the top output it’s not clear what resource it’s out of. I suspect it is out of many types of kernel memory including mbuf clusters, hence no working TCP/UDP but ICMP works. > There’s no swap configured, but that what it looks like it’s trying to do? > > The ‘pf purge’ process is suspicious. There are no pf rules configured on the system (it should be all disabled). > > Any suggestions? (Other than “seriously … 64MB memory?!”). Please show output of commands: grep memory /var/run/dmesg.boot top -ores -d1 sysctl kern.ipc.nmbclusters It would be also very useful to obtain output of "vmstat -z" in a moment of breakage. From owner-freebsd-stable@freebsd.org Mon Feb 12 11:58:10 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1C3BF13098 for ; Mon, 12 Feb 2018 11:58:10 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C465705FE for ; Mon, 12 Feb 2018 11:58:09 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.128.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 19EFC2600DF for ; Mon, 12 Feb 2018 12:58:07 +0100 (CET) To: FreeBSD Stable From: Hans Petter Selasky Subject: [HEADS UP] Mellanox's mlxen.ko will be renamed into mlx4en.ko for 11-stable Message-ID: <76bffd87-c059-8de0-f0b8-e1205d2868da@selasky.org> Date: Mon, 12 Feb 2018 12:55:10 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 11:58:10 -0000 Hi, The change will happen this week. This is part of ongoing work in FreeBSD 11-stable. Make sure to update your /boot/loader.conf by adding mlx4en_load="YES" if you are using the mlxen.ko kernel modules. --HPS From owner-freebsd-stable@freebsd.org Mon Feb 12 17:39:40 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44492F0BA45 for ; Mon, 12 Feb 2018 17:39:40 +0000 (UTC) (envelope-from ask@develooper.com) Received: from mbox1.develooper.com (mbox1.develooper.com [207.171.7.178]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B788E80F8E for ; Mon, 12 Feb 2018 17:39:39 +0000 (UTC) (envelope-from ask@develooper.com) Received: from mbox1.develooper.com (mbox1.develooper.com [127.0.0.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mbox1.develooper.com (Postfix) with ESMTPS id A5374175E5E for ; Mon, 12 Feb 2018 09:39:07 -0800 (PST) Received: (qmail 18980 invoked from network); 12 Feb 2018 17:39:07 -0000 Received: from c-67-188-112-34.hsd1.ca.comcast.net (HELO ?10.0.200.100?) (ask@mail.dev@67.188.112.34) by smtp.develooper.com with ESMTPA; 12 Feb 2018 17:39:07 -0000 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: FreeBSD on 64MB memory From: =?utf-8?Q?Ask_Bj=C3=B8rn_Hansen?= In-Reply-To: <5A8123CE.9050609@grosbein.net> Date: Mon, 12 Feb 2018 09:38:45 -0800 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> <5A8123CE.9050609@grosbein.net> To: Eugene Grosbein X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 17:39:40 -0000 > On Feb 11, 2018, at 9:19 PM, Eugene Grosbein = wrote: >=20 > 12.02.2018 11:56, Ask Bj=C3=B8rn Hansen =D0=BF=D0=B8=D1=88=D0=B5=D1=82: >> Hi, >>=20 >> I have an old Soekris system with 64MB memory that I upgraded from = 10.3 to 11.1 recently. Since then it=E2=80=99s started hanging every few = days. >>=20 >> Today I happened to have a =E2=80=9Ctop=E2=80=9D instance running on = the serial console. The system is minimally responsive to the network = (ICMP and CARP are working, but no services). >>=20 >>> =46rom the top output it=E2=80=99s not clear what resource it=E2=80=99= s out of. >=20 > I suspect it is out of many types of kernel memory including mbuf = clusters, > hence no working TCP/UDP but ICMP works. That makes sense. The console locks up, too (as soon as I ctrl-c=E2=80=99e= d the running top process the console was frozen). >> There=E2=80=99s no swap configured, but that what it looks like = it=E2=80=99s trying to do?=20 >>=20 >> The =E2=80=98pf purge=E2=80=99 process is suspicious. There are no pf = rules configured on the system (it should be all disabled). >>=20 >> Any suggestions? (Other than =E2=80=9Cseriously =E2=80=A6 64MB = memory?!=E2=80=9D). >=20 > Please show output of commands: >=20 > grep memory /var/run/dmesg.boot real memory =3D 67108864 (64 MB) avail memory =3D 42098688 (40 MB) The 24MB are for the kernel? I wonder my 11.1 kernel is less = discriminating with what I compiled in... > top -ores -d1 Shortly after boot: last pid: 1008; load averages: 0.57, 0.62, 0.53 up 0+00:19:31 = 06:24:50 8 processes: 1 running, 7 sleeping CPU: % user, % nice, % system, % interrupt, % idle Mem: 9084K Active, 3644K Inact, 29M Wired, 4862K Buf, 492K Free Swap: PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 911 root 1 22 0 8816K 8844K select 0:39 4.20% ntpd 959 root 1 52 0 10756K 5196K select 0:00 0.00% sshd 709 root 1 20 0 7300K 3224K select 0:00 0.00% devd 1002 root 1 20 0 6712K 2876K pause 0:01 0.49% csh 1008 root 1 49 0 7272K 2356K RUN 0:00 0.00% top 991 root 1 26 0 6400K 2196K wait 0:01 0.00% login 985 root 1 20 0 5952K 1848K nanslp 0:00 0.00% cron 788 root 1 20 0 5920K 1776K select 0:01 0.00% syslogd > sysctl kern.ipc.nmbclusters kern.ipc.nmbclusters: 898 kern.ipc.nmbufs: 5745 kern.ipc.maxmbufmem: 7350272 > It would be also very useful to obtain output of "vmstat -z" in a = moment of breakage. I made it run on the console every 30 seconds, this is the last one I = got. For comparison there=E2=80=99s a =E2=80=9Cshortly after boot=E2=80=9D= version below the =3D=3D=3D=E2=80=99s. Ask Shortly before it stopped responding: ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 256, 0, 128, 7, 128, 0, 0 UMA Zones: 408, 0, 129, 6, 129, 0, 0 UMA Slabs: 56, 0, 1655, 73, 20845, 0, 0 UMA Hash: 128, 0, 11, 20, 14, 0, 0 4 Bucket: 16, 0, 0, 504, 364, 0, 0 6 Bucket: 24, 0, 0, 336, 118, 0, 0 8 Bucket: 32, 0, 0, 378, 54150, 0, 0 12 Bucket: 48, 0, 0, 0, 0, 0, 0 16 Bucket: 64, 0, 2, 313, 1840, 8, 0 32 Bucket: 128, 0, 3, 152, 1516, 0, 0 64 Bucket: 256, 0, 1, 74, 221, 0, 0 128 Bucket: 512, 0, 4, 28, 1324, 0, 0 256 Bucket: 1024, 0, 3, 13, 3917, 4, 0 vmem btag: 28, 0, 3169, 143, 3847, 56, 0 VM OBJECT: 148, 0, 1208, 115, 38997, 0, 0 RADIX NODE: 44, 10738, 1749, 344, 88089, 0, 0 MAP: 140, 0, 3, 81, 3, 0, 0 KMAP ENTRY: 72, 0, 5, 163, 5, 0, 0 MAP ENTRY: 72, 0, 244, 148, 89326, 0, 0 VMSPACE: 232, 0, 10, 75, 2062, 0, 0 fakepg: 68, 0, 0, 0, 0, 0, 0 mt_zone: 2060, 0, 352, 0, 352, 0, 0 16: 16, 0, 1159, 353, 813531, 0, 0 32: 32, 0, 957, 303, 23866, 0, 0 64: 64, 0, 1385, 253, 29315, 0, 0 128: 128, 0, 734, 134, 14162, 0, 0 256: 256, 0, 378, 57, 5062, 0, 0 512: 512, 0, 57, 7, 4807, 0, 0 1024: 1024, 0, 31, 33, 539, 0, 0 2048: 2048, 0, 208, 2, 2908, 0, 0 4096: 4096, 0, 86, 0, 2165, 0, 0 8192: 8192, 0, 0, 0, 0, 0, 0 16384: 16384, 0, 0, 0, 0, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 SLEEPQUEUE: 44, 0, 85, 293, 85, 0, 0 64 pcpu: 8, 0, 2642, 174, 2714, 0, 0 ptr pcpu: 4, 0, 0, 0, 0, 0, 0 Files: 56, 0, 42, 318, 18945, 0, 0 filedesc0: 888, 0, 34, 6, 2085, 0, 0 rl_entry: 28, 0, 23, 265, 23, 0, 0 TURNSTILE: 72, 0, 85, 132, 85, 0, 0 umtx pi: 52, 0, 0, 0, 0, 0, 0 umtx_shm: 52, 0, 0, 0, 0, 0, 0 MAC labels: 20, 0, 0, 0, 0, 0, 0 PROC: 916, 0, 33, 15, 2084, 0, 0 THREAD: 904, 0, 70, 14, 71, 0, 0 cpuset: 40, 0, 49, 52, 49, 0, 0 audit_record: 1112, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 5745, 259, 0, 418909, 2, 0 mbuf: 256, 5745, 263, 273, 773249, 18, 0 mbuf_cluster: 2048, 898, 259, 131, 204839,1868, 0 mbuf_jumbo_page: 4096, 448, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 132, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 74, 0, 0, 0, 0, 0 g_bio: 264, 0, 0, 0, 13391, 0, 0 ttyinq: 152, 0, 15, 37, 60, 0, 0 ttyoutq: 256, 0, 8, 22, 32, 0, 0 nvme_request: 100, 0, 0, 0, 0, 0, 0 cryptop: 60, 0, 0, 0, 0, 0, 0 cryptodesc: 56, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 16, 0, 0, 0, 0, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 VNODE: 284, 0, 1134, 28, 1170, 0, 0 VNODEPOLL: 64, 0, 0, 0, 0, 0, 0 BUF TRIE: 44, 0, 52, 676, 2310, 0, 0 NAMEI: 1024, 0, 0, 16, 55376, 0, 0 rentr: 12, 0, 0, 0, 0, 0, 0 S VFS Cache: 72, 0, 977, 143, 2862, 0, 0 STS VFS Cache: 92, 0, 0, 0, 0, 0, 0 L VFS Cache: 292, 0, 0, 0, 0, 0, 0 LTS VFS Cache: 312, 0, 0, 0, 0, 0, 0 NCLNODE: 360, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 0, 0, 58, 0, 0 Mountpoints: 672, 0, 4, 14, 5, 0, 0 pipe: 416, 0, 0, 0, 1030, 0, 0 procdesc: 80, 0, 0, 0, 0, 0, 0 AIO: 128, 0, 0, 0, 0, 0, 0 AIOP: 16, 0, 0, 0, 0, 0, 0 AIOCB: 428, 0, 0, 0, 0, 0, 0 AIOL: 64, 0, 0, 0, 0, 0, 0 AIOLIO: 172, 0, 0, 0, 0, 0, 0 ksiginfo: 80, 0, 166, 134, 17061, 0, 0 itimer: 232, 0, 0, 0, 0, 0, 0 bridge_rtnode: 36, 0, 0, 0, 0, 0, 0 KNOTE: 72, 0, 0, 0, 0, 0, 0 socket: 524, 1393, 20, 15, 3048, 0, 0 unpcb: 172, 1403, 6, 17, 2053, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0 ipq: 32, 126, 0, 0, 0, 0, 0 udp_inpcb: 304, 1404, 11, 15, 925, 0, 0 udpcb: 20, 1414, 11, 191, 925, 0, 0 tcp_inpcb: 304, 1404, 2, 11, 61, 0, 0 tcpcb: 772, 1395, 2, 3, 61, 0, 0 tcptw: 60, 335, 0, 0, 0, 0, 0 syncache: 128, 15376, 0, 0, 54, 12, 0 hostcache: 76, 15370, 0, 0, 0, 0, 0 sackhole: 20, 0, 0, 0, 0, 0, 0 tcpreass: 20, 202, 0, 0, 0, 0, 0 sctp_ep: 1052, 1395, 0, 0, 0, 0, 0 sctp_asoc: 1656, 40000, 0, 0, 0, 0, 0 sctp_laddr: 24, 80136, 0, 0, 5, 0, 0 sctp_raddr: 540, 80003, 0, 0, 0, 0, 0 sctp_chunk: 104, 400026, 0, 0, 0, 0, 0 sctp_readq: 108, 400007, 0, 0, 0, 0, 0 sctp_stream_msg_out: 72, 400008, 0, 0, 0, 0, 0 sctp_asconf: 24, 400008, 0, 0, 0, 0, 0 sctp_asconf_ack: 24, 400008, 0, 0, 0, 0, 0 udplite_inpcb: 304, 1404, 0, 0, 0, 0, 0 ripcb: 304, 1404, 0, 0, 0, 0, 0 rtentry: 112, 0, 42, 30, 50, 0, 0 pf mtags: 32, 0, 0, 0, 0, 0, 0 pf states: 208, 10013, 0, 0, 0, 0, 0 pf state keys: 64, 0, 0, 0, 0, 0, 0 pf source nodes: 116, 10030, 0, 0, 0, 0, 0 pf table entries: 92, 0, 0, 0, 0, 0, 0 pf table counters: 64, 0, 0, 0, 0, 0, 0 pf frags: 80, 0, 0, 0, 0, 0, 0 pf frag entries: 24, 5040, 0, 0, 0, 0, 0 pf state scrubs: 28, 0, 0, 0, 0, 0, 0 IPFW counters: 16, 0, 1, 63, 1, 0, 0 IPFW dynamic rule: 112, 16416, 0, 0, 0, 0, 0 divcb: 304, 1404, 0, 0, 0, 0, 0 selfd: 32, 0, 38, 340, 2106598, 0, 0 SWAPMETA: 276, 5390, 0, 0, 0, 0, 0 FFS inode: 112, 0, 1109, 151, 1144, 0, 0 FFS1 dinode: 128, 0, 632, 81, 662, 0, 0 FFS2 dinode: 256, 0, 477, 33, 482, 0, 0 md0: 512, 0, 5042, 14, 5050, 0, 0 md1: 512, 0, 4298, 6, 4299, 0, 0 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ~25 minutes after boot: ntp1.us.grundclock.com# vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 256, 0, 128, 7, 128, 0, 0 UMA Zones: 408, 0, 129, 6, 129, 0, 0 UMA Slabs: 56, 0, 1658, 70, 2158, 0, 0 UMA Hash: 128, 0, 11, 20, 14, 0, 0 4 Bucket: 16, 0, 1, 503, 254, 0, 0 6 Bucket: 24, 0, 0, 336, 118, 0, 0 8 Bucket: 32, 0, 2, 376, 149, 0, 0 12 Bucket: 48, 0, 0, 0, 0, 0, 0 16 Bucket: 64, 0, 5, 310, 35, 8, 0 32 Bucket: 128, 0, 6, 149, 396, 0, 0 64 Bucket: 256, 0, 3, 72, 47, 0, 0 128 Bucket: 512, 0, 9, 23, 52, 0, 0 256 Bucket: 1024, 0, 15, 13, 110, 0, 0 vmem btag: 28, 0, 3101, 67, 3133, 22, 0 VM OBJECT: 148, 0, 1187, 109, 12302, 0, 0 RADIX NODE: 44, 10738, 1712, 381, 33112, 0, 0 MAP: 140, 0, 3, 81, 3, 0, 0 KMAP ENTRY: 72, 0, 5, 163, 5, 0, 0 MAP ENTRY: 72, 0, 227, 165, 27414, 0, 0 VMSPACE: 232, 0, 9, 76, 999, 0, 0 fakepg: 68, 0, 0, 0, 0, 0, 0 mt_zone: 2060, 0, 352, 0, 352, 0, 0 16: 16, 0, 1163, 349, 61329, 0, 0 32: 32, 0, 955, 305, 5553, 0, 0 64: 64, 0, 1366, 272, 20940, 0, 0 128: 128, 0, 726, 142, 13237, 0, 0 256: 256, 0, 374, 61, 1906, 0, 0 512: 512, 0, 57, 31, 1553, 0, 0 1024: 1024, 0, 31, 33, 526, 0, 0 2048: 2048, 0, 208, 6, 2703, 0, 0 4096: 4096, 0, 85, 2, 1101, 0, 0 8192: 8192, 0, 0, 0, 0, 0, 0 16384: 16384, 0, 0, 0, 0, 0, 0 32768: 32768, 0, 0, 0, 0, 0, 0 65536: 65536, 0, 0, 0, 0, 0, 0 SLEEPQUEUE: 44, 0, 77, 301, 77, 0, 0 64 pcpu: 8, 0, 2642, 174, 2714, 0, 0 ptr pcpu: 4, 0, 0, 0, 0, 0, 0 Files: 56, 0, 42, 318, 8074, 0, 0 filedesc0: 888, 0, 33, 15, 1022, 0, 0 rl_entry: 28, 0, 16, 272, 16, 0, 0 TURNSTILE: 72, 0, 77, 140, 77, 0, 0 umtx pi: 52, 0, 0, 0, 0, 0, 0 umtx_shm: 52, 0, 0, 0, 0, 0, 0 MAC labels: 20, 0, 0, 0, 0, 0, 0 PROC: 916, 0, 32, 12, 1021, 0, 0 THREAD: 904, 0, 62, 14, 63, 0, 0 cpuset: 40, 0, 49, 52, 49, 0, 0 audit_record: 1112, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 5745, 128, 253, 32698, 0, 0 mbuf: 256, 5745, 1, 263, 57270, 0, 0 mbuf_cluster: 2048, 898, 381, 3, 726, 0, 0 mbuf_jumbo_page: 4096, 448, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 132, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 74, 0, 0, 0, 0, 0 g_bio: 264, 0, 0, 30, 11703, 0, 0 ttyinq: 152, 0, 15, 37, 60, 0, 0 ttyoutq: 256, 0, 8, 22, 32, 0, 0 nvme_request: 100, 0, 0, 0, 0, 0, 0 cryptop: 60, 0, 0, 0, 0, 0, 0 cryptodesc: 56, 0, 0, 0, 0, 0, 0 vtnet_tx_hdr: 16, 0, 0, 0, 0, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 VNODE: 284, 0, 1130, 32, 1165, 0, 0 VNODEPOLL: 64, 0, 0, 0, 0, 0, 0 BUF TRIE: 44, 0, 51, 677, 1736, 0, 0 NAMEI: 1024, 0, 0, 16, 21104, 0, 0 rentr: 12, 0, 0, 0, 0, 0, 0 S VFS Cache: 72, 0, 972, 148, 2134, 0, 0 STS VFS Cache: 92, 0, 0, 0, 0, 0, 0 L VFS Cache: 292, 0, 0, 0, 0, 0, 0 LTS VFS Cache: 312, 0, 0, 0, 0, 0, 0 NCLNODE: 360, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 9, 11, 27, 0, 0 Mountpoints: 672, 0, 4, 14, 5, 0, 0 pipe: 416, 0, 0, 9, 534, 0, 0 procdesc: 80, 0, 0, 0, 0, 0, 0 AIO: 128, 0, 0, 0, 0, 0, 0 AIOP: 16, 0, 0, 0, 0, 0, 0 AIOCB: 428, 0, 0, 0, 0, 0, 0 AIOL: 64, 0, 0, 0, 0, 0, 0 AIOLIO: 172, 0, 0, 0, 0, 0, 0 ksiginfo: 80, 0, 30, 170, 1209, 0, 0 itimer: 232, 0, 0, 0, 0, 0, 0 bridge_rtnode: 36, 0, 0, 0, 0, 0, 0 KNOTE: 72, 0, 0, 0, 0, 0, 0 socket: 524, 1393, 20, 8, 2030, 0, 0 unpcb: 172, 1403, 6, 17, 1748, 0, 0 IPsec SA lft_c: 16, 0, 0, 0, 0, 0, 0 ipq: 32, 126, 0, 0, 0, 0, 0 udp_inpcb: 304, 1404, 11, 15, 262, 0, 0 udpcb: 20, 1414, 11, 191, 262, 0, 0 tcp_inpcb: 304, 1404, 2, 11, 11, 0, 0 tcpcb: 772, 1395, 2, 3, 11, 0, 0 tcptw: 60, 335, 0, 0, 0, 0, 0 syncache: 128, 15376, 0, 0, 4, 0, 0 hostcache: 76, 15370, 0, 0, 0, 0, 0 sackhole: 20, 0, 0, 0, 0, 0, 0 tcpreass: 20, 202, 0, 0, 0, 0, 0 sctp_ep: 1052, 1395, 0, 0, 0, 0, 0 sctp_asoc: 1656, 40000, 0, 0, 0, 0, 0 sctp_laddr: 24, 80136, 0, 0, 5, 0, 0 sctp_raddr: 540, 80003, 0, 0, 0, 0, 0 sctp_chunk: 104, 400026, 0, 0, 0, 0, 0 sctp_readq: 108, 400007, 0, 0, 0, 0, 0 sctp_stream_msg_out: 72, 400008, 0, 0, 0, 0, 0 sctp_asconf: 24, 400008, 0, 0, 0, 0, 0 sctp_asconf_ack: 24, 400008, 0, 0, 0, 0, 0 udplite_inpcb: 304, 1404, 0, 0, 0, 0, 0 ripcb: 304, 1404, 0, 0, 0, 0, 0 rtentry: 112, 0, 42, 30, 50, 0, 0 pf mtags: 32, 0, 0, 0, 0, 0, 0 pf states: 208, 10013, 0, 0, 0, 0, 0 pf state keys: 64, 0, 0, 0, 0, 0, 0 pf source nodes: 116, 10030, 0, 0, 0, 0, 0 pf table entries: 92, 0, 0, 0, 0, 0, 0 pf table counters: 64, 0, 0, 0, 0, 0, 0 pf frags: 80, 0, 0, 0, 0, 0, 0 pf frag entries: 24, 5040, 0, 0, 0, 0, 0 pf state scrubs: 28, 0, 0, 0, 0, 0, 0 IPFW counters: 16, 0, 1, 63, 1, 0, 0 IPFW dynamic rule: 112, 16416, 0, 0, 0, 0, 0 divcb: 304, 1404, 0, 0, 0, 0, 0 selfd: 32, 0, 32, 220, 150507, 0, 0 SWAPMETA: 276, 5390, 0, 0, 0, 0, 0 FFS inode: 112, 0, 1105, 155, 1139, 0, 0 FFS1 dinode: 128, 0, 628, 147, 658, 0, 0 FFS2 dinode: 256, 0, 477, 18, 481, 0, 0 md0: 512, 0, 5041, 7, 5049, 0, 0 md1: 512, 0, 4266, 6, 4267, 0, 0= From owner-freebsd-stable@freebsd.org Mon Feb 12 18:02:17 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65433F0D87A for ; Mon, 12 Feb 2018 18:02:17 +0000 (UTC) (envelope-from ws@rot13.io) Received: from continuity.h.rot13.io (continuity.h.rot13.io [IPv6:2a00:d880:5:2c4::42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "rot13.io", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3E46826B5 for ; Mon, 12 Feb 2018 18:02:16 +0000 (UTC) (envelope-from ws@rot13.io) Received: from continuity.h.rot13.io (localhost [127.0.0.1]) by continuity.h.rot13.io (OpenSMTPD) with ESMTP id 80f2f948 for ; Mon, 12 Feb 2018 18:02:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=rot13.io; h=subject:to :references:from:message-id:date:mime-version:in-reply-to :content-type:content-transfer-encoding; s=dkim1; bh=JeGhaICmbHs t2H8FMkCXVSEZ/3A=; b=cDoFhuyLz6YZsPgmGwGZxJZHR8CerzDk6kWqYVpPRGV QgCED8NxfTe00at6ktbhCYcDX/hLrp1Wqc099pFtPkDe3w9bP6Y5iqqaUM9hJ0za b0jPDre+10+vOh5KC8sjk3y9Ie5rDYQyZ8rHuPi9AnaarOuYOvKu9JpuEnKcj3k/ zI8NK9t8Dw0q2tXlO0aB3OnU7g1d++EFSHGGEFpBfaWbhQ7Im9EDWenUqqIBMlR9 9uKAJXy7HJps+OKX4rAN7QqGsqjYiMa0u+flekzT6ynclkBvWMVL0k1XVvhOFLB2 vQ89iW5lk6Z5qBd7zdHubGqw1S+EdxM4V+Hf6K4doqQ== Received: by continuity.h.rot13.io (OpenSMTPD) with ESMTPSA id 7a726149 (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128:NO) for ; Mon, 12 Feb 2018 18:02:13 +0000 (UTC) Subject: Re: [HEADS UP] Mellanox's mlxen.ko will be renamed into mlx4en.ko for 11-stable To: freebsd-stable@freebsd.org References: <76bffd87-c059-8de0-f0b8-e1205d2868da@selasky.org> From: Wilhelm Schuster Message-ID: <0eaf900b-589f-b22d-b0cf-3cc40af87f75@rot13.io> Date: Mon, 12 Feb 2018 19:02:11 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <76bffd87-c059-8de0-f0b8-e1205d2868da@selasky.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 18:02:17 -0000 On 2018-02-12 12:55, Hans Petter Selasky wrote: > Hi, > > The change will happen this week. > This is part of ongoing work in FreeBSD 11-stable. > > Make sure to update your /boot/loader.conf by adding > mlx4en_load="YES" if you are using the mlxen.ko kernel modules. Does that mean, that interface names will also change? For example from mlxen0 to mlx4en0? If so, I also have to update network configuration in /etc/rc.conf for example. From owner-freebsd-stable@freebsd.org Mon Feb 12 18:04:48 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA924F0DB31 for ; Mon, 12 Feb 2018 18:04:47 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6244C82855 for ; Mon, 12 Feb 2018 18:04:46 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w1CI4Y95029292 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 12 Feb 2018 19:04:34 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: ask@develooper.com Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w1CI4TAb039089 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 13 Feb 2018 01:04:29 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: FreeBSD on 64MB memory To: =?UTF-8?Q?Ask_Bj=c3=b8rn_Hansen?= References: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> <5A8123CE.9050609@grosbein.net> Cc: freebsd-stable@freebsd.org From: Eugene Grosbein Message-ID: <5A81D72A.7020408@grosbein.net> Date: Tue, 13 Feb 2018 01:04:26 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 18:04:48 -0000 13.02.2018 0:38, Ask Bjørn Hansen wrote: >>> I have an old Soekris system with 64MB memory that I upgraded from 10.3 to 11.1 recently. Since then it’s started hanging every few days. >> Please show output of commands: >> >> grep memory /var/run/dmesg.boot > > real memory = 67108864 (64 MB) > avail memory = 42098688 (40 MB) > > The 24MB are for the kernel? I wonder my 11.1 kernel is less discriminating with what I compiled in... You should be running custom kernel with absolute minimum. For example, use "options NO_SWAPPING" to compile out swapping code if your system cannot have any swap area. >> top -ores -d1 > > Shortly after boot: > > last pid: 1008; load averages: 0.57, 0.62, 0.53 up 0+00:19:31 06:24:50 > 8 processes: 1 running, 7 sleeping > CPU: % user, % nice, % system, % interrupt, % idle > Mem: 9084K Active, 3644K Inact, 29M Wired, 4862K Buf, 492K Free > Swap: > > PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND > 911 root 1 22 0 8816K 8844K select 0:39 4.20% ntpd Your Soekris system can live without bloated ntpd, use ntpdate or try sntp to periodically check your clock with cron, unless you need to re-distribute NTP to your LAN. > 959 root 1 52 0 10756K 5196K select 0:00 0.00% sshd > 709 root 1 20 0 7300K 3224K select 0:00 0.00% devd Stop using devd (devd_enable="NO"), Soekris system does not need it, at all. >> sysctl kern.ipc.nmbclusters > > kern.ipc.nmbclusters: 898 Way too low. You should double this at very least. >> It would be also very useful to obtain output of "vmstat -z" in a moment of breakage. > ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP > audit_record: 1112, 0, 0, 0, 0, 0, 0 Remove "options AUDIT" from the kernel config unless you really use this. Same for every other non-mandatory kernel option including PF that may be loaded using kernel modules. > mbuf_packet: 256, 5745, 259, 0, 418909, 2, 0 > mbuf: 256, 5745, 263, 273, 773249, 18, 0 > mbuf_cluster: 2048, 898, 259, 131, 204839,1868, 0 This is very bad as FreeBSD TCP/IP stack just live-locks when it runs out of mbufs/mbuf clusters. You want to eliminate mbuf* allocation failures at any cost. > =================================== > ~25 minutes after boot: > > ntp1.us.grundclock.com# vmstat -z > ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP > mbuf_packet: 256, 5745, 128, 253, 32698, 0, 0 > mbuf: 256, 5745, 1, 263, 57270, 0, 0 > mbuf_cluster: 2048, 898, 381, 3, 726, 0, 0 This is very strange because FAIL counters should not decrease. From owner-freebsd-stable@freebsd.org Mon Feb 12 18:30:35 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C750FF0FB22 for ; Mon, 12 Feb 2018 18:30:35 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5947083B94 for ; Mon, 12 Feb 2018 18:30:35 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: c231746b-1022-11e8-b951-f99fef315fd9 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound2.ore.mailhop.org (Halon) with ESMTPSA id c231746b-1022-11e8-b951-f99fef315fd9; Mon, 12 Feb 2018 18:30:11 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w1CIUWHt035207; Mon, 12 Feb 2018 11:30:32 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1518460232.94819.25.camel@freebsd.org> Subject: Re: FreeBSD on 64MB memory From: Ian Lepore To: Eugene Grosbein , Ask =?ISO-8859-1?Q?Bj=F8rn?= Hansen Cc: freebsd-stable@freebsd.org Date: Mon, 12 Feb 2018 11:30:32 -0700 In-Reply-To: <5A81D72A.7020408@grosbein.net> References: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> <5A8123CE.9050609@grosbein.net> <5A81D72A.7020408@grosbein.net> Content-Type: text/plain; charset="iso-8859-13" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 18:30:35 -0000 On Tue, 2018-02-13 at 01:04 +0700, Eugene Grosbein wrote: > 13.02.2018 0:38, Ask Bjrn Hansen wrote: > > > > > > > > > > > > > > I have an old Soekris system with 64MB memory that I upgraded from 10.3 to 11.1 recently. Since then its started hanging every few days. > > > Please show output of commands: > > > > > > grep memory /var/run/dmesg.boot > > real memory= 67108864 (64 MB) > > avail memory = 42098688 (40 MB) > > > > The 24MB are for the kernel?I wonder my 11.1 kernel is less discriminating with what I compiled in... > You should be running custom kernel with absolute minimum. > For example, use "options NO_SWAPPING" to compile out swapping code if your system > cannot have any swap area. > > > > > > > > > top -ores -d1 > > Shortly after boot: > > > > last pid:1008;load averages:0.57,0.62,0.53up 0+00:19:3106:24:50 > > 8 processes:1 running, 7 sleeping > > CPU:% user,% nice,% system,% interrupt,% idle > > Mem: 9084K Active, 3644K Inact, 29M Wired, 4862K Buf, 492K Free > > Swap: > > > > PID USERNAMETHR PRI NICESIZERES STATETIMEWCPU COMMAND > > 911 root12208816K8844K select0:394.20% ntpd > Your Soekris system can live without bloated ntpd, use ntpdate or try sntp > to periodically check your clock with cron, unless you need to re-distribute > NTP to your LAN. > Heh. I think 1) you don't realize you're saying "you don't need ntpd" to, and 2) you didn't notice the hostname of the system in some of the debugging output (ntp1.us.grundclock.com). :) 24MB physmem gone before the kernel even starts seems a little much. I wonder if some amount of that is being eaten up by a video frame buffer that maybe isn't needed on a headless system? -- Ian From owner-freebsd-stable@freebsd.org Mon Feb 12 18:40:34 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 247A7F109FF for ; Mon, 12 Feb 2018 18:40:34 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9EF2A846E8; Mon, 12 Feb 2018 18:40:33 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w1CIeL1c029531 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 12 Feb 2018 19:40:22 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: ian@freebsd.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w1CIeIMH049364 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 13 Feb 2018 01:40:18 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: FreeBSD on 64MB memory To: Ian Lepore , =?UTF-8?Q?Ask_Bj=c3=b8rn_Hansen?= References: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> <5A8123CE.9050609@grosbein.net> <5A81D72A.7020408@grosbein.net> <1518460232.94819.25.camel@freebsd.org> Cc: freebsd-stable@freebsd.org From: Eugene Grosbein Message-ID: <5A81DF8F.7070000@grosbein.net> Date: Tue, 13 Feb 2018 01:40:15 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <1518460232.94819.25.camel@freebsd.org> Content-Type: text/plain; charset=iso-8859-13 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 18:40:34 -0000 13.02.2018 1:30, Ian Lepore wrote: >>> PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND >>> 911 root 1 22 0 8816K 8844K select 0:39 4.20% ntpd >> Your Soekris system can live without bloated ntpd, use ntpdate or try sntp >> to periodically check your clock with cron, unless you need to re-distribute >> NTP to your LAN. >> > > Heh. I think 1) you don't realize you're saying "you don't need ntpd" > to, and 2) you didn't notice the hostname of the system in some of the > debugging output (ntp1.us.grundclock.com). :) You are partialy right :-) I skipped hostname. Btw, is Soektris system has good enough hardware clock and/or enough horsepower to provide quality public NTP service? Also thinking of lots of garbage traffic these days UDP/123 suffers from... From owner-freebsd-stable@freebsd.org Mon Feb 12 18:43:45 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C01ABF10FC7 for ; Mon, 12 Feb 2018 18:43:45 +0000 (UTC) (envelope-from menyy@mellanox.com) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0067.outbound.protection.outlook.com [104.47.1.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28DC084C0C for ; Mon, 12 Feb 2018 18:43:41 +0000 (UTC) (envelope-from menyy@mellanox.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=AcBmmvgyDm3oOtBxmSKidz5iMGJ+XvBCz8+wVFrvD1I=; b=Pcfek7y9EDCBg6frBH1iJ/3WK8CQUw767Q46BQmh/vF3+YCiIo9dimoXCuXPlyDWjBZtdgGK+K/UmcRnuPfpn7VuYhnDAp+yS5OJ4RENHR+gxE/BJXJCzspCJGgyKGSS8XjuDzXp5uhcy9ORM3PYRoobP1uwbJEvwS4Ie0HPSQI= Received: from VI1PR0501MB2863.eurprd05.prod.outlook.com (10.172.12.8) by VI1PR0501MB1934.eurprd05.prod.outlook.com (10.166.44.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.485.10; Mon, 12 Feb 2018 18:43:39 +0000 Received: from VI1PR0501MB2863.eurprd05.prod.outlook.com ([fe80::910e:557d:9f47:9cb1]) by VI1PR0501MB2863.eurprd05.prod.outlook.com ([fe80::910e:557d:9f47:9cb1%17]) with mapi id 15.20.0485.015; Mon, 12 Feb 2018 18:43:39 +0000 From: Meny Yossefi To: "'FreeBSD-stable@FreeBSD.org'" CC: Slava Shwartsman , Hans Petter Selasky , Konstantin Belousov Subject: RE: [HEADS UP] Mellanox's mlxen.ko will be renamed into mlx4en.ko for 11-stable Thread-Topic: [HEADS UP] Mellanox's mlxen.ko will be renamed into mlx4en.ko for 11-stable Thread-Index: AQHTpCvWg0A/vCFZYUa9AiUHNxdXyKOhGjCQ Date: Mon, 12 Feb 2018 18:43:39 +0000 Message-ID: References: <76bffd87-c059-8de0-f0b8-e1205d2868da@selasky.org>, <0eaf900b-589f-b22d-b0cf-3cc40af87f75@rot13.io> <8b421e4cd4214f08b75d9617979e8249@AM5PR0502MB2916.eurprd05.prod.outlook.com> In-Reply-To: <8b421e4cd4214f08b75d9617979e8249@AM5PR0502MB2916.eurprd05.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [185.125.14.156] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR0501MB1934; 6:jX3P+79uCAjDnmIpHEEieGm7edawlA2rs1vjaTSDSeJ9Id3dUgU3EHprzKucffTjxoPMOrXnFEO19xG++2q8udJUbikwghwkTFXQx74WAXAoWjMCnHHbxm6+2h97FNo9c1CIf5HR/vikoXUKdMFxvGO5FIzPSiZoWjjLahLWK0jK7kcjKdkYSo8DYjyVg53ZSF1i/xQ+ih746iFX5Aq59ViW2GXgjGqrN8/rIAybyCn/poNmc9ZYXxGYaqHpQdFCMFtDqECBZgj1FqNkpNJXYlnG/dFFwfEeFBHuD+y++pq0rAublHxy8Bi48BRphdhkCnf9BW+8ouOLncIONCw/DHJWDkxcHKrZU6UgdBzbA9ScG/+9I9Ezn6nLJR2ib4F6; 5:2SOrrryDX0SaIdVUh+1qjhOcdyAmboR5G36Jd6cCnueEzvrUrY8urUfwTf4ydHMwXofK9d3zgF8jb90MSNqb3G/x++CUigbtC+4KzZAv9sM/LSx3E1u/C/NUWGTmkNZvusc/RZcbOk/iyA9oDsGQIhhEoQmd1ugtAZhMVsIfFaM=; 24:lIqGJXWyQ/GjLryXyGQsf6fR/7dH/OnNkc3knQ6c5bN+z58TStfu5Ea3c1HRPetUBT9tzday9Qm8wkSi2luSj8j+zYMFwtp64wTBFlj1Soo=; 7:Xcgh3pZVD/A8gF65fwWLQyX2BK0HvIIkPJidYCnmeOjq0Bn4phbz+MSUedn/62Y+wG3ucipvvJJaFTdubCry73QayWjfJcRj5L3FtUxvjNx/tO8huHNUFiws8tai/4FsPea+nCVwqpmm1D9VEwEc97eAMEaiX2Fky8o7neeZCNvcE9CFjXqmjHk7o3y6jrrHuxi/H4YCSxexUVDcgJp4NCyC1pWmvx/0zHPiWXLa2IjneHMr9SXlql+OeoU9Np2M x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: af13e422-cf48-4246-74c7-08d57248886e x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603307)(7153060)(7193020); SRVR:VI1PR0501MB1934; x-ms-traffictypediagnostic: VI1PR0501MB1934: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(189930954265078)(45079756050767); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(10201501046)(3231101)(2400082)(944501161)(6055026)(6041288)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123560045)(20161123564045)(6072148)(201708071742011); SRVR:VI1PR0501MB1934; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0501MB1934; x-forefront-prvs: 0581B5AB35 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39380400002)(396003)(376002)(366004)(39860400002)(346002)(199004)(189003)(377424004)(45080400002)(53936002)(2906002)(6116002)(97736004)(3846002)(6436002)(3660700001)(478600001)(2900100001)(33656002)(55016002)(105586002)(86362001)(6246003)(9686003)(6306002)(3280700002)(25786009)(107886003)(966005)(2950100002)(102836004)(14454004)(81156014)(316002)(5250100002)(6916009)(229853002)(54906003)(6506007)(7736002)(8676002)(74316002)(186003)(305945005)(53546011)(106356001)(5660300001)(6346003)(59450400001)(26005)(66066001)(76176011)(8936002)(68736007)(81166006)(7696005)(99286004)(4326008)(32563001)(491001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB1934; H:VI1PR0501MB2863.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=menyy@mellanox.com; x-microsoft-antispam-message-info: kiw6v0LvUkJFQGHUO1MXbZofvzsDSKTeRywaUH1AKV3Re0voGXhuTR9aO1QbQlKW6GzxCVYgkr4IshDlVB9aVw== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: af13e422-cf48-4246-74c7-08d57248886e X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Feb 2018 18:43:39.3851 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB1934 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 18:43:46 -0000 No, interface names will remain as is. -Meny ________________________________________ From: owner-freebsd-stable@freebsd.orgOn Behalf OfWilhelm Schuster Sent: Monday, February 12, 2018 6:02:11 PM (UTC+00:00) Monrovia, Reykjavik To: freebsd-stable@freebsd.org Subject: Re: [HEADS UP] Mellanox's mlxen.ko will be renamed into mlx4en.ko = for 11-stable On 2018-02-12 12:55, Hans Petter Selasky wrote: > Hi, > > The change will happen this week. > This is part of ongoing work in FreeBSD 11-stable. > > Make sure to update your /boot/loader.conf by adding mlx4en_load=3D"YES"= =20 > if you are using the mlxen.ko kernel modules. Does that mean, that interface names will also change? For example from mlxen0 to mlx4en0? If so, I also have to update network configuration in /e= tc/rc.conf for example. _______________________________________________ freebsd-stable@freebsd.org mailing list https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists.= freebsd.org%2Fmailman%2Flistinfo%2Ffreebsd-stable&data=3D02%7C01%7Cfreebsd-= commits-tracker%40mellanox.com%7C0271be467f734782062f08d57242f763%7Ca652971= c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636540554308599515&sdata=3DhzhwlzZ3mDAv= eb8vjgW7PCQAilyykqipUeLshDaTges%3D&reserved=3D0 To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Feb 12 18:47:48 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 585C3F11576 for ; Mon, 12 Feb 2018 18:47:48 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D9C1D850CE for ; Mon, 12 Feb 2018 18:47:47 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 2c6e8826-1025-11e8-bb8e-b35b57339d60 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 2c6e8826-1025-11e8-bb8e-b35b57339d60; Mon, 12 Feb 2018 18:47:28 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w1CIleno035276; Mon, 12 Feb 2018 11:47:40 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1518461260.94819.36.camel@freebsd.org> Subject: Re: FreeBSD on 64MB memory From: Ian Lepore To: Eugene Grosbein , Ask =?ISO-8859-1?Q?Bj=F8rn?= Hansen Cc: freebsd-stable@freebsd.org Date: Mon, 12 Feb 2018 11:47:40 -0700 In-Reply-To: <5A81DF8F.7070000@grosbein.net> References: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> <5A8123CE.9050609@grosbein.net> <5A81D72A.7020408@grosbein.net> <1518460232.94819.25.camel@freebsd.org> <5A81DF8F.7070000@grosbein.net> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 18:47:48 -0000 On Tue, 2018-02-13 at 01:40 +0700, Eugene Grosbein wrote: > 13.02.2018 1:30, Ian Lepore wrote: > > > > > > > > > > > > > > PID USERNAMETHR PRI NICESIZERES STATETIMEWCPU COMMAND > > > > 911 root12208816K8844K select0:394.20% ntpd > > > Your Soekris system can live without bloated ntpd, use ntpdate or try sntp > > > to periodically check your clock with cron, unless you need to re-distribute > > > NTP to your LAN. > > > > > Heh.I think 1) you don't realize you're saying "you don't need ntpd" > > to, and 2) you didn't notice the hostname of the system in some of the > > debugging output (ntp1.us.grundclock.com).:) > You are partialy right :-) I skipped hostname. > > Btw, is Soektris system has good enough hardware clock and/or > enough horsepower to provide quality public NTP service? > Also thinking of lots of garbage traffic these days UDP/123 suffers from... Should be plenty of horsepower. For years I ran a pool.ntp.org server using a 60mhz armv4 system with 64MB ram. You don't need anything special in the way of a clock to run a public ntp server at stratum 2 or lower and achieve millisecond accuracy (or sub-millisecond at stratum 1, with a PPS input). I've never seen a system whose kernel timekeeping was so bad that ntpd couldn't steer the clock and provide accurate time. -- Ian From owner-freebsd-stable@freebsd.org Mon Feb 12 19:57:33 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC5E5F17670 for ; Mon, 12 Feb 2018 19:57:33 +0000 (UTC) (envelope-from freebsd.ed.lists@sumeritec.com) Received: from out1-5.antispamcloud.com (out1-5.antispamcloud.com [185.201.16.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 35FF46A7FB for ; Mon, 12 Feb 2018 19:57:32 +0000 (UTC) (envelope-from freebsd.ed.lists@sumeritec.com) Received: from [153.92.8.106] (helo=srv31.niagahoster.com) by mx1.antispamcloud.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1el8KG-0001Ac-0D; Mon, 12 Feb 2018 08:15:08 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sumeritec.com; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=Ap2yv2SFKC0u36QHr4WkG4RSlO4wlAr3AAPqnMOJGKo=; b=aqppG0JdJV7jy0UZU1IvJ4wbVx vSi1MFRGBNms5GpXqQUXJpgfrO2/rxigHNNoL+HmCz6HGcqDryoviwqM5KSIwhuqEmoLes08SugtU sUEN19OWUtttIeESC5lp8GAxnEzPe4tupSmjWSHiqxTy7qqpA0MbcNMGRNcCVLoMN9whXA/HISSri pQmL+4XdRLE8lVLa+eQyj83LcpJpYYBtIK7LALNxlNj3Mu7vO6uRGHk1iOCvVicfneuLyvBk5XyDY 1HW1zUnqX1crpHewIfv75bnRlZP9SBGxYDE5cjGJnTQYMPH7FNRleTNFdIYQESuBwFwb3t6Feta1w KifT79VQ==; Received: from [114.125.92.253] (port=25026 helo=X220.sumeritec.com) by srv31.niagahoster.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89_1) (envelope-from ) id 1el8JM-0002Vy-C6; Mon, 12 Feb 2018 14:14:03 +0700 Date: Mon, 12 Feb 2018 15:13:57 +0800 From: Erich Dollansky To: Ask =?ISO-8859-1?Q?Bj=F8rn?= Hansen Cc: freebsd-stable@freebsd.org Subject: Re: FreeBSD on 64MB memory Message-ID: <20180212151357.184d7770.freebsd.ed.lists@sumeritec.com> In-Reply-To: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> References: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-OutGoing-Spam-Status: No, score=-1.0 X-AuthUser: freebsd.ed.lists@sumeritec.com X-Originating-IP: 153.92.8.106 X-AntiSpamCloud-Domain: out.niagahoster.com X-AntiSpamCloud-Username: niaga Authentication-Results: antispamcloud.com; auth=pass (login) smtp.auth=niaga@out.niagahoster.com X-AntiSpamCloud-Outgoing-Class: ham X-AntiSpamCloud-Outgoing-Evidence: SB/global_tokens (0.00787421595178) X-Recommended-Action: accept X-Filter-ID: EX5BVjFpneJeBchSMxfU5sXFpuX/f+FdH3E2uxwZeJJ602E9L7XzfQH6nu9C/Fh9KJzpNe6xgvOx q3u0UDjvOxwhh/OHXq89yTMKyt5jpmpac4fNxy319JrRvZ5uLQtIwrVdqn10OJsELCKo2XfnjsdO DYx3b4Y9hvshG6sNAwvG5+Wfeqdslw38xvMxoVpGJODXbtOodkPED+RkHjVGH11lTRTjZf0mPBUL +KQg6/9C4gy1YsDNtkK3ahncA28BK9tLbEmaBmPsQT3aIsBQFbIg7hEshE8+KidcqK9y+QX9ea98 9PUolcdtwICs7KPV5p/0jHI+4jcFgXWf7G0ncztWqrVH3GZyYxeOwcH765j+OVg/7O+tyVElvJ63 vY0Iza6sVdgwkBECjdoM4toPI22pHDwwk7Jb2vlaEyyktkqzFg7Z5LN4nMmqB2NPQrxb173FVX4T cHC2qm5yu5XoZe3pH5DzvDUqekOgPmwIpau8Ksk+aedMfNWSnJswrtlN+IC1NNIHAUMsnQ2fiRx3 /nhktjZR2pVN6aLnobJtRlJuKVf4Swi7SMDyfVwYs6ZH2DLBGg09NvMwDSUc3oIzcoHlTa0FG0ij 7AOlTYklM4I40c2A1DXercZtdf13/qYSnnXm95a0vRtjH3/UwFOlrbOSUAQXUBlrmK2rF329IE/i McIO1WBOnfnW3cHBdnHghT2dEwOjLi+upP7uneYU4xEVniihuDwEGDcmr6e3OPRCqLRApsCQBMb6 jTfResuzKQ5Y/cN9J6/CsRi2IpUkONOReSCSipZZUzz4DwV6jL/qzjqUUzzqs981AUGGae0Tk6e1 OAXxpqsV9tF+uU2A4bl3LUMNQoMoFc+1691Jx6+c0TwPIBJnngHP89wHBJTrgyYAb+RIMMHEuJ/m 6mYN+8ujJ1u47LWL1J+ZiDEFureZ3JKVmi72ocgY5kMQSjs7493d5uYq65xRXNdxWp8TBPYnewwE ECsG/veHVwE17liIweYKCLpLqxRRoM04EfWF X-Report-Abuse-To: spam@quarantine1.antispamcloud.com X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 19:57:34 -0000 Hi, I do not know if this helps here. I faced also memory problems on the older Raspberry generations. Enabling swap helped even with no swap in use. Maybe, you also try it. Erich On Sun, 11 Feb 2018 20:56:02 -0800 Ask Bj=C3=B8rn Hansen wrote: > Hi, >=20 > I have an old Soekris system with 64MB memory that I upgraded from > 10.3 to 11.1 recently. Since then it=E2=80=99s started hanging every few = days. >=20 > Today I happened to have a =E2=80=9Ctop=E2=80=9D instance running on the = serial > console. The system is minimally responsive to the network (ICMP and > CARP are working, but no services). >=20 > From the top output it=E2=80=99s not clear what resource it=E2=80=99s out= of. > There=E2=80=99s no swap configured, but that what it looks like it=E2=80= =99s trying > to do?=20 >=20 > The =E2=80=98pf purge=E2=80=99 process is suspicious. There are no pf rul= es > configured on the system (it should be all disabled). >=20 > Any suggestions? (Other than =E2=80=9Cseriously =E2=80=A6 64MB memory?!= =E2=80=9D). >=20 >=20 > Ask >=20 >=20 > last pid: 2228; load averages: 0.63, 0.65, 0.70 up > 0+21:13:56 04:50:47 36 processes: 2 running, 33 sleeping, 1 > waiting CPU: 0.1% user, 0.0% nice, 11.3% system, 2.4% interrupt, > 86.2% idle Mem: 1616K Active, 10M Inact, 28M Wired, 3099K Buf, 1768K > Free Swap: >=20 > PID UID THR PRI NICE SIZE RES STATE TIME WCPU > COMMAND 11 0 1 155 ki31 0K 8K RUN 16.8H 75.53% > idle 0 0 7 -16 - 0K 64K swapin 155:10 12.00% > kernel 16 0 3 -16 - 0K 24K psleep 10:04 8.15% > pagedaemon 12 0 14 -64 - 0K 112K WAIT 27:21 > 2.80% intr 6 0 1 -16 - 0K 8K pftm 8:56 > 0.73% pf purge 1027 0 1 20 0 7272K 1140K RUN > 6:04 0.52% top 7 0 1 -16 - 0K 8K - > 2:57 0.18% rand_harvestq 21 0 1 16 - 0K 8K > syncer 0:22 0.05% syncer 22 0 1 21 - 0K 8K > vlruwt 0:07 0.01% vnlru 19 0 1 20 - 0K 8K > psleep 0:08 0.01% bufdaemon 788 0 1 20 0 5920K > 1752K select 0:04 0.01% syslogd 20 0 1 20 - > 0K 8K - 0:07 0.01% bufspacedaemon 911 0 1 > 20 0 8816K 8844K kmem a 43:41 0.00% ntpd 996 0 1 > 20 0 13628K 4620K kmem a 0:06 0.00% sshd 985 0 1 > 20 0 5952K 584K kmem a 0:07 0.00% cron 709 0 1 > 20 0 7300K 3364K kmem a 0:01 0.00% devd 2227 0 1 > 24 0 5952K 1232K kmem a 0:00 0.00% cron > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable To > unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Feb 12 21:41:54 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E908F1FABE for ; Mon, 12 Feb 2018 21:41:54 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x229.google.com (mail-io0-x229.google.com [IPv6:2607:f8b0:4001:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B85696F64C for ; Mon, 12 Feb 2018 21:41:53 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x229.google.com with SMTP id n7so18974618iob.0 for ; Mon, 12 Feb 2018 13:41:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=FTwgrK/XJLrCHduWLJR6yT6dVIxqKNFiVI77QpPOv3A=; b=EgUQ+xoCb5z9REYqR/v5WLvD1JUl72Izt0iyNqvHMSbNWfFznOLMPPfDF//uKyfHz4 v8+KpRUJ/Xcv2s5bUQVoo3HpYMXL1yMb7IMB1Tt8ER+IkTBJm2cFuBtw/SME3+Q3Zf5J r+yr2zHvf1Vl9+DxC8DPHxxGHYgivBI50Z/V2DoFmXD3QByGLxxJ1jrk1FyocuG/VVqn R8u9pcapfVJka9Z5nUdqOY4MNPkFJB35+12Sxq2bkDOLlFVxeHmiBXrffs4zpWEigP+j NwB8XoNl4+TBtGUowKI0CvRkaQ0kXCkNSuuH0gvRWqknXmfEl4D2yfs/Im7U74SX3j20 gDmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=FTwgrK/XJLrCHduWLJR6yT6dVIxqKNFiVI77QpPOv3A=; b=a9hiBsQaUhu4hmZOj+SipJQMvM3HE38KFX0B2QbQfsvP8/U/0A/Km1eu5zyvPkOThk M8MmW4ipMZnPe/KArQMR5xqnWeE8P6GFUZAedU+V1WcsySTWLKPk5GPrFP6kuRGGQN35 ecf7T9q2q/gRfNfgMBZx2Mw9BdfV+AYePl8LW3ZeF2cvKcqQFTtYcMEDhSbR7ZV552yh r/dGbG22xH8L3VHMZYcrs/gWyGT2X7XCCCnpBAHWyt0i8MCOBPqYPRLjT/G0PAFoyv1g Hm4fS/ycla8JrIO0p7T/dez8t6/WrwUysNi6qLWlgbIGKLL8dwcMKZbtvo/W7SD8GHTe 3/cw== X-Gm-Message-State: APf1xPAVz4cOcandWpViFoSke/xTWPxGXLovPM5HOhw0LWiKMsIFFLjB MgZdTI7LJ9TSDswsnxBHwusd2/Tf508jak0QFWBLGg== X-Google-Smtp-Source: AH8x226hietx3OI2BNc+kR+AhnqP2h8zObjszViSYWhWCAPAyeRNztxOJ+DjkAwcPLLUN+es72tVjBzMXTgKHcTDUbI= X-Received: by 10.107.107.1 with SMTP id g1mr14402508ioc.63.1518471713033; Mon, 12 Feb 2018 13:41:53 -0800 (PST) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.201.67 with HTTP; Mon, 12 Feb 2018 13:41:52 -0800 (PST) X-Originating-IP: [50.253.99.174] In-Reply-To: References: <5FB97479-C49D-4C6E-8416-015ECA656C14@develooper.com> <5A8123CE.9050609@grosbein.net> From: Warner Losh Date: Mon, 12 Feb 2018 14:41:52 -0700 X-Google-Sender-Auth: M5YQNK-QbLg4Buu7nqrqaSALABc Message-ID: Subject: Re: FreeBSD on 64MB memory To: =?UTF-8?Q?Ask_Bj=C3=B8rn_Hansen?= Cc: Eugene Grosbein , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 21:41:54 -0000 On Mon, Feb 12, 2018 at 10:38 AM, Ask Bj=C3=B8rn Hansen wrote: > > real memory =3D 67108864 (64 MB) > avail memory =3D 42098688 (40 MB) > > The 24MB are for the kernel? I wonder my 11.1 kernel is less > discriminating with what I compiled in... > I'd suggest a non-GENERIC kernel. The GENERIC kernel is like 24MB these days, while a tailored one is more like 5MB or 6MB leaving you with ~55MB after initial memory allocations. Warner From owner-freebsd-stable@freebsd.org Mon Feb 12 21:49:54 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E48CEF2061F for ; Mon, 12 Feb 2018 21:49:53 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mx2.catspoiler.org (mx2.catspoiler.org [IPv6:2607:f740:16::d18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "amnesiac", Issuer "amnesiac" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 837246FC4E; Mon, 12 Feb 2018 21:49:53 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org ([76.212.85.177]) by mx2.catspoiler.org (8.15.2/8.15.2) with ESMTPS id w1CLoQCX066457 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 12 Feb 2018 21:50:28 GMT (envelope-from truckman@FreeBSD.org) Received: from mousie.catspoiler.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTPS id w1CLejaW069529 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 12 Feb 2018 13:49:40 -0800 (PST) (envelope-from truckman@FreeBSD.org) Date: Mon, 12 Feb 2018 13:49:38 -0800 (PST) From: Don Lewis Subject: Re: Ryzen issues on FreeBSD ? (with sort of workaround) To: Eugene Grosbein cc: Mike Tancsa , freebsd-stable@freebsd.org, Peter Moody , Andriy Gapon , Pete French In-Reply-To: <5A71C5AE.6020202@grosbein.net> Message-ID: References: <8e842dec-ade7-37d1-6bd8-856ea1a827ca@sentex.net> <9b769e4e-b098-b294-0bce-8bb1c42e8a59@rootautomation.com> <730eb882-1c6a-afb7-0ada-396db44fb34b@ingresso.co.uk> <8b882970-4d5d-2a96-4dac-779cab07b9ae@sentex.net> <343acf99-3e9e-093a-7390-c142396c2985@sentex.net> <3dd9a61b-511d-db2e-80ca-cbc9a4b65f92@sentex.net> <55913e41-3a8a-9a4d-6862-e09a3d0f4d55@sentex.net> <5e48bbc2-e872-46bd-eece-25acbb180f77@sentex.net> <5A71C5AE.6020202@grosbein.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=iso-8859-5 Content-Transfer-Encoding: QUOTED-PRINTABLE Content-Disposition: INLINE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 21:49:54 -0000 On 31 Jan, Eugene Grosbein wrote: > 31.01.2018 4:36, Mike Tancsa =DF=D8=E8=D5=E2: >> On 1/30/2018 2:51 PM, Mike Tancsa wrote: >>> >>> And sadly, I am still able to hang the compile in about the same place. >>> However, if I set >>=20 >>=20 >> OK, here is a sort of work around. If I have the box a little more busy, >> I can avoid whatever deadlock is going on. In another console I have >> cat /dev/urandom | sha256 >> running while the build runs >>=20 >> ... and I can compile net/samba47 from scratch without the compile >> hanging. This problem also happens on HEAD from today. Should I start >> a new thread on freebsd-current ? Or just file a bug report ? >> The compile worked 4/4 >=20 > That's really strange. Could you try to do "sysctl kern.eventtimer.period= ic=3D1" > and re-do the test without extra load? I'm having really good luck with the kernel patch attached to this message: https://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D417183+0+archive/2018/freeb= sd-hackers/20180211.freebsd-hackers Since applying that patch, I did three poudriere runs to build the set of ~1700 ports that I use. Other than one gmake-related build runaway that I've also seen on my AMD FX-8320E, I didn't see any random port build failures. When I was last did some testing a few weeks ago, lang/go would almost always fail. I also would seem random build failures in lang/guile or finance/gnucash (which uses guile during its build) on both my Ryzen and FX-8320E machines, but those built cleanly all three times. I even built samba 16 times in a row without a hang. From owner-freebsd-stable@freebsd.org Tue Feb 13 00:29:53 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7774F0805A for ; Tue, 13 Feb 2018 00:29:53 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf0-x235.google.com (mail-lf0-x235.google.com [IPv6:2a00:1450:4010:c07::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23F4F770E1 for ; Tue, 13 Feb 2018 00:29:53 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf0-x235.google.com with SMTP id 37so3952912lfs.7 for ; Mon, 12 Feb 2018 16:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:cc; bh=jifWi7SV1ewisCKkLy1Fh6xy7qhKF2mBqgZJHCzwVaw=; b=E4OGo1+yw5iK9lrPicc9TpXNORnSQP0gVvuOE7+LqS1EMxjDitwfMAE5NISW+r+Ka0 kM9VwvOgaN3jhnp7/gCuxGvbLBaA8k1/FWXyl+pB2iMBk5XnED0B3Avwz4ICbce9D/FR B/I1GRpl6J0bhewLoicSk1WGnfX+NyS2ofRMQeMKNK2pMgQB/wM9qL5FKr216vvyY1nC Tcthb+9CQ9I4DYhxGVkR6YGggbfFixr4gFOU5T/KA0WSq09eiKgc2gP+4l7UUaWc4bQF 4bC1oAlnxikSL9ECsH8UrY021VGzon78JOI8UFgRjwBlfMZns9YKJgtNr9LPpSUbCxSj 1jlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:cc; bh=jifWi7SV1ewisCKkLy1Fh6xy7qhKF2mBqgZJHCzwVaw=; b=OvSjdxl9R4FU7H90Hi2VakxOJUbl1wN7qVcdd2zxP5PTQcLqeJdYqbK9YKZq9aTD6d RoclOtzVnaHs/Lm/KJKjPZh6di+VTPjXMd5IPrVCn5kDgTOn6iwG9wsAWei1iADFZv+H Ulj27RSg6Zt5Dda6ffvRTaHklrdPrqK+cg7PWpraW+ZBKc4dQHp24U7Md53ZRA0HuNQD h49FD+tqlvhwZ2YQPd1Z22crRJbic2tmxzdKO94DYD8JuINtPOdXbhqeO9uGUOjqnURM D0yMJMQZ+CowyKbynwD0yT4Kk+p0zOjILLdG418pmozYQ1QTsc85mZZnytF20PmemGCa VD/A== X-Gm-Message-State: APf1xPBhcbM3vZPbEgKLyZAZJIw4ynOIRPbtCPIdXkO4o+F6FsuIPsLl zyd7+RZDAj5M03yexy7cU7MzxQ8Cqle9gjOvhiI= X-Received: by 10.25.229.12 with SMTP id c12mt8918184lfh.106.1518481791490; Mon, 12 Feb 2018 16:29:51 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.179.67.243 with HTTP; Mon, 12 Feb 2018 16:29:50 -0800 (PST) In-Reply-To: References: <0b170dae-b816-ea49-3516-40bfd1deaa2a@bsquare.com> From: Alan Somers Date: Mon, 12 Feb 2018 17:29:50 -0700 X-Google-Sender-Auth: f7duSSYWVRzL6PLCfiwp62XIC5k Message-ID: Subject: Re: Clock occasionally jumps backwards on 11.1-RELEASE Cc: Mike Pumford , FreeBSD Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 00:29:53 -0000 On Tue, Jan 23, 2018 at 8:40 AM, Alan Somers wrote: > On Tue, Jan 23, 2018 at 3:48 AM, Mike Pumford > wrote: > >> On 22/01/2018 17:07, Alan Somers wrote: >> >>> Since upgrading my jail server to 11.1-RELEASE, the clock occasionally >>> jumps backwards by 5-35 minutes for no apparent reason. Has anybody seen >>> something like this? >>> >>> Details >>> ===== >>> >>> * Happens about once a day on my jail server, and has happened at least >>> once on a separate bhyve server. >>> >>> * The jumps almost always happen between 1 and 3 AM, but I've also seen >>> them happen at 06:30 and 20:15. >>> >>> That's the window when the period scripts are run which if you have a >> default configuration and a lot of jails will put the system under a lot of >> stress. >> > > That did not fail to escape my notice. However, none of the jails' > periodic jobs involve the clock in any way. And I wouldn't think that a > high CPU load could cause clock drift, could it? This isn't Windows XP, > after all. > > >> * I'm using the default ntp.conf file. >>> >>> Are you running ntpd inside the jail or on the jail host? On my jail >> systems (which are 10.3 and 11.1) I run ntpd out the jail host (outside all >> jails) and not inside the jails and the jails then get the accurate time as >> the underlying host has accurate time. >> > > Only on the host. > > New info: there is a possibility that my NFS server is hanging for > awhile. That would explain my problem's timing. However, ntpd shouldn't > be accessing any NFS shares, and I wouldn't think that a hung NFS server > should be able to pause the clock. I'm doing a new experiment that should > be more informative. But I'll have to wait until the problem recurs to > learn anything. > I have a little more data now. The problem happens much more frequently than I originally realized, but usually for just a few seconds at a time. It looks like the system is hanging for awhile and then recovering. Or at least, the clocks are hanging. The only other possibility would be for both the realtime _and_ monotonic clocks to jump backwards. In any case, the problem is not ntpd's fault. I don't know what could cause a system to hang for up to 30 minutes without crashing, and I'm not sure how to tell unless it happens during working hours. I'll send another update if I learn more. -Alan From owner-freebsd-stable@freebsd.org Tue Feb 13 00:46:38 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F025EF0967F for ; Tue, 13 Feb 2018 00:46:37 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D7C977CDD for ; Tue, 13 Feb 2018 00:46:37 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 4fa723cf-1057-11e8-bb8e-b35b57339d60 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 4fa723cf-1057-11e8-bb8e-b35b57339d60; Tue, 13 Feb 2018 00:46:22 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w1D0kX0Q036135; Mon, 12 Feb 2018 17:46:33 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1518482793.85310.8.camel@freebsd.org> Subject: Re: Clock occasionally jumps backwards on 11.1-RELEASE From: Ian Lepore To: Alan Somers Cc: FreeBSD Date: Mon, 12 Feb 2018 17:46:33 -0700 In-Reply-To: References: <0b170dae-b816-ea49-3516-40bfd1deaa2a@bsquare.com> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 00:46:38 -0000 On Mon, 2018-02-12 at 17:29 -0700, Alan Somers wrote: > On Tue, Jan 23, 2018 at 8:40 AM, Alan Somers wrote: > > > > > On Tue, Jan 23, 2018 at 3:48 AM, Mike Pumford > > wrote: > > > > > > > > On 22/01/2018 17:07, Alan Somers wrote: > > > > > > > > > > > Since upgrading my jail server to 11.1-RELEASE, the clock occasionally > > > > jumps backwards by 5-35 minutes for no apparent reason.Has anybody seen > > > > something like this? > > > > > > > > Details > > > > ===== > > > > > > > > * Happens about once a day on my jail server, and has happened at least > > > > once on a separate bhyve server. > > > > > > > > * The jumps almost always happen between 1 and 3 AM, but I've also seen > > > > them happen at 06:30 and 20:15. > > > > > > > > That's the window when the period scripts are run which if you have a > > > default configuration and a lot of jails will put the system under a lot of > > > stress. > > > > > That did not fail to escape my notice.However, none of the jails' > > periodic jobs involve the clock in any way.And I wouldn't think that a > > high CPU load could cause clock drift, could it?This isn't Windows XP, > > after all. > > > > > > > > > > * I'm using the default ntp.conf file. > > > > > > > > > > > > Are you running ntpd inside the jail or on the jail host? On my jail > > > systems (which are 10.3 and 11.1) I run ntpd out the jail host (outside all > > > jails) and not inside the jails and the jails then get the accurate time as > > > the underlying host has accurate time. > > > > > Only on the host. > > > > New info: there is a possibility that my NFS server is hanging for > > awhile.That would explain my problem's timing.However, ntpd shouldn't > > be accessing any NFS shares, and I wouldn't think that a hung NFS server > > should be able to pause the clock.I'm doing a new experiment that should > > be more informative.But I'll have to wait until the problem recurs to > > learn anything. > > > I have a little more data now.The problem happens much more frequently > than I originally realized, but usually for just a few seconds at a time. > It looks like the system is hanging for awhile and then recovering.Or at > least, the clocks are hanging.The only other possibility would be for > both the realtime _and_ monotonic clocks to jump backwards.In any case, > the problem is not ntpd's fault.I don't know what could cause a system to > hang for up to 30 minutes without crashing, and I'm not sure how to tell > unless it happens during working hours.I'll send another update if I > learn more. > > -Alan Under the hood, CLOCK_REALTIME and CLOCK_MONOTONIC are the same hardware; if one stops, both do. CLOCK_REALTIME is simply offset somewhat from MONOTONIC, and the offset can change (which is why REALTIME isn't monotonic). If you want to take a snapshot of some independently-running clocks, you can use sysctl kern.timecounter.tc..counter, but that is a snapshot of the raw hardware counter for the clock, which usually rolls over pretty quickly (.mask tells you how many bits the counter has, that and .frequency can be used to figure out how fast it rolls over). -- Ian From owner-freebsd-stable@freebsd.org Tue Feb 13 01:05:17 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62DBFF0AED7 for ; Tue, 13 Feb 2018 01:05:17 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "m5p.com", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D91C578D39 for ; Tue, 13 Feb 2018 01:05:16 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:470:1f07:15ff::1f] (haymarket.m5p.com [IPv6:2001:470:1f07:15ff::1f]) (authenticated bits=0) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTPSA id w1D0nVnM011068 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 12 Feb 2018 19:49:40 -0500 (EST) (envelope-from george+freebsd@m5p.com) Subject: Re: Ryzen issues on FreeBSD (can we sum up yet)? To: freebsd-stable@freebsd.org References: <8e842dec-ade7-37d1-6bd8-856ea1a827ca@sentex.net> <730eb882-1c6a-afb7-0ada-396db44fb34b@ingresso.co.uk> <8b882970-4d5d-2a96-4dac-779cab07b9ae@sentex.net> <343acf99-3e9e-093a-7390-c142396c2985@sentex.net> <3dd9a61b-511d-db2e-80ca-cbc9a4b65f92@sentex.net> <55913e41-3a8a-9a4d-6862-e09a3d0f4d55@sentex.net> <5e48bbc2-e872-46bd-eece-25acbb180f77@sentex.net> <5A71C5AE.6020202@grosbein.net> From: George Mitchell Message-ID: <11382853-af10-c629-1e6a-21ab8fc37f53@m5p.com> Date: Mon, 12 Feb 2018 19:49:24 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="UHiw2ZLU1f9rkPRlZ9ZHFIzgb2BrHpPxL" X-Spam-Status: No, score=0.2 required=10.0 tests=HELO_MISC_IP, RP_MATCHES_RCVD autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mattapan.m5p.com X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mailhost.m5p.com [IPv6:2001:470:1f07:15ff::f7]); Mon, 12 Feb 2018 19:49:41 -0500 (EST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 01:05:17 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --UHiw2ZLU1f9rkPRlZ9ZHFIzgb2BrHpPxL Content-Type: multipart/mixed; boundary="zyBKuChoOjBAhjiBBm9TjE3hYCA3mg28k"; protected-headers="v1" From: George Mitchell To: freebsd-stable@freebsd.org Message-ID: <11382853-af10-c629-1e6a-21ab8fc37f53@m5p.com> Subject: Re: Ryzen issues on FreeBSD (can we sum up yet)? References: <8e842dec-ade7-37d1-6bd8-856ea1a827ca@sentex.net> <9b769e4e-b098-b294-0bce-8bb1c42e8a59@rootautomation.com> <730eb882-1c6a-afb7-0ada-396db44fb34b@ingresso.co.uk> <8b882970-4d5d-2a96-4dac-779cab07b9ae@sentex.net> <343acf99-3e9e-093a-7390-c142396c2985@sentex.net> <3dd9a61b-511d-db2e-80ca-cbc9a4b65f92@sentex.net> <55913e41-3a8a-9a4d-6862-e09a3d0f4d55@sentex.net> <5e48bbc2-e872-46bd-eece-25acbb180f77@sentex.net> <5A71C5AE.6020202@grosbein.net> In-Reply-To: --zyBKuChoOjBAhjiBBm9TjE3hYCA3mg28k Content-Type: text/plain; charset=iso-8859-5 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 02/12/18 16:49, Don Lewis wrote: > [...] > I'm having really good luck with the kernel patch attached to this > message: > https://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D417183+0+archive/2018/f= reebsd-hackers/20180211.freebsd-hackers >=20 > Since applying that patch, I did three poudriere runs to build the set > of ~1700 ports that I use. Other than one gmake-related build runaway > that I've also seen on my AMD FX-8320E, I didn't see any random port > build failures. When I was last did some testing a few weeks ago, > lang/go would almost always fail. I also would seem random build > failures in lang/guile or finance/gnucash (which uses guile during its > build) on both my Ryzen and FX-8320E machines, but those built cleanly > all three times. >=20 > I even built samba 16 times in a row without a hang. > [...] Until this thread started last year, I had been on the verge of upgrading the build server on my net to a Ryzen. Needless to say, I postponed the change. Now it seems there is hope that a resolution for the issue may be in sight. Is it time to survey everyone's experience with the problem, and determine whether the cited patch helps? My sincere thanks in advance. -- George --zyBKuChoOjBAhjiBBm9TjE3hYCA3mg28k-- --UHiw2ZLU1f9rkPRlZ9ZHFIzgb2BrHpPxL Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAlqCNhsACgkQwRES3m+p 4flIiw/+MExWZ3tTUrcrbWjqkgJOlBRENjcoxeC0e1QJpsZRilib4DGu07dtsYxk 6H36b3I1V1lIHFE590ZYK6HIu7+4d4VL9yvCbdEFxzR8EhKTpZtGrXw1s61JHVe9 AZ9r/gTLyLruJpu0LfbUx0Bts51DJoWJOqj+izQVmArB6Ni9icLhBX82ezMv/rmd f2zoF8fpPgIBhB7BdYCq2p7AJD7vyRiBfEMtgXmaxX5nKSbRIJ8EG3qYjQBvzOUn 3ERuSAsCjp7gA8S/zX/X3B4VGHB3QEaDrlo8WUxA9Zk+j4+xhxT2GBbij/0iQLxy uyvEisoBBRT7jDGIEK8iJovwttSu0/GAy5vXsKEO8Q1LtpwjtE9rLDczE3GBSbEh Voca4OL0K5y+WtvMx/sCXL59ipXeGBGfYSFCnjYk0R5MDLqLVmn5s1zducwxYxBL P2wiljcAf5f6jbWH39Lod8KzJkLZ1/aIwKuJScAP7G/HT6X0trbczPQAe8+kBmzv VeISEDh37Qkf9BEf0NW2Ac8nXh67h7Pey2JxlNQ5RLcRYyuhIX9WDTFgnZ3r38Xt fFhvkafLP1TjzI0UkBuHr/LEfFaNiAAUcNv2OhEnZ8pxaZIp33bEwJEaAkwsl3gg VlycYTR3aGoNieTcYMr8ItD5c6Ob3aJ8LGfTdXk9uNgUhYVrLUg= =iidl -----END PGP SIGNATURE----- --UHiw2ZLU1f9rkPRlZ9ZHFIzgb2BrHpPxL-- From owner-freebsd-stable@freebsd.org Tue Feb 13 01:49:02 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46CF9F10899 for ; Tue, 13 Feb 2018 01:49:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5EAB57B6E2; Tue, 13 Feb 2018 01:49:00 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w1D1mxPo076676 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 12 Feb 2018 20:48:59 -0500 (EST) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w1D1mtJE024807; Mon, 12 Feb 2018 20:48:55 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: Ryzen issues on FreeBSD ? (with sort of workaround) To: Don Lewis , Eugene Grosbein Cc: freebsd-stable@freebsd.org, Peter Moody , Andriy Gapon , Pete French References: <8e842dec-ade7-37d1-6bd8-856ea1a827ca@sentex.net> <730eb882-1c6a-afb7-0ada-396db44fb34b@ingresso.co.uk> <8b882970-4d5d-2a96-4dac-779cab07b9ae@sentex.net> <343acf99-3e9e-093a-7390-c142396c2985@sentex.net> <3dd9a61b-511d-db2e-80ca-cbc9a4b65f92@sentex.net> <55913e41-3a8a-9a4d-6862-e09a3d0f4d55@sentex.net> <5e48bbc2-e872-46bd-eece-25acbb180f77@sentex.net> <5A71C5AE.6020202@grosbein.net> From: Mike Tancsa Organization: Sentex Communications Message-ID: <73f83421-97ed-88f8-08be-39cf5a70ed20@sentex.net> Date: Mon, 12 Feb 2018 20:48:56 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=iso-8859-5 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 01:49:02 -0000 On 2/12/2018 4:49 PM, Don Lewis wrote: > On 31 Jan, Eugene Grosbein wrote: >> 31.01.2018 4:36, Mike Tancsa : >>> On 1/30/2018 2:51 PM, Mike Tancsa wrote: >>>> >>>> And sadly, I am still able to hang the compile in about the same place. >>>> However, if I set >>> >>> >>> OK, here is a sort of work around. If I have the box a little more busy, >>> I can avoid whatever deadlock is going on. In another console I have >>> cat /dev/urandom | sha256 >>> running while the build runs >>> >>> ... and I can compile net/samba47 from scratch without the compile >>> hanging. This problem also happens on HEAD from today. Should I start >>> a new thread on freebsd-current ? Or just file a bug report ? >>> The compile worked 4/4 >> >> That's really strange. Could you try to do "sysctl kern.eventtimer.periodic=1" >> and re-do the test without extra load? > > I'm having really good luck with the kernel patch attached to this > message: > https://docs.freebsd.org/cgi/getmsg.cgi?fetch=417183+0+archive/2018/freebsd-hackers/20180211.freebsd-hackers > > Since applying that patch, I did three poudriere runs to build the set > of ~1700 ports that I use. Other than one gmake-related build runaway > that I've also seen on my AMD FX-8320E, I didn't see any random port > build failures. When I was last did some testing a few weeks ago, > lang/go would almost always fail. I also would seem random build > failures in lang/guile or finance/gnucash (which uses guile during its > build) on both my Ryzen and FX-8320E machines, but those built cleanly > all three times. > > I even built samba 16 times in a row without a hang. > Cool! I will give it a try! In other news, the latest BIOS patch from ASUS for my motherboard *seems* to have fixed the random hangs. In the BIOS, I had to dial down the memory one notch, disable q-states and disable core boost for my non X Ryzen CPUs. I did that Friday, and running a load that would typically lock up the box, survived the weekend. Same with the box I have in Zoo. No random freeze ups. I was also able to take the amdtemp and amdsmn code from HEAD and compile it on STABLE to confirm the CPU is / was not overheating. Peak temp in the low 50s even with the CPU cores are all maxed out. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada From owner-freebsd-stable@freebsd.org Tue Feb 13 03:54:33 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A105F1BC64 for ; Tue, 13 Feb 2018 03:54:33 +0000 (UTC) (envelope-from peter@hda3.com) Received: from mail-qk0-x22e.google.com (mail-qk0-x22e.google.com [IPv6:2607:f8b0:400d:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7A9F81844 for ; Tue, 13 Feb 2018 03:54:32 +0000 (UTC) (envelope-from peter@hda3.com) Received: by mail-qk0-x22e.google.com with SMTP id b130so3916059qkg.9 for ; Mon, 12 Feb 2018 19:54:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hda3-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=qzU9oHb80r8lIVtSWDJyuPBwzcjs8FPQwSmtyEXjz5E=; b=BEFIRr6a5c192KMT+MVcxVWy9oZZOOYGSfIm7WpokPSfM6LEpaHa/zMjYKOlrC5hnX uzjZs4FSF6eKujDlXsUBuLa1XOJMtXc8v1KLWVMgxprfKRUSGF83TH6dmN52kCX0YpWn Gs2YTN69CkzQUVXmt288Z22qP+NntiASq+/FrapV0crNt8HhuV4qc9N3sLyLeHcFoBHR RMQwfASIRZATnTBCByZIKZl2kZg1Nc4YcBEH12j9JGro7PsGG0YNemq4zzCtXiXdt1tb p+uxpr+haXkaaMwo6oSoge7nJqjyC29dyu8LEGIYZ3FjP1mbfTIDpTh927+9xHSTo4Dg M8gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=qzU9oHb80r8lIVtSWDJyuPBwzcjs8FPQwSmtyEXjz5E=; b=NTsxXS4SmBlbp5hFEf3wUmj5dqqUriNUau67VT5olPgq6SWYFgZN1HoQLTcl+oi/vl Xsp8IkFeYB//meJKENy9YVgfL7b0nkTCG/fMiQsU5uZxLUIdIAlV/f36KSIZxVjmLfrn h78KPEQisc8rhdvlekL0Gm+tAx90KCD8JTQrrsq3DLTEhVQkX4w4zCM+gLXVz34xOD8+ 3wQ+w5JuuIt3oMjLGhuzJtbaCRbFVio32o2zpTMF/Y6YXV1YoaBr6y58vCakOFwOBbL0 WCl6QGGqM+3bw86IzolipnJXu2goqGlfdMzll/Dei9xDHz1SzTTrtDKWsTx6QEctO13J dPOg== X-Gm-Message-State: APf1xPDvZxAENZc+H+shoi4kPdTuADaRm4YZ//d+mZYIkrV5s2Z3IZfA q8i683EZRZBddSeFS8DJslspeYly2Fsl4zEY8+hxoA== X-Google-Smtp-Source: AH8x225wtQED4EQBpZBu9QRprXCvtNX7/eUOIFF2J5bOYTgGFAefjm9EDINmuPum/8vRojPugauCw3F+DDtnyoT03kE= X-Received: by 10.55.44.65 with SMTP id s62mr20685827qkh.203.1518494071985; Mon, 12 Feb 2018 19:54:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.36.228 with HTTP; Mon, 12 Feb 2018 19:54:11 -0800 (PST) In-Reply-To: References: <8e842dec-ade7-37d1-6bd8-856ea1a827ca@sentex.net> <9b769e4e-b098-b294-0bce-8bb1c42e8a59@rootautomation.com> <730eb882-1c6a-afb7-0ada-396db44fb34b@ingresso.co.uk> <8b882970-4d5d-2a96-4dac-779cab07b9ae@sentex.net> <343acf99-3e9e-093a-7390-c142396c2985@sentex.net> <3dd9a61b-511d-db2e-80ca-cbc9a4b65f92@sentex.net> <55913e41-3a8a-9a4d-6862-e09a3d0f4d55@sentex.net> <5e48bbc2-e872-46bd-eece-25acbb180f77@sentex.net> <5A71C5AE.6020202@grosbein.net> From: Peter Moody Date: Mon, 12 Feb 2018 19:54:11 -0800 Message-ID: Subject: Re: Ryzen issues on FreeBSD ? (with sort of workaround) To: Don Lewis Cc: Eugene Grosbein , Pete French , FreeBSD-STABLE Mailing List , Andriy Gapon Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 03:54:33 -0000 On Mon, Feb 12, 2018 at 1:49 PM, Don Lewis wrote: > On 31 Jan, Eugene Grosbein wrote: >> 31.01.2018 4:36, Mike Tancsa =D0=BF=D0=B8=D1=88=D0=B5=D1=82: >>> On 1/30/2018 2:51 PM, Mike Tancsa wrote: >>>> >>>> And sadly, I am still able to hang the compile in about the same place= . >>>> However, if I set >>> >>> >>> OK, here is a sort of work around. If I have the box a little more busy= , >>> I can avoid whatever deadlock is going on. In another console I have >>> cat /dev/urandom | sha256 >>> running while the build runs >>> >>> ... and I can compile net/samba47 from scratch without the compile >>> hanging. This problem also happens on HEAD from today. Should I start >>> a new thread on freebsd-current ? Or just file a bug report ? >>> The compile worked 4/4 >> >> That's really strange. Could you try to do "sysctl kern.eventtimer.perio= dic=3D1" >> and re-do the test without extra load? > > I'm having really good luck with the kernel patch attached to this > message: > https://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D417183+0+archive/2018/fre= ebsd-hackers/20180211.freebsd-hackers I'm new to this; what are the chances that this gets into -STABLE in the near future? From owner-freebsd-stable@freebsd.org Tue Feb 13 13:41:58 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7CA9DF00B81 for ; Tue, 13 Feb 2018 13:41:58 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 18DDD78F9F for ; Tue, 13 Feb 2018 13:41:58 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w1DDfvf6075950 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 13 Feb 2018 08:41:57 -0500 (EST) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w1DDftjB027817; Tue, 13 Feb 2018 08:41:56 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: Ryzen issues on FreeBSD ? (summary of 4 issues) (seemingly solved!) From: Mike Tancsa To: Mark Millard , nimrodl@gmail.com, michaelp@bsquare.com, FreeBSD-STABLE Mailing List References: <744bbe18-80c4-d057-c88d-fbe480ee9abb@sentex.net> Organization: Sentex Communications Message-ID: Date: Tue, 13 Feb 2018 08:41:54 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <744bbe18-80c4-d057-c88d-fbe480ee9abb@sentex.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 13:41:58 -0000 OK, this is all mostly solved for me it seems. points below inline On 1/24/2018 9:42 AM, Mike Tancsa wrote: > I think perhaps a good time to summarize as a few issues seem to be going on > > a) fragile BIOS settings. There seems to be a number of issues around > RAM speeds and disabled C-STATES that impact stability. Specifically, > lowering the default frequency from 2400 to 2133 seems to help some > users with crashes / lockups under heavy loads. Also disabling core boost on non X cpus (ie 1600 vs 1600x) and making sure the CPU is not overheating. On my ASUS board using a back ported version of amdtemp and amdsmn I confirmed the temp does not go above 50C at full load. Setting the FAN speed to turbo seems to help reduce the max temp the CPU would get. > b) CPUs manufactured prior to week 25 (some say week 33?) have a > hardware defect that manifests itself as segfaults in heavy compiles. I > was able to confirm this on 1 of the CPUs I had using a Linux setup. It > seems to confirm this, you need to physically look at the CPU for the > manufacturing date :( Not sure how to trigger it on FreeBSD reliably, > but there is a github project I used to verify on Linux > (https://github.com/suaefar/ryzen-test) AMD sent me 3 new CPUs without issue. Turn around was about 1 week from Canada to the US and back. > > c) The idle lockup bug. This *seems* to be confirmed on Linux as well > http://blog.programster.org/ubuntu-16-04-compile-custom-kernel-for-ryzen > and > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1690085 Perhaps the settings in a), as well as the most recent BIOS update seems to have fixed this issue for me. It sure seemed like a hardware issue, but then again it could be a side effect of d). However, I was never able to break into the debugger using a debugging kernel in HEAD so I suspect it was more hardware related than anything. BIOS Information Vendor: American Megatrends Inc. Version: 3803 Release Date: 01/22/2018 Address: 0xF0000 This is on a Product Name: PRIME X370-PRO Version: Rev X.0x > > d) Compile failures of some ports. For myself and one other user, > compiling net/samba47 reliably hangs in roughly the same place. Its not > clear if this is related to any of the above bugs or not. This too seems to be fixed! The patch in https://docs.freebsd.org/cgi/getmsg.cgi?fetch=417183+0+archive/2018/freebsd-hackers/20180211.freebsd-hackers seems to stop the deadlock. I did 90 builds on RELENG_11 with this patch over night and no deadlocks. For half the builds I had 2 guest VMs also building. For the second half, it was the only thing running on the box and its working as expected All this just in time for my Epyc based system to arrive! ---Mike From owner-freebsd-stable@freebsd.org Tue Feb 13 16:07:48 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE622F0CAD6 for ; Tue, 13 Feb 2018 16:07:48 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from smtp.vangyzen.net (hotblack.vangyzen.net [199.48.133.146]) by mx1.freebsd.org (Postfix) with ESMTP id 66534803C0; Tue, 13 Feb 2018 16:07:47 +0000 (UTC) (envelope-from eric@vangyzen.net) Received: from sweettea.beer.town (unknown [76.164.8.130]) by smtp.vangyzen.net (Postfix) with ESMTPSA id B74265646B; Tue, 13 Feb 2018 10:07:39 -0600 (CST) Subject: Re: Ryzen issues on FreeBSD ? (with sort of workaround) To: Peter Moody , Don Lewis Cc: FreeBSD-STABLE Mailing List , Andriy Gapon , Pete French References: <8e842dec-ade7-37d1-6bd8-856ea1a827ca@sentex.net> <730eb882-1c6a-afb7-0ada-396db44fb34b@ingresso.co.uk> <8b882970-4d5d-2a96-4dac-779cab07b9ae@sentex.net> <343acf99-3e9e-093a-7390-c142396c2985@sentex.net> <3dd9a61b-511d-db2e-80ca-cbc9a4b65f92@sentex.net> <55913e41-3a8a-9a4d-6862-e09a3d0f4d55@sentex.net> <5e48bbc2-e872-46bd-eece-25acbb180f77@sentex.net> <5A71C5AE.6020202@grosbein.net> From: Eric van Gyzen Message-ID: Date: Tue, 13 Feb 2018 10:07:39 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 16:07:48 -0000 On 02/12/2018 21:54, Peter Moody wrote: >> I'm having really good luck with the kernel patch attached to this >> message: >> https://docs.freebsd.org/cgi/getmsg.cgi?fetch=417183+0+archive/2018/freebsd-hackers/20180211.freebsd-hackers > > I'm new to this; what are the chances that this gets into -STABLE in > the near future? Pretty high, I imagine. Add yourself to these to stay informed: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225584 https://reviews.freebsd.org/D14347 Eric From owner-freebsd-stable@freebsd.org Tue Feb 13 21:46:59 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 805C9F03AB3 for ; Tue, 13 Feb 2018 21:46:59 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D4937388A; Tue, 13 Feb 2018 21:46:59 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w1DLkwoN066169 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 13 Feb 2018 16:46:58 -0500 (EST) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w1DLkukF031312; Tue, 13 Feb 2018 16:46:56 -0500 (EST) (envelope-from mike@sentex.net) To: FreeBSD-STABLE Mailing List From: Mike Tancsa Subject: FreeBSD on AMD Epyc boards Organization: Sentex Communications Message-ID: <8037bf98-acc8-6981-d25b-3b58330dbd33@sentex.net> Date: Tue, 13 Feb 2018 16:46:58 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="------------181331DCA557B80647FF53F4" Content-Language: en-US X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 21:46:59 -0000 This is a multi-part message in MIME format. --------------181331DCA557B80647FF53F4 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit To have a bit of a work around for the Intel Meltdown bug (yes, no Spectre), I wanted to try out some AMD based CPUs. So far so good using a SuperMicro H11SSL-i. A decent server board using an Epyc CPU. All the things you need and expect for a server grade MB ipmi to provide remote management (SoL to BIOS and OS) and hardware info. The ipmi.ko driver works great with RELENG11. This allows for hardware watchdog support too. amdtemp from CURRENT also works to provide cpu temp info, although ipmitool does this as well. Seems to sit at about 50W when idle and tops out at about 180W with 2 SSD drives and all cores busy. Note, the bug in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225584 and discussed in https://reviews.freebsd.org/D14347 manifests itself fairly readily. However, the patch fixes the problem Attached is some dmesg info for the curious. Seems like a decent board for FreeBSD and we are going to start deploying in a couple of spots once we do some more burn in and testing. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada --------------181331DCA557B80647FF53F4 Content-Type: text/plain; charset=UTF-8; name="supermicro.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="supermicro.txt" Q29weXJpZ2h0IChjKSAxOTkyLTIwMTggVGhlIEZyZWVCU0QgUHJvamVjdC4NCkNvcHlyaWdo dCAoYykgMTk3OSwgMTk4MCwgMTk4MywgMTk4NiwgMTk4OCwgMTk4OSwgMTk5MSwgMTk5Miwg MTk5MywgMTk5NA0KICAgICAgICBUaGUgUmVnZW50cyBvZiB0aGUgVW5pdmVyc2l0eSBvZiBD YWxpZm9ybmlhLiBBbGwgcmlnaHRzIHJlc2VydmVkLg0KRnJlZUJTRCBpcyBhIHJlZ2lzdGVy ZWQgdHJhZGVtYXJrIG9mIFRoZSBGcmVlQlNEIEZvdW5kYXRpb24uDQpGcmVlQlNEIDExLjEt U1RBQkxFICMwIHIzMjkyNDFNOiBUdWUgRmViIDEzIDE2OjIwOjU4IEVTVCAyMDE4DQogICAg bWR0YW5jc2FAZXB5Yy1ic2Quc2VudGV4LmNhOi91c3Ivb2JqL3Vzci9zcmMvc3lzL3NlcnZl ciBhbWQ2NA0KRnJlZUJTRCBjbGFuZyB2ZXJzaW9uIDUuMC4xICh0YWdzL1JFTEVBU0VfNTAx L2ZpbmFsIDMyMDg4MCkgKGJhc2VkIG9uIExMVk0gNS4wLjEpDQpTUkFUOiBObyBtZW1vcnkg Zm91bmQgZm9yIENQVSAwDQpWVCh2Z2EpOiByZXNvbHV0aW9uIDY0MHg0ODANCkNQVTogQU1E IEVQWUMgNzI4MSAxNi1Db3JlIFByb2Nlc3NvciAgICAgICAgICAgICAgICAgKDIxMDAuMDUt TUh6IEs4LWNsYXNzIENQVSkNCiAgT3JpZ2luPSJBdXRoZW50aWNBTUQiICBJZD0weDgwMGYx MiAgRmFtaWx5PTB4MTcgIE1vZGVsPTB4MSAgU3RlcHBpbmc9Mg0KICBGZWF0dXJlcz0weDE3 OGJmYmZmPEZQVSxWTUUsREUsUFNFLFRTQyxNU1IsUEFFLE1DRSxDWDgsQVBJQyxTRVAsTVRS UixQR0UsTUNBLENNT1YsUEFULFBTRTM2LENMRkxVU0gsTU1YLEZYU1IsU1NFLFNTRTIsSFRU Pg0KICBGZWF0dXJlczI9MHg3ZWQ4MzIwYjxTU0UzLFBDTE1VTFFEUSxNT04sU1NTRTMsRk1B LENYMTYsU1NFNC4xLFNTRTQuMixNT1ZCRSxQT1BDTlQsQUVTTkksWFNBVkUsT1NYU0FWRSxB VlgsRjE2QyxSRFJBTkQ+DQogIEFNRCBGZWF0dXJlcz0weDJlNTAwODAwPFNZU0NBTEwsTlgs TU1YKyxGRlhTUixQYWdlMUdCLFJEVFNDUCxMTT4NCiAgQU1EIEZlYXR1cmVzMj0weDM1YzIz M2ZmPExBSEYsQ01QLFNWTSxFeHRBUElDLENSOCxBQk0sU1NFNEEsTUFTLFByZWZldGNoLE9T VlcsU0tJTklULFdEVCxUQ0UsVG9wb2xvZ3ksUENYQyxQTlhDLERCRSxQTDJJLE1XQUlUWD4N CiAgU3RydWN0dXJlZCBFeHRlbmRlZCBGZWF0dXJlcz0weDIwOWMwMWE5PEZTR1NCQVNFLEJN STEsQVZYMixTTUVQLEJNSTIsUkRTRUVELEFEWCxTTUFQLENMRkxVU0hPUFQsU0hBPg0KICBY U0FWRSBGZWF0dXJlcz0weGY8WFNBVkVPUFQsWFNBVkVDLFhJTlVTRSxYU0FWRVM+DQogIEFN RCBFeHRlbmRlZCBGZWF0dXJlIEV4dGVuc2lvbnMgSUQgRUJYPTB4NzxDTFpFUk8sSVJQZXJm LFhTYXZlRXJQdHI+DQogIFNWTTogTlAsTlJJUCxWQ2xlYW4sQUZsdXNoLERBc3Npc3QsTkFz aWRzPTMyNzY4DQogIFRTQzogUC1zdGF0ZSBpbnZhcmlhbnQsIHBlcmZvcm1hbmNlIHN0YXRp c3RpY3MNCnJlYWwgbWVtb3J5ICA9IDM0MzU5NzM4MzY4ICgzMjc2OCBNQikNCmF2YWlsIG1l bW9yeSA9IDMzMjE1NDk2MTkyICgzMTY3NiBNQikNCkV2ZW50IHRpbWVyICJMQVBJQyIgcXVh bGl0eSAxMDANCkFDUEkgQVBJQyBUYWJsZTogPCA+DQpGcmVlQlNEL1NNUDogTXVsdGlwcm9j ZXNzb3IgU3lzdGVtIERldGVjdGVkOiAzMiBDUFVzDQpGcmVlQlNEL1NNUDogMSBwYWNrYWdl KHMpIHggMTYgY29yZShzKSB4IDIgaGFyZHdhcmUgdGhyZWFkcw0KcmFuZG9tOiB1bmJsb2Nr aW5nIGRldmljZS4NCmlvYXBpYzA6IENoYW5naW5nIEFQSUMgSUQgdG8gMTI4DQppb2FwaWMx OiBDaGFuZ2luZyBBUElDIElEIHRvIDEyOQ0KaW9hcGljMjogQ2hhbmdpbmcgQVBJQyBJRCB0 byAxMzANCmlvYXBpYzM6IENoYW5naW5nIEFQSUMgSUQgdG8gMTMxDQppb2FwaWM0OiBDaGFu Z2luZyBBUElDIElEIHRvIDEzMg0KaW9hcGljMCA8VmVyc2lvbiAyLjE+IGlycXMgMC0yMyBv biBtb3RoZXJib2FyZA0KaW9hcGljMSA8VmVyc2lvbiAyLjE+IGlycXMgMjQtNTUgb24gbW90 aGVyYm9hcmQNCmlvYXBpYzIgPFZlcnNpb24gMi4xPiBpcnFzIDU2LTg3IG9uIG1vdGhlcmJv YXJkDQppb2FwaWMzIDxWZXJzaW9uIDIuMT4gaXJxcyA4OC0xMTkgb24gbW90aGVyYm9hcmQN CmlvYXBpYzQgPFZlcnNpb24gMi4xPiBpcnFzIDEyMC0xNTEgb24gbW90aGVyYm9hcmQNClNN UDogQVAgQ1BVICMyNyBMYXVuY2hlZCENClNNUDogQVAgQ1BVICMxOSBMYXVuY2hlZCENClNN UDogQVAgQ1BVICMxMSBMYXVuY2hlZCENClNNUDogQVAgQ1BVICM5IExhdW5jaGVkIQ0KU01Q OiBBUCBDUFUgIzEzIExhdW5jaGVkIQ0KU01QOiBBUCBDUFUgIzI5IExhdW5jaGVkIQ0KU01Q OiBBUCBDUFUgIzcgTGF1bmNoZWQhDQpTTVA6IEFQIENQVSAjNSBMYXVuY2hlZCENClNNUDog QVAgQ1BVICM0IExhdW5jaGVkIQ0KU01QOiBBUCBDUFUgIzI2IExhdW5jaGVkIQ0KU01QOiBB UCBDUFUgIzI4IExhdW5jaGVkIQ0KU01QOiBBUCBDUFUgIzMxIExhdW5jaGVkIQ0KU01QOiBB UCBDUFUgIzggTGF1bmNoZWQhDQpTTVA6IEFQIENQVSAjMjMgTGF1bmNoZWQhDQpTTVA6IEFQ IENQVSAjMjEgTGF1bmNoZWQhDQpTTVA6IEFQIENQVSAjMjIgTGF1bmNoZWQhDQpTTVA6IEFQ IENQVSAjMjAgTGF1bmNoZWQhDQpTTVA6IEFQIENQVSAjMTcgTGF1bmNoZWQhDQpTTVA6IEFQ IENQVSAjMTggTGF1bmNoZWQhDQpTTVA6IEFQIENQVSAjMTYgTGF1bmNoZWQhDQpTTVA6IEFQ IENQVSAjNiBMYXVuY2hlZCENClNNUDogQVAgQ1BVICMzMCBMYXVuY2hlZCENClNNUDogQVAg Q1BVICMxMCBMYXVuY2hlZCENClNNUDogQVAgQ1BVICMxMiBMYXVuY2hlZCENClNNUDogQVAg Q1BVICMyNSBMYXVuY2hlZCENClNNUDogQVAgQ1BVICMyNCBMYXVuY2hlZCENClNNUDogQVAg Q1BVICMzIExhdW5jaGVkIQ0KU01QOiBBUCBDUFUgIzEgTGF1bmNoZWQhDQpTTVA6IEFQIENQ VSAjMiBMYXVuY2hlZCENClNNUDogQVAgQ1BVICMxNSBMYXVuY2hlZCENClNNUDogQVAgQ1BV ICMxNCBMYXVuY2hlZCENClRpbWVjb3VudGVyICJUU0MiIGZyZXF1ZW5jeSAyMTAwMDQ5OTE3 IEh6IHF1YWxpdHkgMTAwMA0KcmFuZG9tOiBlbnRyb3B5IGRldmljZSBleHRlcm5hbCBpbnRl cmZhY2UNCm5ldG1hcDogbG9hZGVkIG1vZHVsZQ0KbW9kdWxlX3JlZ2lzdGVyX2luaXQ6IE1P RF9MT0FEICh2ZXNhLCAweGZmZmZmZmZmODBjZjEzZjAsIDApIGVycm9yIDE5DQpyYW5kb206 IHJlZ2lzdGVyaW5nIGZhc3Qgc291cmNlIEludGVsIFNlY3VyZSBLZXkgUk5HDQpyYW5kb206 IGZhc3QgcHJvdmlkZXI6ICJJbnRlbCBTZWN1cmUgS2V5IFJORyINCmtiZDEgYXQga2JkbXV4 MA0KbmV4dXMwDQp2dHZnYTA6IDxWVCBWR0EgZHJpdmVyPiBvbiBtb3RoZXJib2FyZA0KY3J5 cHRvc29mdDA6IDxzb2Z0d2FyZSBjcnlwdG8+IG9uIG1vdGhlcmJvYXJkDQphZXNuaTA6IDxB RVMtQ0JDLEFFUy1YVFMsQUVTLUdDTSxBRVMtSUNNPiBvbiBtb3RoZXJib2FyZA0KYWNwaTA6 IDxTVVBFUk0gU1VQRVJNPiBvbiBtb3RoZXJib2FyZA0KYWNwaTA6IFBvd2VyIEJ1dHRvbiAo Zml4ZWQpDQpjcHUwOiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHUxOiA8QUNQSSBDUFU+IG9u IGFjcGkwDQpjcHUyOiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHUzOiA8QUNQSSBDUFU+IG9u IGFjcGkwDQpjcHU0OiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHU1OiA8QUNQSSBDUFU+IG9u IGFjcGkwDQpjcHU2OiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHU3OiA8QUNQSSBDUFU+IG9u IGFjcGkwDQpjcHU4OiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHU5OiA8QUNQSSBDUFU+IG9u IGFjcGkwDQpjcHUxMDogPEFDUEkgQ1BVPiBvbiBhY3BpMA0KY3B1MTE6IDxBQ1BJIENQVT4g b24gYWNwaTANCmNwdTEyOiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHUxMzogPEFDUEkgQ1BV PiBvbiBhY3BpMA0KY3B1MTQ6IDxBQ1BJIENQVT4gb24gYWNwaTANCmNwdTE1OiA8QUNQSSBD UFU+IG9uIGFjcGkwDQpjcHUxNjogPEFDUEkgQ1BVPiBvbiBhY3BpMA0KY3B1MTc6IDxBQ1BJ IENQVT4gb24gYWNwaTANCmNwdTE4OiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHUxOTogPEFD UEkgQ1BVPiBvbiBhY3BpMA0KY3B1MjA6IDxBQ1BJIENQVT4gb24gYWNwaTANCmNwdTIxOiA8 QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHUyMjogPEFDUEkgQ1BVPiBvbiBhY3BpMA0KY3B1MjM6 IDxBQ1BJIENQVT4gb24gYWNwaTANCmNwdTI0OiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHUy NTogPEFDUEkgQ1BVPiBvbiBhY3BpMA0KY3B1MjY6IDxBQ1BJIENQVT4gb24gYWNwaTANCmNw dTI3OiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpjcHUyODogPEFDUEkgQ1BVPiBvbiBhY3BpMA0K Y3B1Mjk6IDxBQ1BJIENQVT4gb24gYWNwaTANCmNwdTMwOiA8QUNQSSBDUFU+IG9uIGFjcGkw DQpjcHUzMTogPEFDUEkgQ1BVPiBvbiBhY3BpMA0KYXR0aW1lcjA6IDxBVCB0aW1lcj4gcG9y dCAweDQwLTB4NDMgaXJxIDAgb24gYWNwaTANClRpbWVjb3VudGVyICJpODI1NCIgZnJlcXVl bmN5IDExOTMxODIgSHogcXVhbGl0eSAwDQpFdmVudCB0aW1lciAiaTgyNTQiIGZyZXF1ZW5j eSAxMTkzMTgyIEh6IHF1YWxpdHkgMTAwDQphdHJ0YzA6IDxBVCByZWFsdGltZSBjbG9jaz4g cG9ydCAweDcwLTB4NzEgb24gYWNwaTANCmF0cnRjMDogcmVnaXN0ZXJlZCBhcyBhIHRpbWUt b2YtZGF5IGNsb2NrLCByZXNvbHV0aW9uIDEuMDAwMDAwcw0KRXZlbnQgdGltZXIgIlJUQyIg ZnJlcXVlbmN5IDMyNzY4IEh6IHF1YWxpdHkgMA0KVGltZWNvdW50ZXIgIkFDUEktZmFzdCIg ZnJlcXVlbmN5IDM1Nzk1NDUgSHogcXVhbGl0eSA5MDANCmFjcGlfdGltZXIwOiA8MzItYml0 IHRpbWVyIGF0IDMuNTc5NTQ1TUh6PiBwb3J0IDB4ODA4LTB4ODBiIG9uIGFjcGkwDQpocGV0 MDogPEhpZ2ggUHJlY2lzaW9uIEV2ZW50IFRpbWVyPiBpb21lbSAweGZlZDAwMDAwLTB4ZmVk MDAzZmYgb24gYWNwaTANClRpbWVjb3VudGVyICJIUEVUIiBmcmVxdWVuY3kgMTQzMTgxODAg SHogcXVhbGl0eSA5NTANCkV2ZW50IHRpbWVyICJIUEVUIiBmcmVxdWVuY3kgMTQzMTgxODAg SHogcXVhbGl0eSAzNTANCkV2ZW50IHRpbWVyICJIUEVUMSIgZnJlcXVlbmN5IDE0MzE4MTgw IEh6IHF1YWxpdHkgMzUwDQpFdmVudCB0aW1lciAiSFBFVDIiIGZyZXF1ZW5jeSAxNDMxODE4 MCBIeiBxdWFsaXR5IDM1MA0KcGNpYjA6IDxBQ1BJIEhvc3QtUENJIGJyaWRnZT4gcG9ydCAw eGNmOC0weGNmZiBvbiBhY3BpMA0KcGNpYjA6IF9PU0MgcmV0dXJuZWQgZXJyb3IgMHgxMA0K cGNpMDogPEFDUEkgUENJIGJ1cz4gb24gcGNpYjANCnBjaWIxOiA8QUNQSSBQQ0ktUENJIGJy aWRnZT4gYXQgZGV2aWNlIDEuMSBvbiBwY2kwDQpwY2kxOiA8QUNQSSBQQ0kgYnVzPiBvbiBw Y2liMQ0KcGNpYjI6IDxQQ0ktUENJIGJyaWRnZT4gaXJxIDI0IGF0IGRldmljZSAwLjAgb24g cGNpMQ0KcGNpMjogPFBDSSBidXM+IG9uIHBjaWIyDQp2Z2FwY2kwOiA8VkdBLWNvbXBhdGli bGUgZGlzcGxheT4gcG9ydCAweDQwMDAtMHg0MDdmIG1lbSAweGVlMDAwMDAwLTB4ZWVmZmZm ZmYsMHhlZjAwMDAwMC0weGVmMDFmZmZmIGlycSAyNCBhdCBkZXZpY2UgMC4wIG9uIHBjaTIN CnZnYXBjaTA6IEJvb3QgdmlkZW8gZGV2aWNlDQpwY2liMzogPEFDUEkgUENJLVBDSSBicmlk Z2U+IGF0IGRldmljZSAxLjIgb24gcGNpMA0KcGNpMzogPEFDUEkgUENJIGJ1cz4gb24gcGNp YjMNCnhoY2kwOiA8QVNNZWRpYSBBU00xMDQyQSBVU0IgMy4wIGNvbnRyb2xsZXI+IG1lbSAw eGVmOTAwMDAwLTB4ZWY5MDdmZmYgYXQgZGV2aWNlIDAuMCBvbiBwY2kzDQp4aGNpMDogMzIg Ynl0ZXMgY29udGV4dCBzaXplLCA2NC1iaXQgRE1BDQp4aGNpMDogVW5hYmxlIHRvIG1hcCBN U0ktWCB0YWJsZSANCnVzYnVzMCBvbiB4aGNpMA0KdXNidXMwOiA1LjBHYnBzIFN1cGVyIFNw ZWVkIFVTQiB2My4wDQpwY2liNDogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGF0IGRldmljZSAx LjQgb24gcGNpMA0KcGNpNDogPEFDUEkgUENJIGJ1cz4gb24gcGNpYjQNCmlnYjA6IDxJbnRl bChSKSBQUk8vMTAwMCBOZXR3b3JrIENvbm5lY3Rpb24sIFZlcnNpb24gLSAyLjUuMy1rPiBw b3J0IDB4MzAwMC0weDMwMWYgbWVtIDB4ZWY4MDAwMDAtMHhlZjg3ZmZmZiwweGVmODgwMDAw LTB4ZWY4ODNmZmYgaXJxIDM2IGF0IGRldmljZSAwLjAgb24gcGNpNA0KaWdiMDogVXNpbmcg TVNJWCBpbnRlcnJ1cHRzIHdpdGggNSB2ZWN0b3JzDQppZ2IwOiBFdGhlcm5ldCBhZGRyZXNz OiAwYzpjNDo3YTplNjphYjpmYQ0KaWdiMDogQm91bmQgcXVldWUgMCB0byBjcHUgMA0KaWdi MDogQm91bmQgcXVldWUgMSB0byBjcHUgMQ0KaWdiMDogQm91bmQgcXVldWUgMiB0byBjcHUg Mg0KaWdiMDogQm91bmQgcXVldWUgMyB0byBjcHUgMw0KaWdiMDogbmV0bWFwIHF1ZXVlcy9z bG90czogVFggNC8xMDI0LCBSWCA0LzEwMjQNCnBjaWI1OiA8QUNQSSBQQ0ktUENJIGJyaWRn ZT4gYXQgZGV2aWNlIDEuNSBvbiBwY2kwDQpwY2k1OiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2li NQ0KaWdiMTogPEludGVsKFIpIFBSTy8xMDAwIE5ldHdvcmsgQ29ubmVjdGlvbiwgVmVyc2lv biAtIDIuNS4zLWs+IHBvcnQgMHgyMDAwLTB4MjAxZiBtZW0gMHhlZjcwMDAwMC0weGVmNzdm ZmZmLDB4ZWY3ODAwMDAtMHhlZjc4M2ZmZiBpcnEgNDAgYXQgZGV2aWNlIDAuMCBvbiBwY2k1 DQppZ2IxOiBVc2luZyBNU0lYIGludGVycnVwdHMgd2l0aCA1IHZlY3RvcnMNCmlnYjE6IEV0 aGVybmV0IGFkZHJlc3M6IDBjOmM0OjdhOmU2OmFiOmZiDQppZ2IxOiBCb3VuZCBxdWV1ZSAw IHRvIGNwdSA0DQppZ2IxOiBCb3VuZCBxdWV1ZSAxIHRvIGNwdSA1DQppZ2IxOiBCb3VuZCBx dWV1ZSAyIHRvIGNwdSA2DQppZ2IxOiBCb3VuZCBxdWV1ZSAzIHRvIGNwdSA3DQppZ2IxOiBu ZXRtYXAgcXVldWVzL3Nsb3RzOiBUWCA0LzEwMjQsIFJYIDQvMTAyNA0KcGNpYjY6IDxBQ1BJ IFBDSS1QQ0kgYnJpZGdlPiBhdCBkZXZpY2UgNy4xIG9uIHBjaTANCnBjaTY6IDxBQ1BJIFBD SSBidXM+IG9uIHBjaWI2DQpwY2k2OiA8dW5rbm93bj4gYXQgZGV2aWNlIDAuMCAobm8gZHJp dmVyIGF0dGFjaGVkKQ0KcGNpNjogPGVuY3J5cHQvZGVjcnlwdD4gYXQgZGV2aWNlIDAuMiAo bm8gZHJpdmVyIGF0dGFjaGVkKQ0KeGhjaTE6IDxYSENJIChnZW5lcmljKSBVU0IgMy4wIGNv bnRyb2xsZXI+IG1lbSAweGVmMjAwMDAwLTB4ZWYyZmZmZmYgaXJxIDM3IGF0IGRldmljZSAw LjMgb24gcGNpNg0KeGhjaTE6IDY0IGJ5dGVzIGNvbnRleHQgc2l6ZSwgNjQtYml0IERNQQ0K eGhjaTE6IFVuYWJsZSB0byBtYXAgTVNJLVggdGFibGUgDQp1c2J1czEgb24geGhjaTENCnVz YnVzMTogNS4wR2JwcyBTdXBlciBTcGVlZCBVU0IgdjMuMA0KcGNpYjc6IDxBQ1BJIFBDSS1Q Q0kgYnJpZGdlPiBhdCBkZXZpY2UgOC4xIG9uIHBjaTANCnBjaTc6IDxBQ1BJIFBDSSBidXM+ IG9uIHBjaWI3DQpwY2k3OiA8dW5rbm93bj4gYXQgZGV2aWNlIDAuMCAobm8gZHJpdmVyIGF0 dGFjaGVkKQ0KcGNpNzogPGVuY3J5cHQvZGVjcnlwdD4gYXQgZGV2aWNlIDAuMSAobm8gZHJp dmVyIGF0dGFjaGVkKQ0KYWhjaTA6IDxBTUQgS0VSTkNaIEFIQ0kgU0FUQSBjb250cm9sbGVy PiBtZW0gMHhlZjYwMjAwMC0weGVmNjAyZmZmIGlycSA0MiBhdCBkZXZpY2UgMC4yIG9uIHBj aTcNCmFoY2kwOiBBSENJIHYxLjMxIHdpdGggMSA2R2JwcyBwb3J0cywgUG9ydCBNdWx0aXBs aWVyIHN1cHBvcnRlZCB3aXRoIEZCUw0KYWhjaWNoMDogPEFIQ0kgY2hhbm5lbD4gYXQgY2hh bm5lbCAwIG9uIGFoY2kwDQppc2FiMDogPFBDSS1JU0EgYnJpZGdlPiBhdCBkZXZpY2UgMjAu MyBvbiBwY2kwDQppc2EwOiA8SVNBIGJ1cz4gb24gaXNhYjANCnBjaWI4OiA8QUNQSSBIb3N0 LVBDSSBicmlkZ2U+IG9uIGFjcGkwDQpwY2liODogX09TQyByZXR1cm5lZCBlcnJvciAweDEw DQpwY2k4OiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liOA0KcGNpYjk6IDxBQ1BJIFBDSS1QQ0kg YnJpZGdlPiBhdCBkZXZpY2UgMy4xIG9uIHBjaTgNCnBjaTk6IDxBQ1BJIFBDSSBidXM+IG9u IHBjaWI5DQp4aGNpMjogPEFTTWVkaWEgQVNNMTA0MkEgVVNCIDMuMCBjb250cm9sbGVyPiBt ZW0gMHhlYmYwMDAwMC0weGViZjA3ZmZmIGF0IGRldmljZSAwLjAgb24gcGNpOQ0KeGhjaTI6 IDMyIGJ5dGVzIGNvbnRleHQgc2l6ZSwgNjQtYml0IERNQQ0KeGhjaTI6IFVuYWJsZSB0byBt YXAgTVNJLVggdGFibGUgDQp1c2J1czIgb24geGhjaTINCnVzYnVzMjogNS4wR2JwcyBTdXBl ciBTcGVlZCBVU0IgdjMuMA0KcGNpYjEwOiA8QUNQSSBQQ0ktUENJIGJyaWRnZT4gYXQgZGV2 aWNlIDMuMiBvbiBwY2k4DQpwY2kxMDogPEFDUEkgUENJIGJ1cz4gb24gcGNpYjEwDQp4aGNp MzogPEFTTWVkaWEgQVNNMTA0MkEgVVNCIDMuMCBjb250cm9sbGVyPiBtZW0gMHhlYmUwMDAw MC0weGViZTA3ZmZmIGF0IGRldmljZSAwLjAgb24gcGNpMTANCnhoY2kzOiAzMiBieXRlcyBj b250ZXh0IHNpemUsIDY0LWJpdCBETUENCnhoY2kzOiBVbmFibGUgdG8gbWFwIE1TSS1YIHRh YmxlIA0KdXNidXMzIG9uIHhoY2kzDQp1c2J1czM6IDUuMEdicHMgU3VwZXIgU3BlZWQgVVNC IHYzLjANCnBjaWIxMTogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGF0IGRldmljZSA3LjEgb24g cGNpOA0KcGNpMTE6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWIxMQ0KcGNpMTE6IDx1bmtub3du PiBhdCBkZXZpY2UgMC4wIChubyBkcml2ZXIgYXR0YWNoZWQpDQpwY2kxMTogPGVuY3J5cHQv ZGVjcnlwdD4gYXQgZGV2aWNlIDAuMiAobm8gZHJpdmVyIGF0dGFjaGVkKQ0KeGhjaTQ6IDxY SENJIChnZW5lcmljKSBVU0IgMy4wIGNvbnRyb2xsZXI+IG1lbSAweGViOTAwMDAwLTB4ZWI5 ZmZmZmYgaXJxIDY5IGF0IGRldmljZSAwLjMgb24gcGNpMTENCnhoY2k0OiA2NCBieXRlcyBj b250ZXh0IHNpemUsIDY0LWJpdCBETUENCnhoY2k0OiBVbmFibGUgdG8gbWFwIE1TSS1YIHRh YmxlIA0KdXNidXM0IG9uIHhoY2k0DQp1c2J1czQ6IDUuMEdicHMgU3VwZXIgU3BlZWQgVVNC IHYzLjANCnBjaWIxMjogPEFDUEkgUENJLVBDSSBicmlkZ2U+IGF0IGRldmljZSA4LjEgb24g cGNpOA0KcGNpMTI6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWIxMg0KcGNpMTI6IDx1bmtub3du PiBhdCBkZXZpY2UgMC4wIChubyBkcml2ZXIgYXR0YWNoZWQpDQpwY2kxMjogPGVuY3J5cHQv ZGVjcnlwdD4gYXQgZGV2aWNlIDAuMSAobm8gZHJpdmVyIGF0dGFjaGVkKQ0KcGNpYjEzOiA8 QUNQSSBIb3N0LVBDSSBicmlkZ2U+IG9uIGFjcGkwDQpwY2liMTM6IF9PU0MgcmV0dXJuZWQg ZXJyb3IgMHgxMA0KcGNpMTM6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWIxMw0KcGNpYjE0OiA8 QUNQSSBQQ0ktUENJIGJyaWRnZT4gYXQgZGV2aWNlIDcuMSBvbiBwY2kxMw0KcGNpMTQ6IDxB Q1BJIFBDSSBidXM+IG9uIHBjaWIxNA0KcGNpMTQ6IDx1bmtub3duPiBhdCBkZXZpY2UgMC4w IChubyBkcml2ZXIgYXR0YWNoZWQpDQpwY2kxNDogPGVuY3J5cHQvZGVjcnlwdD4gYXQgZGV2 aWNlIDAuMiAobm8gZHJpdmVyIGF0dGFjaGVkKQ0KcGNpYjE1OiA8QUNQSSBQQ0ktUENJIGJy aWRnZT4gYXQgZGV2aWNlIDguMSBvbiBwY2kxMw0KcGNpMTU6IDxBQ1BJIFBDSSBidXM+IG9u IHBjaWIxNQ0KcGNpMTU6IDx1bmtub3duPiBhdCBkZXZpY2UgMC4wIChubyBkcml2ZXIgYXR0 YWNoZWQpDQpwY2kxNTogPGVuY3J5cHQvZGVjcnlwdD4gYXQgZGV2aWNlIDAuMSAobm8gZHJp dmVyIGF0dGFjaGVkKQ0KYWhjaTE6IDxBTUQgS0VSTkNaIEFIQ0kgU0FUQSBjb250cm9sbGVy PiBtZW0gMHhlOTYwMjAwMC0weGU5NjAyZmZmIGlycSAxMDYgYXQgZGV2aWNlIDAuMiBvbiBw Y2kxNQ0KYWhjaTE6IEFIQ0kgdjEuMzEgd2l0aCA4IDZHYnBzIHBvcnRzLCBQb3J0IE11bHRp cGxpZXIgc3VwcG9ydGVkIHdpdGggRkJTDQphaGNpY2gxOiA8QUhDSSBjaGFubmVsPiBhdCBj aGFubmVsIDAgb24gYWhjaTENCmFoY2ljaDI6IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwg MSBvbiBhaGNpMQ0KYWhjaWNoMzogPEFIQ0kgY2hhbm5lbD4gYXQgY2hhbm5lbCAyIG9uIGFo Y2kxDQphaGNpY2g0OiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDMgb24gYWhjaTENCmFo Y2ljaDU6IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgNCBvbiBhaGNpMQ0KYWhjaWNoNjog PEFIQ0kgY2hhbm5lbD4gYXQgY2hhbm5lbCA1IG9uIGFoY2kxDQphaGNpY2g3OiA8QUhDSSBj aGFubmVsPiBhdCBjaGFubmVsIDYgb24gYWhjaTENCmFoY2ljaDg6IDxBSENJIGNoYW5uZWw+ IGF0IGNoYW5uZWwgNyBvbiBhaGNpMQ0KYWhjaWVtMDogPEFIQ0kgZW5jbG9zdXJlIG1hbmFn ZW1lbnQgYnJpZGdlPiBvbiBhaGNpMQ0KcGNpYjE2OiA8QUNQSSBIb3N0LVBDSSBicmlkZ2U+ IG9uIGFjcGkwDQpwY2liMTY6IF9PU0MgcmV0dXJuZWQgZXJyb3IgMHgxMA0KcGNpMTY6IDxB Q1BJIFBDSSBidXM+IG9uIHBjaWIxNg0KcGNpYjE3OiA8QUNQSSBQQ0ktUENJIGJyaWRnZT4g YXQgZGV2aWNlIDcuMSBvbiBwY2kxNg0KcGNpMTc6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWIx Nw0KcGNpMTc6IDx1bmtub3duPiBhdCBkZXZpY2UgMC4wIChubyBkcml2ZXIgYXR0YWNoZWQp DQpwY2kxNzogPGVuY3J5cHQvZGVjcnlwdD4gYXQgZGV2aWNlIDAuMiAobm8gZHJpdmVyIGF0 dGFjaGVkKQ0KcGNpYjE4OiA8QUNQSSBQQ0ktUENJIGJyaWRnZT4gYXQgZGV2aWNlIDguMSBv biBwY2kxNg0KcGNpMTg6IDxBQ1BJIFBDSSBidXM+IG9uIHBjaWIxOA0KcGNpMTg6IDx1bmtu b3duPiBhdCBkZXZpY2UgMC4wIChubyBkcml2ZXIgYXR0YWNoZWQpDQpwY2kxODogPGVuY3J5 cHQvZGVjcnlwdD4gYXQgZGV2aWNlIDAuMSAobm8gZHJpdmVyIGF0dGFjaGVkKQ0KYWhjaTI6 IDxBTUQgS0VSTkNaIEFIQ0kgU0FUQSBjb250cm9sbGVyPiBtZW0gMHhlNzIwMjAwMC0weGU3 MjAyZmZmIGlycSAxMzggYXQgZGV2aWNlIDAuMiBvbiBwY2kxOA0KYWhjaTI6IEFIQ0kgdjEu MzEgd2l0aCA4IDZHYnBzIHBvcnRzLCBQb3J0IE11bHRpcGxpZXIgc3VwcG9ydGVkIHdpdGgg RkJTDQphaGNpY2g5OiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDAgb24gYWhjaTINCmFo Y2ljaDEwOiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDEgb24gYWhjaTINCmFoY2ljaDEx OiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDIgb24gYWhjaTINCmFoY2ljaDEyOiA8QUhD SSBjaGFubmVsPiBhdCBjaGFubmVsIDMgb24gYWhjaTINCmFoY2ljaDEzOiA8QUhDSSBjaGFu bmVsPiBhdCBjaGFubmVsIDQgb24gYWhjaTINCmFoY2ljaDE0OiA8QUhDSSBjaGFubmVsPiBh dCBjaGFubmVsIDUgb24gYWhjaTINCmFoY2ljaDE1OiA8QUhDSSBjaGFubmVsPiBhdCBjaGFu bmVsIDYgb24gYWhjaTINCmFoY2ljaDE2OiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDcg b24gYWhjaTINCmFoY2llbTE6IDxBSENJIGVuY2xvc3VyZSBtYW5hZ2VtZW50IGJyaWRnZT4g b24gYWhjaTINCmFjcGlfYnV0dG9uMDogPFBvd2VyIEJ1dHRvbj4gb24gYWNwaTANCnVhcnQw OiA8MTY1NTAgb3IgY29tcGF0aWJsZT4gcG9ydCAweDNmOC0weDNmZiBpcnEgNCBmbGFncyAw eDEwIG9uIGFjcGkwDQp1YXJ0MTogPDE2NTUwIG9yIGNvbXBhdGlibGU+IHBvcnQgMHgyZjgt MHgyZmYgaXJxIDMgb24gYWNwaTANCmlwbWkwOiA8SVBNSSBTeXN0ZW0gSW50ZXJmYWNlPiBw b3J0IDB4Y2E0LDB4Y2E1IG9uIGFjcGkwDQppcG1pMDogS0NTIG1vZGUgZm91bmQgYXQgaW8g MHhjYTQgb24gYWNwaQ0Kb3JtMDogPElTQSBPcHRpb24gUk9NPiBhdCBpb21lbSAweGMwMDAw LTB4YzdmZmYgb24gaXNhMA0KcHBjMDogY2Fubm90IHJlc2VydmUgSS9PIHBvcnQgcmFuZ2UN Cmh3cHN0YXRlMDogPENvb2xgbidRdWlldCAyLjA+IG9uIGNwdTANClpGUyBmaWxlc3lzdGVt IHZlcnNpb246IDUNClpGUyBzdG9yYWdlIHBvb2wgdmVyc2lvbjogZmVhdHVyZXMgc3VwcG9y dCAoNTAwMCkNClRpbWVjb3VudGVycyB0aWNrIGV2ZXJ5IDEuMDAwIG1zZWMNCmlwbWkwOiBJ UE1JIGRldmljZSByZXYuIDEsIGZpcm13YXJlIHJldi4gMS4yOCwgdmVyc2lvbiAyLjANCmlw bWkwOiBOdW1iZXIgb2YgY2hhbm5lbHMgMg0KaXBtaTA6IEF0dGFjaGVkIHdhdGNoZG9nDQp1 Z2VuMy4xOiA8MHgxYjIxIFhIQ0kgcm9vdCBIVUI+IGF0IHVzYnVzMw0KdWdlbjQuMTogPDB4 MTAyMiBYSENJIHJvb3QgSFVCPiBhdCB1c2J1czQNCnVnZW4yLjE6IDwweDFiMjEgWEhDSSBy b290IEhVQj4gYXQgdXNidXMyDQp1Z2VuMC4xOiA8MHgxYjIxIFhIQ0kgcm9vdCBIVUI+IGF0 IHVzYnVzMA0KdWdlbjEuMTogPDB4MTAyMiBYSENJIHJvb3QgSFVCPiBhdCB1c2J1czENCnVo dWIwOiBhaGNpZW0wOiA8MHgxYjIxIFhIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDMu MDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czMNClVuc3VwcG9ydGVkIGVuY2xvc3VyZSBpbnRl cmZhY2UNCihhcHJvYmUyOmFoY2llbTA6MDowOjApOiBTRVBfQVRUTiBJREVOVElGWS4gQUNC OiA2NyBlYyAwMiAwMCAwMCA0MCAwMCAwMCAwMCAwMCA4MCAwMA0KdWh1YjI6IChhcHJvYmUy OmFoY2llbTA6MDowOjApOiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZA0K PDB4MTAyMiBYSENJIHJvb3QgSFVCLCBjbGFzcyA5LzAsIHJldiAzLjAwLzEuMDAsIGFkZHIg MT4gb24gdXNidXM0DQp1aHViMTogKGFwcm9iZTI6PDB4MTAyMiBYSENJIHJvb3QgSFVCLCBj bGFzcyA5LzAsIHJldiAzLjAwLzEuMDAsIGFkZHIgMT4gb24gdXNidXMxDQphaGNpZW0xOiB1 aHViMzogYWhjaWVtMDowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcg0KPDB4 MWIyMSBYSENJIHJvb3QgSFVCLCBjbGFzcyA5LzAsIHJldiAzLjAwLzEuMDAsIGFkZHIgMT4g b24gdXNidXMwDQpVbnN1cHBvcnRlZCBlbmNsb3N1cmUgaW50ZXJmYWNlDQp1aHViNDogKGFw cm9iZTA6YWhjaWVtMTowOjA6MCk6IFNFUF9BVFROIElERU5USUZZLiBBQ0I6IDY3IGVjIDAy IDAwIDAwIDQwIDAwIDAwIDAwIDAwIDgwIDAwDQo8MHgxYjIxIFhIQ0kgcm9vdCBIVUIsIGNs YXNzIDkvMCwgcmV2IDMuMDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czINCihhcHJvYmUwOmFo Y2llbTE6MDowOjApOiBDQU0gc3RhdHVzOiBDQ0IgcmVxdWVzdCB3YXMgaW52YWxpZA0KKGFw cm9iZTA6YWhjaWVtMTowOjA6MCk6IEVycm9yIDIyLCBVbnJldHJ5YWJsZSBlcnJvcg0KYWRh MCBhdCBhaGNpY2gzIGJ1cyAwIHNjYnVzMyB0YXJnZXQgMCBsdW4gMA0KYWRhMDogPFNhbXN1 bmcgU1NEIDg1MCBFVk8gMjUwR0IgRU1UMDJCNlE+IEFDUy0yIEFUQSBTQVRBIDMueCBkZXZp Y2UNCmFkYTA6IFNlcmlhbCBOdW1iZXIgUzJSNU5CMEoxMDQ3OTRMDQphZGEwOiA2MDAuMDAw TUIvcyB0cmFuc2ZlcnMgKFNBVEEgMy54LCBVRE1BNiwgUElPIDUxMmJ5dGVzKQ0KYWRhMDog Q29tbWFuZCBRdWV1ZWluZyBlbmFibGVkDQphZGEwOiAyMzg0NzVNQiAoNDg4Mzk3MTY4IDUx MiBieXRlIHNlY3RvcnMpDQphZGEwOiBxdWlya3M9MHgzPDRLLE5DUV9UUklNX0JST0tFTj4N CmFkYTEgYXQgYWhjaWNoNCBidXMgMCBzY2J1czQgdGFyZ2V0IDAgbHVuIDANCmFkYTE6IDxT YW1zdW5nIFNTRCA4NTAgRVZPIDI1MEdCIEVNVDAzQjZRPiBBQ1MtMiBBVEEgU0FUQSAzLngg ZGV2aWNlDQphZGExOiBTZXJpYWwgTnVtYmVyIFMzUFpORjBKOTAzMTA3Sg0KYWRhMTogNjAw LjAwME1CL3MgdHJhbnNmZXJzIChTQVRBIDMueCwgVURNQTYsIFBJTyA1MTJieXRlcykNCmFk YTE6IENvbW1hbmQgUXVldWVpbmcgZW5hYmxlZA0KYWRhMTogMjM4NDc1TUIgKDQ4ODM5NzE2 OCA1MTIgYnl0ZSBzZWN0b3JzKQ0KYWRhMTogcXVpcmtzPTB4Mzw0SyxOQ1FfVFJJTV9CUk9L RU4+DQpUcnlpbmcgdG8gbW91bnQgcm9vdCBmcm9tIHpmczp6cm9vdC9ST09UL2RlZmF1bHQg W10uLi4NClJvb3QgbW91bnQgd2FpdGluZyBmb3I6IHVzYnVzNCB1c2J1czMgdXNidXMyIHVz YnVzMSB1c2J1czANCnVodWIzOiA0IHBvcnRzIHdpdGggNCByZW1vdmFibGUsIHNlbGYgcG93 ZXJlZA0KdWh1YjQ6IDQgcG9ydHMgd2l0aCA0IHJlbW92YWJsZSwgc2VsZiBwb3dlcmVkDQp1 aHViMjogNCBwb3J0cyB3aXRoIDQgcmVtb3ZhYmxlLCBzZWxmIHBvd2VyZWQNCnVodWIwOiA0 IHBvcnRzIHdpdGggNCByZW1vdmFibGUsIHNlbGYgcG93ZXJlZA0KdWh1YjE6IDQgcG9ydHMg d2l0aCA0IHJlbW92YWJsZSwgc2VsZiBwb3dlcmVkDQp1Z2VuNC4yOiA8dmVuZG9yIDB4MDU1 NyBwcm9kdWN0IDB4NzAwMD4gYXQgdXNidXM0DQp1aHViNSBvbiB1aHViMg0KdWh1YjU6IDx2 ZW5kb3IgMHgwNTU3IHByb2R1Y3QgMHg3MDAwLCBjbGFzcyA5LzAsIHJldiAyLjAwLzAuMDAs IGFkZHIgMT4gb24gdXNidXM0DQp1Z2VuMC4yOiA8dmVuZG9yIDB4MTNiYSBHZW5lcmljIFVT QiBLQj4gYXQgdXNidXMwDQp1a2JkMCBvbiB1aHViMw0KdWtiZDA6IDx2ZW5kb3IgMHgxM2Jh IEdlbmVyaWMgVVNCIEtCLCBjbGFzcyAwLzAsIHJldiAxLjEwLzAuMDEsIGFkZHIgMT4gb24g dXNidXMwDQprYmQyIGF0IHVrYmQwDQp1aHViNTogNCBwb3J0cyB3aXRoIDMgcmVtb3ZhYmxl LCBzZWxmIHBvd2VyZWQNCnVnZW40LjM6IDx2ZW5kb3IgMHgwNTU3IHByb2R1Y3QgMHgyNDE5 PiBhdCB1c2J1czQNCnVrYmQxIG9uIHVodWI1DQp1a2JkMTogPHZlbmRvciAweDA1NTcgcHJv ZHVjdCAweDI0MTksIGNsYXNzIDAvMCwgcmV2IDEuMTAvMS4wMCwgYWRkciAyPiBvbiB1c2J1 czQNCmtiZDMgYXQgdWtiZDENCmlnYjA6IGxpbmsgc3RhdGUgY2hhbmdlZCB0byBVUA0KdW1z MCBvbiB1aHViNQ0KdW1zMDogPHZlbmRvciAweDA1NTcgcHJvZHVjdCAweDI0MTksIGNsYXNz IDAvMCwgcmV2IDEuMTAvMS4wMCwgYWRkciAyPiBvbiB1c2J1czQNCnVtczA6IDMgYnV0dG9u cyBhbmQgW1pdIGNvb3JkaW5hdGVzIElEPTANCnVtczEgb24gdWh1YjMNCnVtczE6IDx2ZW5k b3IgMHgxM2JhIEdlbmVyaWMgVVNCIEtCLCBjbGFzcyAwLzAsIHJldiAxLjEwLzAuMDEsIGFk ZHIgMT4gb24gdXNidXMwDQp1bXMxOiAzIGJ1dHRvbnMgYW5kIFtYWVpdIGNvb3JkaW5hdGVz IElEPTENCmFtZHNtbjA6IDxBTUQgRmFtaWx5IDE3aCBTeXN0ZW0gTWFuYWdlbWVudCBOZXR3 b3JrPiBvbiBob3N0YjANCmFtZHNtbjE6IDxBTUQgRmFtaWx5IDE3aCBTeXN0ZW0gTWFuYWdl bWVudCBOZXR3b3JrPiBvbiBob3N0YjM5DQphbWRzbW4yOiA8QU1EIEZhbWlseSAxN2ggU3lz dGVtIE1hbmFnZW1lbnQgTmV0d29yaz4gb24gaG9zdGI0Ng0KYW1kc21uMzogPEFNRCBGYW1p bHkgMTdoIFN5c3RlbSBNYW5hZ2VtZW50IE5ldHdvcms+IG9uIGhvc3RiNTMNCmFtZHRlbXAw OiA8QU1EIENQVSBPbi1EaWUgVGhlcm1hbCBTZW5zb3JzPiBvbiBob3N0YjANCmFtZHRlbXAx OiA8QU1EIENQVSBPbi1EaWUgVGhlcm1hbCBTZW5zb3JzPiBvbiBob3N0YjM5DQphbWR0ZW1w MjogPEFNRCBDUFUgT24tRGllIFRoZXJtYWwgU2Vuc29ycz4gb24gaG9zdGI0Ng0KYW1kdGVt cDM6IDxBTUQgQ1BVIE9uLURpZSBUaGVybWFsIFNlbnNvcnM+IG9uIGhvc3RiNTMNCg0KDQoj IHN5c2N0bCAtYSBkZXYuYW1kdGVtcA0KZGV2LmFtZHRlbXAuMy5jb3JlMC5zZW5zb3IwOiAz Ni4xQw0KZGV2LmFtZHRlbXAuMy5zZW5zb3Jfb2Zmc2V0OiAwDQpkZXYuYW1kdGVtcC4zLiVw YXJlbnQ6IGhvc3RiNTMNCmRldi5hbWR0ZW1wLjMuJXBucGluZm86IA0KZGV2LmFtZHRlbXAu My4lbG9jYXRpb246IA0KZGV2LmFtZHRlbXAuMy4lZHJpdmVyOiBhbWR0ZW1wDQpkZXYuYW1k dGVtcC4zLiVkZXNjOiBBTUQgQ1BVIE9uLURpZSBUaGVybWFsIFNlbnNvcnMNCmRldi5hbWR0 ZW1wLjIuY29yZTAuc2Vuc29yMDogMzMuNkMNCmRldi5hbWR0ZW1wLjIuc2Vuc29yX29mZnNl dDogMA0KZGV2LmFtZHRlbXAuMi4lcGFyZW50OiBob3N0YjQ2DQpkZXYuYW1kdGVtcC4yLiVw bnBpbmZvOiANCmRldi5hbWR0ZW1wLjIuJWxvY2F0aW9uOiANCmRldi5hbWR0ZW1wLjIuJWRy aXZlcjogYW1kdGVtcA0KZGV2LmFtZHRlbXAuMi4lZGVzYzogQU1EIENQVSBPbi1EaWUgVGhl cm1hbCBTZW5zb3JzDQpkZXYuYW1kdGVtcC4xLmNvcmUwLnNlbnNvcjA6IDMzLjZDDQpkZXYu YW1kdGVtcC4xLnNlbnNvcl9vZmZzZXQ6IDANCmRldi5hbWR0ZW1wLjEuJXBhcmVudDogaG9z dGIzOQ0KZGV2LmFtZHRlbXAuMS4lcG5waW5mbzogDQpkZXYuYW1kdGVtcC4xLiVsb2NhdGlv bjogDQpkZXYuYW1kdGVtcC4xLiVkcml2ZXI6IGFtZHRlbXANCmRldi5hbWR0ZW1wLjEuJWRl c2M6IEFNRCBDUFUgT24tRGllIFRoZXJtYWwgU2Vuc29ycw0KZGV2LmFtZHRlbXAuMC5jb3Jl MC5zZW5zb3IwOiAzNi4xQw0KZGV2LmFtZHRlbXAuMC5zZW5zb3Jfb2Zmc2V0OiAwDQpkZXYu YW1kdGVtcC4wLiVwYXJlbnQ6IGhvc3RiMA0KZGV2LmFtZHRlbXAuMC4lcG5waW5mbzogDQpk ZXYuYW1kdGVtcC4wLiVsb2NhdGlvbjogDQpkZXYuYW1kdGVtcC4wLiVkcml2ZXI6IGFtZHRl bXANCmRldi5hbWR0ZW1wLjAuJWRlc2M6IEFNRCBDUFUgT24tRGllIFRoZXJtYWwgU2Vuc29y cw0KZGV2LmFtZHRlbXAuJXBhcmVudDogDQpyb290QGVweWMtYnNkOi9ob21lL21kdGFuY3Nh ICMgc3lzY3RsIC1hIHwgZ3JlcCB0ZW1wZXJhDQpkZXYuY3B1LjMxLnRlbXBlcmF0dXJlOiAz Ni4xQw0KZGV2LmNwdS4zMC50ZW1wZXJhdHVyZTogMzYuMUMNCmRldi5jcHUuMjkudGVtcGVy YXR1cmU6IDM2LjFDDQpkZXYuY3B1LjI4LnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS4y Ny50ZW1wZXJhdHVyZTogMzYuMUMNCmRldi5jcHUuMjYudGVtcGVyYXR1cmU6IDM2LjFDDQpk ZXYuY3B1LjI1LnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS4yNC50ZW1wZXJhdHVyZTog MzYuMUMNCmRldi5jcHUuMjMudGVtcGVyYXR1cmU6IDM2LjFDDQpkZXYuY3B1LjIyLnRlbXBl cmF0dXJlOiAzNi4xQw0KZGV2LmNwdS4yMS50ZW1wZXJhdHVyZTogMzYuMUMNCmRldi5jcHUu MjAudGVtcGVyYXR1cmU6IDM2LjFDDQpkZXYuY3B1LjE5LnRlbXBlcmF0dXJlOiAzNi4xQw0K ZGV2LmNwdS4xOC50ZW1wZXJhdHVyZTogMzYuMUMNCmRldi5jcHUuMTcudGVtcGVyYXR1cmU6 IDM2LjFDDQpkZXYuY3B1LjE2LnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS4xNS50ZW1w ZXJhdHVyZTogMzYuMUMNCmRldi5jcHUuMTQudGVtcGVyYXR1cmU6IDM2LjFDDQpkZXYuY3B1 LjEzLnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS4xMi50ZW1wZXJhdHVyZTogMzYuMUMN CmRldi5jcHUuMTEudGVtcGVyYXR1cmU6IDM2LjFDDQpkZXYuY3B1LjEwLnRlbXBlcmF0dXJl OiAzNi4xQw0KZGV2LmNwdS45LnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS44LnRlbXBl cmF0dXJlOiAzNi4xQw0KZGV2LmNwdS43LnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS42 LnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS41LnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2 LmNwdS40LnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS4zLnRlbXBlcmF0dXJlOiAzNi4x Qw0KZGV2LmNwdS4yLnRlbXBlcmF0dXJlOiAzNi4xQw0KZGV2LmNwdS4xLnRlbXBlcmF0dXJl OiAzNi4xQw0KZGV2LmNwdS4wLnRlbXBlcmF0dXJlOiAzNi4xQw0Kcm9vdEBlcHljLWJzZDov aG9tZS9tZHRhbmNzYSAjIA0KDQojIGlwbWl0b29sIHNlbnNvcg0KQ1BVIFRlbXAgICAgICAg ICB8IDI1LjAwMCAgICAgfCBkZWdyZWVzIEMgIHwgb2sgICAgfCA1LjAwMCAgICAgfCA1LjAw MCAgICAgfCAxMC4wMDAgICAgfCA5NS4wMDAgICAgfCAxMDAuMDAwICAgfCAxMDAuMDAwICAg DQpTeXN0ZW0gVGVtcCAgICAgIHwgMzguMDAwICAgICB8IGRlZ3JlZXMgQyAgfCBvayAgICB8 IDUuMDAwICAgICB8IDUuMDAwICAgICB8IDEwLjAwMCAgICB8IDgwLjAwMCAgICB8IDg1LjAw MCAgICB8IDkwLjAwMCAgICANClBlcmlwaGVyYWwgVGVtcCAgfCAzNC4wMDAgICAgIHwgZGVn cmVlcyBDICB8IG9rICAgIHwgNS4wMDAgICAgIHwgNS4wMDAgICAgIHwgMTAuMDAwICAgIHwg ODAuMDAwICAgIHwgODUuMDAwICAgIHwgOTAuMDAwICAgIA0KVlJNQ3B1IFRlbXAgICAgICB8 IDM3LjAwMCAgICAgfCBkZWdyZWVzIEMgIHwgb2sgICAgfCA1LjAwMCAgICAgfCA1LjAwMCAg ICAgfCAxMC4wMDAgICAgfCA5NS4wMDAgICAgfCAxMDAuMDAwICAgfCAxMDUuMDAwICAgDQpW Uk1Tb2MgVGVtcCAgICAgIHwgMzcuMDAwICAgICB8IGRlZ3JlZXMgQyAgfCBvayAgICB8IDUu MDAwICAgICB8IDUuMDAwICAgICB8IDEwLjAwMCAgICB8IDk1LjAwMCAgICB8IDEwMC4wMDAg ICB8IDEwNS4wMDAgICANClZSTUFCQ0QgVGVtcCAgICAgfCAzNi4wMDAgICAgIHwgZGVncmVl cyBDICB8IG9rICAgIHwgNS4wMDAgICAgIHwgNS4wMDAgICAgIHwgMTAuMDAwICAgIHwgOTUu MDAwICAgIHwgMTAwLjAwMCAgIHwgMTA1LjAwMCAgIA0KVlJNRUZHSCBUZW1wICAgICB8IDM0 LjAwMCAgICAgfCBkZWdyZWVzIEMgIHwgb2sgICAgfCA1LjAwMCAgICAgfCA1LjAwMCAgICAg fCAxMC4wMDAgICAgfCA5NS4wMDAgICAgfCAxMDAuMDAwICAgfCAxMDUuMDAwICAgDQpQMS1E SU1NQTEgVGVtcCAgIHwgbmEgICAgICAgICB8ICAgICAgICAgICAgfCBuYSAgICB8IG5hICAg ICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8 IG5hICAgICAgICANClAxLURJTU1CMSBUZW1wICAgfCBuYSAgICAgICAgIHwgICAgICAgICAg ICB8IG5hICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAg ICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIA0KUDEtRElNTUMxIFRlbXAgICB8IG5hICAg ICAgICAgfCAgICAgICAgICAgIHwgbmEgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBu YSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgDQpQMS1ESU1N RDEgVGVtcCAgIHwgbmEgICAgICAgICB8ICAgICAgICAgICAgfCBuYSAgICB8IG5hICAgICAg ICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5h ICAgICAgICANClAxLURJTU1FMSBUZW1wICAgfCAyNy4wMDAgICAgIHwgZGVncmVlcyBDICB8 IG9rICAgIHwgNS4wMDAgICAgIHwgNS4wMDAgICAgIHwgMTAuMDAwICAgIHwgODAuMDAwICAg IHwgODUuMDAwICAgIHwgOTAuMDAwICAgIA0KUDEtRElNTUYxIFRlbXAgICB8IDI3LjAwMCAg ICAgfCBkZWdyZWVzIEMgIHwgb2sgICAgfCA1LjAwMCAgICAgfCA1LjAwMCAgICAgfCAxMC4w MDAgICAgfCA4MC4wMDAgICAgfCA4NS4wMDAgICAgfCA5MC4wMDAgICAgDQpQMS1ESU1NRzEg VGVtcCAgIHwgbmEgICAgICAgICB8ICAgICAgICAgICAgfCBuYSAgICB8IG5hICAgICAgICB8 IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAg ICAgICANClAxLURJTU1IMSBUZW1wICAgfCBuYSAgICAgICAgIHwgICAgICAgICAgICB8IG5h ICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwg bmEgICAgICAgIHwgbmEgICAgICAgIA0KRkFOMSAgICAgICAgICAgICB8IG5hICAgICAgICAg fCAgICAgICAgICAgIHwgbmEgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAg ICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgDQpGQU4yICAgICAgICAg ICAgIHwgbmEgICAgICAgICB8ICAgICAgICAgICAgfCBuYSAgICB8IG5hICAgICAgICB8IG5h ICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAg ICANCkZBTjMgICAgICAgICAgICAgfCBuYSAgICAgICAgIHwgICAgICAgICAgICB8IG5hICAg IHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEg ICAgICAgIHwgbmEgICAgICAgIA0KRkFONCAgICAgICAgICAgICB8IG5hICAgICAgICAgfCAg ICAgICAgICAgIHwgbmEgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAg fCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgDQpGQU41ICAgICAgICAgICAg IHwgbmEgICAgICAgICB8ICAgICAgICAgICAgfCBuYSAgICB8IG5hICAgICAgICB8IG5hICAg ICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICAN CkZBTkEgICAgICAgICAgICAgfCAzNzAwLjAwMCAgIHwgUlBNICAgICAgICB8IG9rICAgIHwg MzAwLjAwMCAgIHwgNTAwLjAwMCAgIHwgNzAwLjAwMCAgIHwgMjUzMDAuMDAwIHwgMjU0MDAu MDAwIHwgMjU1MDAuMDAwIA0KRkFOQiAgICAgICAgICAgICB8IG5hICAgICAgICAgfCAgICAg ICAgICAgIHwgbmEgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBu YSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgDQoxMlYgICAgICAgICAgICAgIHwg MTEuNzkyICAgICB8IFZvbHRzICAgICAgfCBvayAgICB8IDkuNjgwICAgICB8IDkuOTM2ICAg ICB8IDEwLjcwNCAgICB8IDEzLjcxMiAgICB8IDE0LjQ4MCAgICB8IDE0LjczNiAgICANCjVW Q0MgICAgICAgICAgICAgfCA0Ljk2MCAgICAgIHwgVm9sdHMgICAgICB8IG9rICAgIHwgMy45 NDAgICAgIHwgNC4wMzAgICAgIHwgNC4zNjAgICAgIHwgNS41OTAgICAgIHwgNS45MjAgICAg IHwgNi4wMTAgICAgIA0KMy4zVkNDICAgICAgICAgICB8IDMuMjkzICAgICAgfCBWb2x0cyAg ICAgIHwgb2sgICAgfCAyLjYxMyAgICAgfCAyLjY4MSAgICAgfCAyLjg4NSAgICAgfCAzLjcx OCAgICAgfCAzLjkyMiAgICAgfCAzLjk5MCAgICAgDQpWQkFUICAgICAgICAgICAgIHwgMHg0 ICAgICAgICB8IGRpc2NyZXRlICAgfCAweDA0ZmZ8IG5hICAgICAgICB8IG5hICAgICAgICB8 IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICANClAxX1ZE RENSICAgICAgICAgfCAwLjk4NSAgICAgIHwgVm9sdHMgICAgICB8IG9rICAgIHwgMC40MDAg ICAgIHwgMC40OTkgICAgIHwgMC40OTkgICAgIHwgMS4zNDUgICAgIHwgMS4zNDUgICAgIHwg MS4zOTkgICAgIA0KUDFfVk1FTUFCQ0QgICAgICB8IDEuMTg5ICAgICAgfCBWb2x0cyAgICAg IHwgb2sgICAgfCAwLjk3OSAgICAgfCAxLjAwMyAgICAgfCAxLjA4MSAgICAgfCAxLjM4NyAg ICAgfCAxLjQ2NSAgICAgfCAxLjQ4OSAgICAgDQpQMV9WTUVNRUZHSCAgICAgIHwgMS4xOTMg ICAgICB8IFZvbHRzICAgICAgfCBvayAgICB8IDAuOTc2ICAgICB8IDAuOTk3ICAgICB8IDEu MDc0ICAgICB8IDEuMzg5ICAgICB8IDEuNDY2ICAgICB8IDEuNDg3ICAgICANClZERF81X0RV QUwgICAgICAgfCA1LjA5OSAgICAgIHwgVm9sdHMgICAgICB8IG9rICAgIHwgNC4wMTkgICAg IHwgNC4xMzkgICAgIHwgNC40MzkgICAgIHwgNS43MjkgICAgIHwgNi4wMjkgICAgIHwgNi4x NDkgICAgIA0KVkREXzMzX0RVQUwgICAgICB8IDMuMjkzICAgICAgfCBWb2x0cyAgICAgIHwg b2sgICAgfCAyLjYxMyAgICAgfCAyLjY4MSAgICAgfCAyLjg4NSAgICAgfCAzLjcxOCAgICAg fCAzLjkyMiAgICAgfCAzLjk5MCAgICAgDQpQMV9TT0NSVU4gICAgICAgIHwgMC45NzIgICAg ICB8IFZvbHRzICAgICAgfCBvayAgICB8IDAuMzAwICAgICB8IDAuNDk2ICAgICB8IDAuNDk2 ICAgICB8IDEuMTQ3ICAgICB8IDEuMTQ3ICAgICB8IDEuMzQzICAgICANClAxX1NPQ0RVQUwg ICAgICAgfCAwLjg4NiAgICAgIHwgVm9sdHMgICAgICB8IG9rICAgIHwgMC43MTEgICAgIHwg MC43MjUgICAgIHwgMC43ODEgICAgIHwgMS4wMTIgICAgIHwgMS4wNjggICAgIHwgMS4wODIg ICAgIA0KQ2hhc3NpcyBJbnRydSAgICB8IDB4MCAgICAgICAgfCBkaXNjcmV0ZSAgIHwgMHgw MDAwfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBu YSAgICAgICAgfCBuYSAgICAgICAgDQpDUFUgVGVtcCAgICAgICAgIHwgMzUuMDAwICAgICB8 IGRlZ3JlZXMgQyAgfCBvayAgICB8IDUuMDAwICAgICB8IDUuMDAwICAgICB8IDEwLjAwMCAg ICB8IDk1LjAwMCAgICB8IDEwMC4wMDAgICB8IDEwMC4wMDAgICANClN5c3RlbSBUZW1wICAg ICAgfCAzOC4wMDAgICAgIHwgZGVncmVlcyBDICB8IG9rICAgIHwgNS4wMDAgICAgIHwgNS4w MDAgICAgIHwgMTAuMDAwICAgIHwgODAuMDAwICAgIHwgODUuMDAwICAgIHwgOTAuMDAwICAg IA0KUGVyaXBoZXJhbCBUZW1wICB8IDM1LjAwMCAgICAgfCBkZWdyZWVzIEMgIHwgb2sgICAg fCA1LjAwMCAgICAgfCA1LjAwMCAgICAgfCAxMC4wMDAgICAgfCA4MC4wMDAgICAgfCA4NS4w MDAgICAgfCA5MC4wMDAgICAgDQpWUk1DcHUgVGVtcCAgICAgIHwgMzguMDAwICAgICB8IGRl Z3JlZXMgQyAgfCBvayAgICB8IDUuMDAwICAgICB8IDUuMDAwICAgICB8IDEwLjAwMCAgICB8 IDk1LjAwMCAgICB8IDEwMC4wMDAgICB8IDEwNS4wMDAgICANClZSTVNvYyBUZW1wICAgICAg fCA0MS4wMDAgICAgIHwgZGVncmVlcyBDICB8IG9rICAgIHwgNS4wMDAgICAgIHwgNS4wMDAg ICAgIHwgMTAuMDAwICAgIHwgOTUuMDAwICAgIHwgMTAwLjAwMCAgIHwgMTA1LjAwMCAgIA0K VlJNQUJDRCBUZW1wICAgICB8IDM2LjAwMCAgICAgfCBkZWdyZWVzIEMgIHwgb2sgICAgfCA1 LjAwMCAgICAgfCA1LjAwMCAgICAgfCAxMC4wMDAgICAgfCA5NS4wMDAgICAgfCAxMDAuMDAw ICAgfCAxMDUuMDAwICAgDQpWUk1FRkdIIFRlbXAgICAgIHwgMzUuMDAwICAgICB8IGRlZ3Jl ZXMgQyAgfCBvayAgICB8IDUuMDAwICAgICB8IDUuMDAwICAgICB8IDEwLjAwMCAgICB8IDk1 LjAwMCAgICB8IDEwMC4wMDAgICB8IDEwNS4wMDAgICANClAxLURJTU1BMSBUZW1wICAgfCBu YSAgICAgICAgIHwgICAgICAgICAgICB8IG5hICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAg IHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIA0KUDEt RElNTUIxIFRlbXAgICB8IG5hICAgICAgICAgfCAgICAgICAgICAgIHwgbmEgICAgfCBuYSAg ICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAg fCBuYSAgICAgICAgDQpQMS1ESU1NQzEgVGVtcCAgIHwgbmEgICAgICAgICB8ICAgICAgICAg ICAgfCBuYSAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAg ICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICANClAxLURJTU1EMSBUZW1wICAgfCBuYSAg ICAgICAgIHwgICAgICAgICAgICB8IG5hICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwg bmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIA0KUDEtRElN TUUxIFRlbXAgICB8IDMwLjAwMCAgICAgfCBkZWdyZWVzIEMgIHwgb2sgICAgfCA1LjAwMCAg ICAgfCA1LjAwMCAgICAgfCAxMC4wMDAgICAgfCA4MC4wMDAgICAgfCA4NS4wMDAgICAgfCA5 MC4wMDAgICAgDQpQMS1ESU1NRjEgVGVtcCAgIHwgMzAuMDAwICAgICB8IGRlZ3JlZXMgQyAg fCBvayAgICB8IDUuMDAwICAgICB8IDUuMDAwICAgICB8IDEwLjAwMCAgICB8IDgwLjAwMCAg ICB8IDg1LjAwMCAgICB8IDkwLjAwMCAgICANClAxLURJTU1HMSBUZW1wICAgfCBuYSAgICAg ICAgIHwgICAgICAgICAgICB8IG5hICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEg ICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIA0KUDEtRElNTUgx IFRlbXAgICB8IG5hICAgICAgICAgfCAgICAgICAgICAgIHwgbmEgICAgfCBuYSAgICAgICAg fCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAg ICAgICAgDQpGQU4xICAgICAgICAgICAgIHwgbmEgICAgICAgICB8ICAgICAgICAgICAgfCBu YSAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8 IG5hICAgICAgICB8IG5hICAgICAgICANCkZBTjIgICAgICAgICAgICAgfCBuYSAgICAgICAg IHwgICAgICAgICAgICB8IG5hICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAg ICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIA0KRkFOMyAgICAgICAg ICAgICB8IG5hICAgICAgICAgfCAgICAgICAgICAgIHwgbmEgICAgfCBuYSAgICAgICAgfCBu YSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAgICAgfCBuYSAgICAg ICAgDQpGQU40ICAgICAgICAgICAgIHwgbmEgICAgICAgICB8ICAgICAgICAgICAgfCBuYSAg ICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5h ICAgICAgICB8IG5hICAgICAgICANCkZBTjUgICAgICAgICAgICAgfCBuYSAgICAgICAgIHwg ICAgICAgICAgICB8IG5hICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAg IHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIA0KRkFOQSAgICAgICAgICAg ICB8IDM2MDAuMDAwICAgfCBSUE0gICAgICAgIHwgb2sgICAgfCAzMDAuMDAwICAgfCA1MDAu MDAwICAgfCA3MDAuMDAwICAgfCAyNTMwMC4wMDAgfCAyNTQwMC4wMDAgfCAyNTUwMC4wMDAg DQpGQU5CICAgICAgICAgICAgIHwgbmEgICAgICAgICB8ICAgICAgICAgICAgfCBuYSAgICB8 IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAg ICAgICB8IG5hICAgICAgICANCjEyViAgICAgICAgICAgICAgfCAxMS42NjQgICAgIHwgVm9s dHMgICAgICB8IG9rICAgIHwgOS42ODAgICAgIHwgOS45MzYgICAgIHwgMTAuNzA0ICAgIHwg MTMuNzEyICAgIHwgMTQuNDgwICAgIHwgMTQuNzM2ICAgIA0KNVZDQyAgICAgICAgICAgICB8 IDQuOTYwICAgICAgfCBWb2x0cyAgICAgIHwgb2sgICAgfCAzLjk0MCAgICAgfCA0LjAzMCAg ICAgfCA0LjM2MCAgICAgfCA1LjU5MCAgICAgfCA1LjkyMCAgICAgfCA2LjAxMCAgICAgDQoz LjNWQ0MgICAgICAgICAgIHwgMy4yNzYgICAgICB8IFZvbHRzICAgICAgfCBvayAgICB8IDIu NjEzICAgICB8IDIuNjgxICAgICB8IDIuODg1ICAgICB8IDMuNzE4ICAgICB8IDMuOTIyICAg ICB8IDMuOTkwICAgICANClZCQVQgICAgICAgICAgICAgfCAweDQgICAgICAgIHwgZGlzY3Jl dGUgICB8IDB4MDRmZnwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIHwgbmEg ICAgICAgIHwgbmEgICAgICAgIHwgbmEgICAgICAgIA0KUDFfVkREQ1IgICAgICAgICB8IDAu OTg1ICAgICAgfCBWb2x0cyAgICAgIHwgb2sgICAgfCAwLjQwMCAgICAgfCAwLjQ5OSAgICAg fCAwLjQ5OSAgICAgfCAxLjM0NSAgICAgfCAxLjM0NSAgICAgfCAxLjM5OSAgICAgDQpQMV9W TUVNQUJDRCAgICAgIHwgMS4xOTUgICAgICB8IFZvbHRzICAgICAgfCBvayAgICB8IDAuOTc5 ICAgICB8IDEuMDAzICAgICB8IDEuMDgxICAgICB8IDEuMzg3ICAgICB8IDEuNDY1ICAgICB8 IDEuNDg5ICAgICANClAxX1ZNRU1FRkdIICAgICAgfCAxLjIwMCAgICAgIHwgVm9sdHMgICAg ICB8IG9rICAgIHwgMC45NzYgICAgIHwgMC45OTcgICAgIHwgMS4wNzQgICAgIHwgMS4zODkg ICAgIHwgMS40NjYgICAgIHwgMS40ODcgICAgIA0KVkREXzVfRFVBTCAgICAgICB8IDUuMDk5 ICAgICAgfCBWb2x0cyAgICAgIHwgb2sgICAgfCA0LjAxOSAgICAgfCA0LjEzOSAgICAgfCA0 LjQzOSAgICAgfCA1LjcyOSAgICAgfCA2LjAyOSAgICAgfCA2LjE0OSAgICAgDQpWRERfMzNf RFVBTCAgICAgIHwgMy4yOTMgICAgICB8IFZvbHRzICAgICAgfCBvayAgICB8IDIuNjEzICAg ICB8IDIuNjgxICAgICB8IDIuODg1ICAgICB8IDMuNzE4ICAgICB8IDMuOTIyICAgICB8IDMu OTkwICAgICANClAxX1NPQ1JVTiAgICAgICAgfCAwLjk3OSAgICAgIHwgVm9sdHMgICAgICB8 IG9rICAgIHwgMC4zMDAgICAgIHwgMC40OTYgICAgIHwgMC40OTYgICAgIHwgMS4xNDcgICAg IHwgMS4xNDcgICAgIHwgMS4zNDMgICAgIA0KUDFfU09DRFVBTCAgICAgICB8IDAuODg2ICAg ICAgfCBWb2x0cyAgICAgIHwgb2sgICAgfCAwLjcxMSAgICAgfCAwLjcyNSAgICAgfCAwLjc4 MSAgICAgfCAxLjAxMiAgICAgfCAxLjA2OCAgICAgfCAxLjA4MiAgICAgDQpDaGFzc2lzIElu dHJ1ICAgIHwgMHgwICAgICAgICB8IGRpc2NyZXRlICAgfCAweDAwMDB8IG5hICAgICAgICB8 IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAgICAgICB8IG5hICAg ICANCg0K --------------181331DCA557B80647FF53F4-- From owner-freebsd-stable@freebsd.org Wed Feb 14 08:15:55 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5BE0F0AF01 for ; Wed, 14 Feb 2018 08:15:55 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BD766A4B1 for ; Wed, 14 Feb 2018 08:15:54 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.89 (FreeBSD)) (envelope-from ) id 1elsEL-000P8e-6f; Wed, 14 Feb 2018 09:15:53 +0100 Date: Wed, 14 Feb 2018 09:15:53 +0100 From: Kurt Jaeger To: Mike Tancsa Cc: FreeBSD-STABLE Mailing List Subject: Re: FreeBSD on AMD Epyc boards Message-ID: <20180214081553.GF89804@home.opsec.eu> References: <8037bf98-acc8-6981-d25b-3b58330dbd33@sentex.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8037bf98-acc8-6981-d25b-3b58330dbd33@sentex.net> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Feb 2018 08:15:55 -0000 Hi! > To have a bit of a work around for the Intel Meltdown bug (yes, no > Spectre), I wanted to try out some AMD based CPUs. So far so good using > a SuperMicro H11SSL-i. A decent server board using an Epyc CPU. All > the things you need and expect for a server grade MB On the plus side: 16+16 cores, on the minus: A low CPU tact of 2.2 GHz. Would a box like this be better for a package build host instead of 4+4 cores with 3.x GHz ? -- pi@opsec.eu +49 171 3101372 2 years to go ! From owner-freebsd-stable@freebsd.org Wed Feb 14 11:10:34 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C380F17B7E for ; Wed, 14 Feb 2018 11:10:34 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (mail.soaustin.net [192.108.105.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.soaustin.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0ED971314 for ; Wed, 14 Feb 2018 11:10:33 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from lonesome.com (bones.soaustin.net [192.108.105.22]) by mail.soaustin.net (Postfix) with ESMTPSA id E24D5FB9; Wed, 14 Feb 2018 05:10:25 -0600 (CST) Date: Wed, 14 Feb 2018 05:10:24 -0600 From: Mark Linimon To: Kurt Jaeger Cc: Mike Tancsa , FreeBSD-STABLE Mailing List Subject: package building performance (was: Re: FreeBSD on AMD Epyc boards) Message-ID: <20180214111024.GA9330@lonesome.com> References: <8037bf98-acc8-6981-d25b-3b58330dbd33@sentex.net> <20180214081553.GF89804@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180214081553.GF89804@home.opsec.eu> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Feb 2018 11:10:34 -0000 On Wed, Feb 14, 2018 at 09:15:53AM +0100, Kurt Jaeger wrote: > On the plus side: 16+16 cores, on the minus: A low CPU tact of 2.2 GHz. > Would a box like this be better for a package build host instead of 4+4 cores > with 3.x GHz ? In my experience, "it depends". I think that above a certain number of cores, I/O will dominate. I _think_; I have never done any metrics on any of this. The dominant term of the equation is, as you might guess, RAM. Previous experience suggests that you need at least 2GB per build. By default, nbuilds is set equal to ncores. Less than 2GB-per and you're going to be unhappy. (It's true that for modern systems, where large amounts of RAM are standard, that this is probably no longer a concern.) Put it this way: with 4 cores and 16GB and netbooting (7GB of which was devoted to md(4)), I was having lots of problems on powerpc64. The same machine with 64GB gives me no problems. My guess is that after RAM, there is I/O, ncores, and speed. But I'm just speculating. mcl From owner-freebsd-stable@freebsd.org Wed Feb 14 12:41:21 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A84AF1F14E for ; Wed, 14 Feb 2018 12:41:21 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id ED3BD75215 for ; Wed, 14 Feb 2018 12:41:19 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w1ECfIFN090020 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Wed, 14 Feb 2018 07:41:19 -0500 (EST) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.ca [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w1ECfH6v034624; Wed, 14 Feb 2018 07:41:17 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: FreeBSD on AMD Epyc boards To: Kurt Jaeger Cc: FreeBSD-STABLE Mailing List References: <8037bf98-acc8-6981-d25b-3b58330dbd33@sentex.net> <20180214081553.GF89804@home.opsec.eu> From: Mike Tancsa Organization: Sentex Communications Message-ID: <03042d50-610f-2614-1495-430f5f88434b@sentex.net> Date: Wed, 14 Feb 2018 07:41:17 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180214081553.GF89804@home.opsec.eu> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Feb 2018 12:41:21 -0000 On 2/14/2018 3:15 AM, Kurt Jaeger wrote: > > On the plus side: 16+16 cores, on the minus: A low CPU tact of 2.2 GHz. > Would a box like this be better for a package build host instead of 4+4 cores > with 3.x GHz ? jail server. Lots of processes ---Mike > -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada From owner-freebsd-stable@freebsd.org Fri Feb 16 01:00:23 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1ECFEF047F3; Fri, 16 Feb 2018 01:00:23 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE8CD796D6; Fri, 16 Feb 2018 01:00:19 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id F1E9420D7D; Thu, 15 Feb 2018 20:00:18 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Thu, 15 Feb 2018 20:00:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=cc :content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=Lf/pY97HnoU5QTlCncOgVnA729ais0izgQ78X2Hd+jA=; b=gvBifQfS oLPAZuOVTo/bjBG5/GZ1T+zadvv4HOb3GRDbM7vIGpntNlLH5a3jK7dXFO1HiTjf g+YhcvVTkTTuz+NS5FXffFyA3OgBfidhcoz7oqjlIa20GyV6fUMO5x8iSAAsrjL9 3WsffI/MQYNEWpMTuti5Fp2g9tCGR4forLyHiTtyhLFJ4wDTp487B3x3zSNEbQJc SvuJm3274LDSYHP41etWgp+awDoRqxurPSl43YCVDP5JxzTsuStvfnAgxIkyy1VP iUCzuLdzxEXNw5hjz6LiO94iG3fmHLei7ybQa+dt5j0/zLRPSGP49DLmTkwtjnxq qJZ2cPoQWvM+Pw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=Lf/pY97HnoU5QTlCncOgVnA729ais 0izgQ78X2Hd+jA=; b=BvCrdwc0vSbiYd7bbur/N6V8x0qwcaqj3prhUJAfybteS VDHvG5bIwvP2dANcElRipGk+7QrER+mfnbgCPIU2AfUAGXCFyKtk0WdFYHhBZwtN 9aRzzN9jv4roHcpU2Lei96lWA5C4XaYKlrGcMGSwfXRXtjabMatFvHWap4XnUkXy hTZ6l45fxXZEwFTaVz1STLWFcPqnu3RG8q94a0jolsbjYsjzDykQ5eKsvdo7IyGF s1e4oJBThkh/BevsClIbfjKgKMi82MaRBgBBrvfxpim41DqfKNihFKiW7ZtcuTYW 1T2f9VdCfjiCxlrkRtd3DQfVzufdmXpwnXzCak1AQ== X-ME-Sender: Received: from desktop.parsley.growveg.org (parsley.growveg.org [82.70.91.97]) by mail.messagingengine.com (Postfix) with ESMTPA id 7DE80246D0; Thu, 15 Feb 2018 20:00:18 -0500 (EST) From: tech-lists Subject: is it possible to chroot into arm64 from amd64? To: freebsd-virtualization@freebsd.org Cc: freebsd-stable@freebsd.org Message-ID: <30cfa8a3-76e4-73ac-3b2a-2d64ed24aef8@zyxst.net> Date: Fri, 16 Feb 2018 01:00:16 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Feb 2018 01:00:23 -0000 Hi, Is it possible to have on an amd64 system, a chrooted or jailed arm64 environment? It's possible to cross-compile an arm6 environment on amd64 and then chroot into the arm6 environment - is the same possible for arm64? I can cross-compile arm64 like this: # SYSROOT=/crossbuild/arm64 # cd /usr/src # make -j32 TARGET=arm64 TARGET_ARCH=aarch64 buildworld # make DESTDIR=$SYSROOT TARGET=arm64 TARGET_ARCH=aarch64 installworld # make DESTDIR=$SYSROOT TARGET=arm TARGET_ARCH=armv6 distribution # cp /usr/local/bin/qemu-aarch64-static $SYSROOT/sbin # /usr/sbin/binmiscctl add arm64 --interpreter "/usr/local/bin/qemu-aarch64" \ --magic "\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00" \ --mask "\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff" \ --size 20 --set-enabled # mount -t devfs devfs ${DESTDIR}/dev then: # chroot -u root $SYSROOT then I get the following error: chroot: /bin/csh: No such file or directory # so looked at the chroot, and bin/csh is there! # ls -lah $SYSROOT/bin |less total 9144 drwxr-xr-x 2 root wheel 1.0K Feb 16 00:43 . drwxr-xr-x 18 root wheel 512B Feb 16 00:44 .. -r-xr-xr-x 2 root wheel 195K Feb 16 00:43 [ -r-xr-xr-x 1 root wheel 195K Feb 16 00:43 cat -r-xr-xr-x 1 root wheel 195K Feb 16 00:43 chflags -r-xr-xr-x 1 root wheel 195K Feb 16 00:43 chio -r-xr-xr-x 1 root wheel 195K Feb 16 00:43 chmod -r-xr-xr-x 1 root wheel 195K Feb 16 00:43 cp -r-xr-xr-x 2 root wheel 451K Feb 16 00:43 csh what am I doing wrong? -- J. From owner-freebsd-stable@freebsd.org Fri Feb 16 01:22:09 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB429F0623D for ; Fri, 16 Feb 2018 01:22:09 +0000 (UTC) (envelope-from jonc@chen.org.nz) Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8136C7AA18 for ; Fri, 16 Feb 2018 01:22:08 +0000 (UTC) (envelope-from jonc@chen.org.nz) Received: by mail-qt0-x234.google.com with SMTP id x27so2050300qtm.12 for ; Thu, 15 Feb 2018 17:22:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chen-org-nz.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Rdb5CyqeVz87BXb6WhLJ5Jb1QcugWr1f3khKG5wF6xA=; b=XUXCfn1maab59IMouOl0Ag+5iz77rh8r8Q2kYtMJeTQ7IHhQgAcGEv6chcrpj5OYIM MsCfGyVJAAgm5d+Vj08iHzYltFM3TxAwJSN4Xm7OP1Mr5wP8+cFcr1hfgxUYhOAS6KOK w6N5M0mjk/TMR8IebinsHVJDyogXjYaeKAPnfFbRHRMiZ5lLpmhZSCD+dtF1t1W1u0BO 1DL/rinPblwrL/nmoTPcsr9F9607pWPiIP8OSg1l87clBxPeVMzCoflOwHGy7IehgPeK uqYxq3mgDP8s4GaueqmKvT5qGc6OcIywSqvtWqUmJQDMH6+Tookz/tRXRkFz0YFMlpkh 6xNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Rdb5CyqeVz87BXb6WhLJ5Jb1QcugWr1f3khKG5wF6xA=; b=DwlBYrCWdTQfixyBzZ91deX45s55Qumrd0ASzz4V3lmUQ/nSRnvXkVewSb6P1x0H3o oNEkZMURieajQcLczKrkKAQ0WE+gCpfhtHhrMvqoksteXnfNw/g8VA18UX23bvt7wOYl 545VTxV6fgUHlZWYTsmkXzzfsGONf1bMUb3cGdythBbwBQ9taFZqOKaYG2mejk6EskHc UDp/+553jAyGX16B3LgdoUg0NEOkUik2ysinpzYZ4Q3LK2v2qYlxkU/FCyR6r9y35pqd vKJTVnlOQpJL2dKfh5/kTruA1nn2a3Cwi0TbWYtz1hhuLQEZcEsee94BnWHb/8c7LaCX AJtw== X-Gm-Message-State: APf1xPDl/gcRfwFL18TEZK67r17vMznA+h1Dm1U8j50pDG2J0Yo06XKQ 4PeATnn2kR/HJdVLzRpY2L3Egz7KKxQjrO8TnTPRTQ== X-Google-Smtp-Source: AH8x227ZFrq9ON8NW+bVQE+rt/Pm+KpIwSnj+4ImVcbVV/3GyVNZGCxBYUThUqiQ5oRpZ5TcEpkHf+KjinJaxMi08dI= X-Received: by 10.200.54.100 with SMTP id n33mr2393534qtb.271.1518744127695; Thu, 15 Feb 2018 17:22:07 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.80.241 with HTTP; Thu, 15 Feb 2018 17:22:07 -0800 (PST) X-Originating-IP: [203.99.129.1] In-Reply-To: <30cfa8a3-76e4-73ac-3b2a-2d64ed24aef8@zyxst.net> References: <30cfa8a3-76e4-73ac-3b2a-2d64ed24aef8@zyxst.net> From: Jonathan Chen Date: Fri, 16 Feb 2018 14:22:07 +1300 Message-ID: Subject: Re: is it possible to chroot into arm64 from amd64? To: tech-lists Cc: freebsd-virtualization@freebsd.org, freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Feb 2018 01:22:10 -0000 On 16 February 2018 at 14:00, tech-lists wrote: > Hi, > > Is it possible to have on an amd64 system, a chrooted or jailed arm64 > environment? It's possible to cross-compile an arm6 environment on > amd64 and then chroot into the arm6 environment - is the same possible > for arm64? I can cross-compile arm64 like this: > > # SYSROOT=/crossbuild/arm64 > # cd /usr/src > # make -j32 TARGET=arm64 TARGET_ARCH=aarch64 buildworld > # make DESTDIR=$SYSROOT TARGET=arm64 TARGET_ARCH=aarch64 installworld > > # make DESTDIR=$SYSROOT TARGET=arm TARGET_ARCH=armv6 distribution > # cp /usr/local/bin/qemu-aarch64-static $SYSROOT/sbin > > # /usr/sbin/binmiscctl add arm64 --interpreter > "/usr/local/bin/qemu-aarch64" \ [...] You've copied qemu-aarch64-static to $SYSROOT/sbin. binmiscctl(8) should use /sbin/qemu-aarch64-static as its arm64 interpreter in the arm64-chroot. Cheers. -- Jonathan Chen From owner-freebsd-stable@freebsd.org Fri Feb 16 11:25:53 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55DC2F09D9C; Fri, 16 Feb 2018 11:25:53 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D2901743E5; Fri, 16 Feb 2018 11:25:52 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id EEBC420A8C; Fri, 16 Feb 2018 06:25:51 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Fri, 16 Feb 2018 06:25:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=2AXdReon5D2a++aEzy41xM+jETgcD I6Lv8uLxBFlioo=; b=SvcCGlsE8GFSvMdQDi9qmFGykAm7GOK6FjVLOLOf2G5cx +d0TcnRcAuo+kea2mC8ySXv5A17uUrrM/0S8BB64F8b0LuZqVY6TayF7OXq81qyy ThkTqC4CjjHztbJHXUfpkR/kGofei1hKI1UrguLGuxFJ5yxRgNoVuDaR1cwh/2BT umip5LDfzJbVeVzfFXJcarcrd67H2C2S0ZB84fbIcfolIRtFFWBgaYrL2t+zV95c bsxbYqWvVSxPHMyqZZLUFRlmQs8VF0xBFIGp2lbBYp/RHNwu1lIprdwA2VPvaVAQ hxik7X0uuAqT3Sfzvb78g5kIY8ZCb3DtkPNnrG3EA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=2AXdRe on5D2a++aEzy41xM+jETgcDI6Lv8uLxBFlioo=; b=W8cQBjfcrL17V50PfulBoa 77KYm7Cr/2ErVyvG0CBfXTsVpZQ1WPngje9ha/lwXUFFFSKCgFjyhOjMpEhxduvq X0LSqZQPXZgLm+pJ4gWlvvRhYri04n8aK9Fjqo0k7BEmTZz0q42JlP+3chuFgIMj JK0xB/qSoJmr6eh4AFh8PKBh7Vo31iWqbjlBFScNW0zuzO21QsQvTlJHrbt+I+mL 8BSOFpmgGtQ46VzuO3aeMb42ymZuAfWnQMDPhCR3pxmrBFxYFfYrBCOHikhCHfPK ORCdMbKfWKPEuIUn9lJiRZHxlypx4JidUeQH4XKTMyLzcqClmjkDwf6l0l/of/4A == X-ME-Sender: Received: from desktop.parsley.growveg.org (parsley.growveg.org [82.70.91.97]) by mail.messagingengine.com (Postfix) with ESMTPA id 51B2824550; Fri, 16 Feb 2018 06:25:51 -0500 (EST) Subject: Re: is it possible to chroot into arm64 from amd64? To: Jonathan Chen Cc: freebsd-stable@freebsd.org, freebsd-virtualization@freebsd.org References: <30cfa8a3-76e4-73ac-3b2a-2d64ed24aef8@zyxst.net> From: tech-lists Message-ID: <770ca572-657a-9cbf-4ab5-70804f07e1cc@zyxst.net> Date: Fri, 16 Feb 2018 11:25:50 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Feb 2018 11:25:53 -0000 On 16/02/2018 01:22, Jonathan Chen wrote: > You've copied qemu-aarch64-static to $SYSROOT/sbin. binmiscctl(8) > should use /sbin/qemu-aarch64-static as its arm64 interpreter in the > arm64-chroot. arrrgh, thanks for spotting that. I'll try again -- J. From owner-freebsd-stable@freebsd.org Fri Feb 16 14:15:55 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88D44F176DC; Fri, 16 Feb 2018 14:15:55 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2D4757BB87; Fri, 16 Feb 2018 14:15:55 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 88CEC2082A; Fri, 16 Feb 2018 09:15:54 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Fri, 16 Feb 2018 09:15:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=8qjWpMsb3HlUR0PMZfUhB2BOgyFjn dl+sOwqe63Fzb8=; b=Ks8FJx47nK5cm55Jf4EUh7TK23Y2HIqByycJXhyp8rwZl 1PYRZex6W+1EmY/zRhazctFKFLuwIf8Q+/etiJhIpY51o9qrrsLWwPhQR4/bJybh UOaRDsLSXhWn60yA8rqt6cRj7XKDYhBWAaxBzmUaWfq5pFB1eS23zFFmJpTOdMAw 4y75raqbP/Zb+Y9xZ334qjaeI7qmPLrxIpw9cUsy3dHwy86pKRFlpfA1CBXSpOil B1wm9FJeF/kpkmTqOQh8H7Hvt4fgBGX7FrqbaoM+eeAsIauH0N76NC/j1wCk5WVb Zbvvk6mXJ8saH9VVGQSeO20KlB160Tb21hlRWsx/A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=8qjWpM sb3HlUR0PMZfUhB2BOgyFjndl+sOwqe63Fzb8=; b=cPSRu0foPhjSGIHnbL8iD/ vC5j8I62yKk3h3sa8/GKvzjpe42YsBWVPDuaKkXCndWH8gkzEKKKSLaqZZp/bKMT zjuW0ePmI3CRh7OqJPz3angL7u+wZReW6PXThtQ6QgqnEF+iiUUpzjUfSbMpAreL kFOAc9sMve+KoEIMIspooo4cyej+QJ73M9Tbjg1zBzDgPtRD4H0LVGUz+U8jaPqy AqfBTvmJ47gkM/BWlGUhQl3ABG5V8HKyesyP6MjzBH2PztTNd7uq36gkCihMt9K7 suyW1yQPU6AB9KfnKj87X6WxOp+KSJb6wzUpF/BYi6nDM2m8NIl7NL7IFBIsY4qw == X-ME-Sender: Received: from desktop.parsley.growveg.org (parsley.growveg.org [82.70.91.97]) by mail.messagingengine.com (Postfix) with ESMTPA id E361C24550; Fri, 16 Feb 2018 09:15:53 -0500 (EST) Subject: Re: is it possible to chroot into arm64 from amd64? To: =?UTF-8?Q?Mika=c3=abl_Urankar?= References: <30cfa8a3-76e4-73ac-3b2a-2d64ed24aef8@zyxst.net> Cc: freebsd-virtualization@freebsd.org, freebsd-stable@freebsd.org From: tech-lists Message-ID: Date: Fri, 16 Feb 2018 14:15:52 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Feb 2018 14:15:55 -0000 On 16/02/2018 08:21, Mikaël Urankar wrote: > Put  /usr/local/bin/qemu-aarch64-static in > $SYSROOT/usr/local/bin/qemu-aarch64-static > you can use "service qemu_user_static start" to add the interpreter That did it! :D many thanks. I had a bit of a problem with ldconfig but it seems ok so far. Here's what I did from start to finish in case anyone else is interested: ### root@sm1:/# chflags -Rf noschg crossbuild/ root@sm1:/# rm -Rf crossbuild/* root@sm1:/# mkdir crossbuild/arm64 root@sm1:/# sh # SYSROOT=/crossbuild/arm64 # cd /usr/src # make -j96 cleandir && make -j96 clean && make -j96 clean # make -j96 TARGET=arm64 TARGET_ARCH=aarch64 buildworld # make DESTDIR=$SYSROOT TARGET=arm64 TARGET_ARCH=aarch64 installworld # make DESTDIR=$SYSROOT TARGET=arm64 TARGET_ARCH=aarch64 distribution # cp /usr/local/bin/qemu-aarch64-static $SYSROOT/usr/local/bin/qemu-aarch64-static cp: /crossbuild/arm64/usr/local/bin/qemu-aarch64-static: No such file or directory # cp /usr/local/bin/qemu-aarch64-static $SYSROOT/usr/local/bin/ cp: directory /crossbuild/arm64/usr/local/bin does not exist # ls -lah $SYSROOT/usr/local total 8 drwxr-xr-x 2 root wheel 512B Feb 16 12:28 . drwxr-xr-x 13 root wheel 512B Feb 16 12:28 .. # mkdir $SYSROOT/usr/local/bin # cp /usr/local/bin/qemu-aarch64-static $SYSROOT/usr/local/bin/ # service qemu_user_static start Cannot 'start' qemu_user_static. Set qemu_user_static_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'. # echo "qemu_user_static_enable="YES"" >>/etc/rc.conf # service qemu_user_static start # mount -t devfs devfs ${DESTDIR}/dev # chroot -u root $SYSROOT root@sm1:/ # root@sm1:~ # file /usr/bin/file /usr/bin/file: ELF 64-bit LSB executable, ARM aarch64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.1 (1101509), FreeBSD-style, stripped root@sm1:~ # service start ldconfig ps: empty file: Invalid argument start does not exist in /etc/rc.d or the local startup directories (/usr/local/etc/rc.d), or is not executable root@sm1:~ # which ldconfig /sbin/ldconfig root@sm1:~ # /sbin/ldconfig root@sm1:~ # root@sm1:~ # exit # -- J. From owner-freebsd-stable@freebsd.org Fri Feb 16 20:35:53 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39455F0FC9B for ; Fri, 16 Feb 2018 20:35:53 +0000 (UTC) (envelope-from david.boyd49@twc.com) Received: from dnvrco-cmomta01.email.rr.com (dnvrco-outbound-snat.email.rr.com [107.14.73.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BFE3170BA6 for ; Fri, 16 Feb 2018 20:35:52 +0000 (UTC) (envelope-from david.boyd49@twc.com) Received: from bashful.bsd1.net ([74.138.140.144]) by cmsmtp with ESMTPA id mmjOeIwHlIEdXmmjRekPvs; Fri, 16 Feb 2018 20:35:46 +0000 Message-ID: <1518813341.8392.2.camel@twc.com> Subject: VM image for 11.1-STABLE exhibiting same problem as 12.0-CURRENT. MFC? From: David Boyd To: freebsd-stable@freebsd.org Date: Fri, 16 Feb 2018 15:35:41 -0500 X-Mailer: Evolution 3.22.6 (3.22.6-10.el7) Mime-Version: 1.0 X-CMAE-Envelope: MS4wfAVPcfmO8E2Ng/ar2x6o/3x4BY6MIBdUABl5QmimFPcMyBTD0HlSSLE3Mgc13UpjYRubwItIFSWa6NKNR9Q0YREyYr5cVqujgYB+s4MCYOUI/mzb7oBZ 1Hq1nva8nncsEteJqHPQtFqc4AgahlicfnfiwzMuhpfRUY0q991TKjOihvrfNvr/VXCUsoot2MjPmw== Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Feb 2018 20:35:53 -0000 VM image for 11.1-STABLE began exhibiting same dislike for USB 3.0 ports as problem reported in PR 225794 for 12.0-CURRENT. Prior to FreeBSD-11.1-STABLE-amd64-20180215-r329320.vmdk.xz no such problem was observed. From owner-freebsd-stable@freebsd.org Sat Feb 17 09:09:28 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 884F7F1DCD1 for ; Sat, 17 Feb 2018 09:09:28 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mx2.catspoiler.org (mx2.catspoiler.org [IPv6:2607:f740:16::d18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "amnesiac", Issuer "amnesiac" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2760F6FDD5 for ; Sat, 17 Feb 2018 09:09:28 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org ([76.212.85.177]) by mx2.catspoiler.org (8.15.2/8.15.2) with ESMTPS id w1H9A7km081799 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Sat, 17 Feb 2018 09:10:08 GMT (envelope-from truckman@FreeBSD.org) Received: from mousie.catspoiler.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTPS id w1H7rrEL099052 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 17 Feb 2018 01:09:19 -0800 (PST) (envelope-from truckman@FreeBSD.org) Date: Sat, 17 Feb 2018 01:09:19 -0800 (PST) From: Don Lewis Subject: Re: package building performance (was: Re: FreeBSD on AMD Epyc boards) To: Mark Linimon cc: FreeBSD-STABLE Mailing List In-Reply-To: <20180214111024.GA9330@lonesome.com> Message-ID: References: <8037bf98-acc8-6981-d25b-3b58330dbd33@sentex.net> <20180214081553.GF89804@home.opsec.eu> <20180214111024.GA9330@lonesome.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=us-ascii Content-Disposition: INLINE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2018 09:09:28 -0000 On 14 Feb, Mark Linimon wrote: > On Wed, Feb 14, 2018 at 09:15:53AM +0100, Kurt Jaeger wrote: >> On the plus side: 16+16 cores, on the minus: A low CPU tact of 2.2 GHz. >> Would a box like this be better for a package build host instead of 4+4 cores >> with 3.x GHz ? > > In my experience, "it depends". > > I think that above a certain number of cores, I/O will dominate. I _think_; > I have never done any metrics on any of this. > > The dominant term of the equation is, as you might guess, RAM. Previous > experience suggests that you need at least 2GB per build. By default, > nbuilds is set equal to ncores. Less than 2GB-per and you're going to be > unhappy. > > (It's true that for modern systems, where large amounts of RAM are standard, > that this is probably no longer a concern.) > > Put it this way: with 4 cores and 16GB and netbooting (7GB of which was > devoted to md(4)), I was having lots of problems on powerpc64. The same > machine with 64GB gives me no problems. > > My guess is that after RAM, there is I/O, ncores, and speed. But I'm just > speculating. I've been configuring 4 GB per builder, so on my 8-core 16-thread Ryzen machine, that means 64 GB of RAM. I also set USE_TMPS to "wrkdir data localbase" in poudriere.conf, so I'm leaning pretty heavily on RAM. I do figure that zfs clone is more efficient than tmpfs for the builder jails. With this configuration, building my default set of ports is pretty much CPU-bound. When it starts building the the larger ports that need a lot of space for WRKDIR, like openoffice-4, openoffice-devel, libreoffice, chromium, etc. the machine does end up using a lot of swap space, but it is mostly dead data from the wrkdirs, so generally there isn't a lot of paging activity. I also have ALLOW_MAKE_JOBS=yes to load up the CPUs a bit more, though I did get the best results with MAKE_JOBS_NUMBER=7 building my default port set on this machine. The hard drive is a fairly old WD Green that I removed from one of my other machines, and it is plenty fast enough to keep CPU idle % at or near zero most of the time during the build run. I did just try out "poudriere bulk -a" on this machine to build ports for 11.1-RELEASE amd64 and got these results: [111amd64-default] [2018-02-14_23h40m24s] [committing:] Queued: 29787 Built: 29277 Failed: 59 Skipped: 112 Ignored: 339 Tobuild: 0 Time: 47:39:48 I did notice some periods of high idle CPU during this run, but a lot of that was due to a bunch of the builders in the fetch state at the same time. Without that, the runtime would have been lower. On the other hand, some ports failed due to a gmake issue, and others looked like they failed due to having problems with ALLOW_MAKE_JOBS=yes. The runtime would have been higher without those problems. As far as Epyc goes, I think the larger core count would win. A lot depends on how effective cache is for this workload, so it would be interesting to plot poudriere run time vs. clock speed. If cache misses dominate execution time, then lowering the clock speed would not hurt that much. Something important to keep in mind with Threadripper and Epync is NUMA. For best results, all of the memory channels should be used and the work should be distributed so that the processes on each core primarily access RAM local to that CPU die. If this isn't the case then the infinity fabric that connects all of the CPU die will be the bottleneck. The lower core clock speed on Epyc lessens that penalty, but it is still something to be avoided if possible. Something else to consider is price/performance. If you want to build packages for four OS/arch combinations, then doing it in parallel on four Ryzen machines is likely to be both cheaper and faster than doing the same builds sequentially on an Epyc machine with 4x the core count and RAM. It is unfortunate that there don't seem to be any server-grade Ryzen motherboards. They all seem to be gamer boards with a lot of unnecessary bling. From owner-freebsd-stable@freebsd.org Sat Feb 17 13:40:16 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5340F0E978 for ; Sat, 17 Feb 2018 13:40:16 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 056987E8F7; Sat, 17 Feb 2018 13:40:14 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Sat, 17 Feb 2018 14:40:13 +0100 Authentication-Results: connect.ultra-secure.de; iprev=pass; auth=pass (plain); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 217.71.83.52 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=217.71.83.52; helo=[192.168.1.200]; envelope-from= Received: from [192.168.1.200] (217-071-083-052.ip-tech.ch [217.71.83.52]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 30DFADC0-BCCD-4DAB-9B2D-B258D738CE45.1 envelope-from (authenticated bits=0); Sat, 17 Feb 2018 14:40:11 +0100 From: Rainer Duffner Message-Id: <2FD2F807-C0A9-4610-9781-477ACF0B6D6F@ultra-secure.de> Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: package building performance (was: Re: FreeBSD on AMD Epyc boards) Date: Sat, 17 Feb 2018 14:40:07 +0100 In-Reply-To: Cc: freebsd-stable@freebsd.org To: Don Lewis References: <8037bf98-acc8-6981-d25b-3b58330dbd33@sentex.net> <20180214081553.GF89804@home.opsec.eu> <20180214111024.GA9330@lonesome.com> X-Mailer: Apple Mail (2.3445.5.20) X-Haraka-GeoIP: EU, CH, 451km X-Haraka-ASN: 24951 X-Haraka-GeoIP-Received: X-Haraka-ASN: 24951 217.71.80.0/20 X-Haraka-ASN-CYMRU: asn=24951 net=217.71.80.0/20 country=CH assignor=ripencc date=2003-08-07 X-Haraka-FCrDNS: 217-071-083-052.ip-tech.ch X-Haraka-p0f: os="Mac OS X " link_type="DSL" distance=12 total_conn=1 shared_ip=N X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HTML_MESSAGE autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 4953, bad: 1, connections: 5433, history: 4952, asn_score: 670, asn_connections: 705, asn_good: 670, asn_bad: 0, pass:asn, asn_all_good, relaying Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2018 13:40:17 -0000 > Am 17.02.2018 um 10:09 schrieb Don Lewis : >=20 > It is unfortunate that there don't seem to be any server-grade Ryzen > motherboards. They all seem to be gamer boards with a lot of > unnecessary bling. That=E2=80=99s because few people use servers to build packages. Increasingly, all the other things related to a server are becoming = important (fast memory, fast networking, fast I/O) and because = everything else is expensive, it=E2=80=99s simply not economical to = skimp on the CPU when everything else (SSD, 40GB switch ports, rack = space etc.pp.) costs the same. From owner-freebsd-stable@freebsd.org Sat Feb 17 19:47:29 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B81ABF08434 for ; Sat, 17 Feb 2018 19:47:29 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from resqmta-po-06v.sys.comcast.net (resqmta-po-06v.sys.comcast.net [IPv6:2001:558:fe16:19:96:114:154:165]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "resqmta-po-01v.sys.comcast.net", Issuer "COMODO RSA Organization Validation Secure Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B72D68B92 for ; Sat, 17 Feb 2018 19:47:29 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from resomta-po-10v.sys.comcast.net ([96.114.154.234]) by resqmta-po-06v.sys.comcast.net with ESMTP id n8RqetcRRbv6En8SFezUaN; Sat, 17 Feb 2018 19:47:27 +0000 Received: from koitsu.org ([71.198.44.84]) by resomta-po-10v.sys.comcast.net with SMTP id n8SEeBR8RyFm4n8SFeWBgV; Sat, 17 Feb 2018 19:47:27 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 6C88D1AF18B; Sat, 17 Feb 2018 11:47:26 -0800 (PST) Date: Sat, 17 Feb 2018 11:47:26 -0800 From: Jeremy Chadwick To: freebsd-stable@freebsd.org Subject: stable/11 r329462 - Meltdown/Spectre MFC questions Message-ID: <20180217194726.GA79666@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.7.2 (2016-11-26) X-CMAE-Envelope: MS4wfIN9svmr/SDf4VE3ZEY0L/gOoDmhMseMentSIEUnDQxTZH8DZm6P/2v346MPu5CAsCRbuG8tMJQX1quOCCGX5s609uBvWgf3zCTiDifXI0DLJfoSXNTS AJlrSaJ917+Mq+3320FUssuCBSIuGIeQ5UiPeMqQ4kB4pjIErTfUWDhl X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2018 19:47:29 -0000 Reference: https://svnweb.freebsd.org/base?view=revision&revision=329462 Do the following new loader tunables and sysctls have documentation anywhere? I ask because I wish to know how to turn all of this off (yes you heard me correctly), as not all systems necessarily require mitigation of these flaws. Best I can tell from skimming source: vm.pmap.pti - Description: Page Table Isolation enabled - Loader tunable, visible in sysctl (read-only) - Integer - Default value: depends on CPU model and capabilities, see function pti_get_default(); looks like AMD = 0, any CPU with RDCL_NO capability enabled = 0, else 1 hw.ibrs_active - Description: Indirect Branch Restricted Speculation active - sysctl (read-only) - Integer - Real-time indicator as to if IBRS is currently on or off hw.ibrs_disable - Description: Disable Indirect Branch Restricted Speculation - Loader tunable and sysctl tunable (read-write) - Integer - Default value: unsure. Variable declaration has 1 but SYSCTL_PROC() macro has 0. Thank you. -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@freebsd.org Sat Feb 17 20:19:39 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E61B6F0B2DF for ; Sat, 17 Feb 2018 20:19:38 +0000 (UTC) (envelope-from matt.xtaz@gmail.com) Received: from mail-wr0-x22d.google.com (mail-wr0-x22d.google.com [IPv6:2a00:1450:400c:c0c::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56E3B6A4B2 for ; Sat, 17 Feb 2018 20:19:38 +0000 (UTC) (envelope-from matt.xtaz@gmail.com) Received: by mail-wr0-x22d.google.com with SMTP id k32so5998331wrk.4 for ; Sat, 17 Feb 2018 12:19:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to:user-agent; bh=OzZlipeulWFlwSsG40TUs6qslaxcjtP48yucm1FAF1c=; b=Mnoo9FHZKtyqpmO1+4mLC7HNh87ThMpprRSGe8pIiBcgv+zw0liDaSwbJwQ7mdzCld ZSX4UES+aIM2QPDXdERji8gjO5j/gUoyHVkgLIqgYRguzRAcPKddz+zp7HQ8JPzg7wPU gGD9WvsovmLamz+lrWrqi5r5k0YlNFl1PTVZMxpYM4m8ogT3IcgiM3uvFQPrHfC5FwQq 4NMhAhUR4L/tgbrJw1NxF37FW/sIFigW7YYwLBqW2KYyZBoC/zX8fXbi14yX5j5Pb2kk LYzaCFG+kssX99nhh94qgt4WN7A/XW0WfmJndR3s/E9WSda3JIqtjS+hGo38svVsHLLL hP7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=OzZlipeulWFlwSsG40TUs6qslaxcjtP48yucm1FAF1c=; b=TGi1eiES4y6fE+WvzP/4Kfm7aEHqu9F+iAfgntiAZGXZXo29brbMJ4RZqYJXQFlWWk pS25uMUr7SMVuTb35m2OGOVN/9AJh5X8t0uTEcuomuNi3McWmKyMSB+7lMPndgEH64lA XnUFpU7t6UuCBSHHoVxw8zl4gtepd3n2qkTm9nhmL0PadgAGrLEH/F/NF2pwkSLc0fkj ajYYTw0e32+sMRZk9PxMAIgMyir5Gx8l31lCtSFwx9Ghzm/XSUEa3qjnoKj2MCOsltmq 05EW6ClSYLkG1g0SqdHiYJlHTb0UszaFzKQNb1srJgDBXI9V73ddgljM3zpQbzgRzTb1 t/Dw== X-Gm-Message-State: APf1xPCgY9DQvdLVgPYlrvh37E7HWVMVt32uhYY3Ur2gRTd+1eUGwKxz dKbbdtwfpn277jim5erduo/L5Ni8 X-Google-Smtp-Source: AH8x224cnj/W7ugHyaX4qlNaZ/4elt5pVVQtPGqiKUzZwZsb/6cUQGDSU1ec5D969nEyaM+jzkYiow== X-Received: by 10.223.168.111 with SMTP id l102mr9699447wrc.84.1518898777367; Sat, 17 Feb 2018 12:19:37 -0800 (PST) Received: from gmail.com (tao.xtaz.uk. [2001:8b0:fe33::10]) by smtp.gmail.com with ESMTPSA id m191sm16239176wma.42.2018.02.17.12.19.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 17 Feb 2018 12:19:37 -0800 (PST) Date: Sat, 17 Feb 2018 20:19:34 +0000 From: Matt Smith To: Jeremy Chadwick Cc: freebsd-stable@freebsd.org Subject: Re: stable/11 r329462 - Meltdown/Spectre MFC questions Message-ID: <20180217201934.GA51895@gmail.com> Mail-Followup-To: Matt Smith , Jeremy Chadwick , freebsd-stable@freebsd.org References: <20180217194726.GA79666@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <20180217194726.GA79666@icarus.home.lan> User-Agent: Mutt/1.9.3 (2018-01-21) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2018 20:19:39 -0000 On Feb 17 11:47, Jeremy Chadwick wrote: >Reference: https://svnweb.freebsd.org/base?view=revision&revision=329462 > >Do the following new loader tunables and sysctls have documentation >anywhere? I ask because I wish to know how to turn all of this off (yes >you heard me correctly), as not all systems necessarily require >mitigation of these flaws. > +1. I have an Intel Atom D525 "Pineview" which I'm led to believe doesn't have these flaws and therefore unless it's detected and disabled automatically I too would like to have documentation on how to view the current status, and disable it as required. And thank you for pointing this out. I can now just wait a while to see what comes along rather than accidentally upgrading it and killing the already really slow performance. -- Matt From owner-freebsd-stable@freebsd.org Sat Feb 17 20:38:10 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F756F0CDAF for ; Sat, 17 Feb 2018 20:38:10 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (unknown [IPv6:2a02:b90:3002:411::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 059466B3F7 for ; Sat, 17 Feb 2018 20:38:09 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from cpc73666-dals20-2-0-cust303.20-2.cable.virginm.net ([82.47.237.48] helo=foula.drayhouse.twisted.org.uk) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.89 (FreeBSD)) (envelope-from ) id 1en9FG-000DOq-71 for freebsd-stable@freebsd.org; Sat, 17 Feb 2018 20:38:06 +0000 Subject: Re: stable/11 r329462 - Meltdown/Spectre MFC questions To: freebsd-stable@freebsd.org References: <20180217194726.GA79666@icarus.home.lan> <20180217201934.GA51895@gmail.com> From: Pete French Message-ID: Date: Sat, 17 Feb 2018 20:38:06 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180217201934.GA51895@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2018 20:38:10 -0000 On 17/02/2018 20:19, Matt Smith wrote: > And thank you for pointing this out. I can now just wait a while to see > what comes along rather than accidentally upgrading it and killing the > already really slow performance. > I was just looking at this too, and wondering what (if any) the performance impact is on FreeBSD. I had a quick google to see if I could fine anything on current about it, but no luck. Anyone done any measurements ? -pete. From owner-freebsd-stable@freebsd.org Sat Feb 17 23:36:29 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11DF9F1BB5D for ; Sat, 17 Feb 2018 23:36:29 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-vk0-x22c.google.com (mail-vk0-x22c.google.com [IPv6:2607:f8b0:400c:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82786731A3 for ; Sat, 17 Feb 2018 23:36:28 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-vk0-x22c.google.com with SMTP id z190so1817245vkg.1 for ; Sat, 17 Feb 2018 15:36:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=1E3ZJpcjixxSb9srg9gUNGlTpHFY/hePWUcfxI3sLLI=; b=Hsa/Z6dTH5nz3+yHFI6UrMl3UL/nmIeLt6hMwTERXhl8TN3j4pYGCjqC154abz6qy6 UobPvp5Q7mkv16bEdmfReoHRFJj2AHniFWcpOV4NDGRfjo9ROQ1Uq2u6HPvQhkMASYB9 qUuaKPefMEX07lP/ypt8rmtLoNxC+lBfLB42TTOwAevwzE2prQHeDMhfpqosT7pzEaGu w1d7Upeas/pWnxlibr/DzwhiI9YH6PcjZGef6Q9odFV5xUBS3og5hBJ+Hw2juTRLk5+v PPJFJ0brCujE0fD1iC0+EFG5CEGbyweN4uSYeFjc5P/J2E/8mzqaFkhN0nA3ES0GAg16 lxlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=1E3ZJpcjixxSb9srg9gUNGlTpHFY/hePWUcfxI3sLLI=; b=CfKMZARyCTvWau2Oy846K3wg2bhCmafS0jbvzDmiG9ngIwIxgYe61x+ZsG4ARnQ6cT YyztLDQ0CdXYFnX+mWGT1t8P+3mfVPatPlajcZXGyDkEiPUatPtDmYYL8KkRrzTbJH2A NBZdZHscnKGn92Ooa3KdXf55/2Gwf8SDG9UrKfkvU0QGA/b196xmFIf+5boHjjBt4aAM XIyUcsxqK2lZl/e+ntuTWK8r/PA3crEOs8KDovFG9gZMSOuAB0NA0M4vW8Q/A7+NM39q fPr13QHTARx6jEKDZzXxvYaax/3pqcmRRK+DLWRqA11BOPPC4UPHLupMDhUztt9rukkc vV7w== X-Gm-Message-State: APf1xPB35FnXA3pr9JSkcC/5ULQX1h3E3BaAtBZIZtTQx5IGZhos5zFx pHUdFsaxbpLbbUaiekUKfB5m3gq2vGXZ671zlNNTsOlf X-Google-Smtp-Source: AH8x224G+hzKmVjqOdp3yudR2DYRTP/GEi86ZLxt6vCXI6ZWyypLJ4SHjeikxbOuil5ytk681QKchXTUQA024xk1xoM= X-Received: by 10.31.99.130 with SMTP id x124mr7211858vkb.155.1518910587708; Sat, 17 Feb 2018 15:36:27 -0800 (PST) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.39.199 with HTTP; Sat, 17 Feb 2018 15:36:26 -0800 (PST) In-Reply-To: References: <20180217194726.GA79666@icarus.home.lan> <20180217201934.GA51895@gmail.com> From: Kevin Oberman Date: Sat, 17 Feb 2018 15:36:26 -0800 X-Google-Sender-Auth: v67lnWyxKsH8_J84nhgpgSbFs8k Message-ID: Subject: Re: stable/11 r329462 - Meltdown/Spectre MFC questions To: Pete French Cc: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2018 23:36:29 -0000 On Sat, Feb 17, 2018 at 12:38 PM, Pete French wrote: > > > On 17/02/2018 20:19, Matt Smith wrote: > > And thank you for pointing this out. I can now just wait a while to see >> what comes along rather than accidentally upgrading it and killing the >> already really slow performance. >> >> I was just looking at this too, and wondering what (if any) the > performance impact is on FreeBSD. I had a quick google to see if I could > fine anything on current about it, but no luck. Anyone done any > measurements ? > > -pete. Well, I would also like a bit of information on this. When I get a moment I'll see if I can find anything on this on the current@ archive. Looks like all are loadables, so need to be defined in /boot/loader.conf and can only be changed on a boot with the exception of hw.ibrs_disable which is settable either at boot or via sysctl. Looks like that is the one to twiddle to check on impact. I'm currently updating my stable system as this hit the tree after my morning updates. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683