From owner-freebsd-stable@freebsd.org Sun Sep 29 15:35:57 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 901BB12360C for ; Sun, 29 Sep 2019 15:35:57 +0000 (UTC) (envelope-from john@spikefishsolutions.com) Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46h8kh0sCSz4dvL for ; Sun, 29 Sep 2019 15:35:55 +0000 (UTC) (envelope-from john@spikefishsolutions.com) Received: by mail-lf1-x132.google.com with SMTP id w67so5184831lff.4 for ; Sun, 29 Sep 2019 08:35:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spikefishsolutions.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=zcBNVsOm7iNWAOs7xK2TWQZBtHuJqkBOY9GaWzTSktU=; b=D2XC9hvo3CtVTGcuFRy0DcEXvdxyYnT9fJxxFePEx7rP25vowCVjGovJKS1jaLMPjU 6QSWEB+B0EXFgxvq0aDcZXRJZ5ZWNmOu6qvBKurIJi2fVdd/pM99vYNPFByyoZs9ZoPD XtIyNc+wYR2bI9ETuuaqLY/xmS9z+wDQGbZ7c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=zcBNVsOm7iNWAOs7xK2TWQZBtHuJqkBOY9GaWzTSktU=; b=P/FyODntgHhS6IOfMfilUO1gfrbeyXinm5Z0Mf+UCHiz//PtLOIuKnuvtyu+rWsAUj niMVnCSl+KlMaZy7gYE1/Me832PTxcRVRQDEQDbaKR7lhSYZ2c5s/fC3nYupfrvh8+6J EHZITOK+52TSeJvkO9g9LMbCQ7KB/P7PH5b3NbTm7iqUPlGnoHPgzlnsNEbU/7/2/0Vk NM2UyuXgCtb9lmnjmCtDAecaYFrBXHXlBtJPSuNB9ZyjbywZYEzJBAbhRZEpDYi4n4G+ JCfpWtKyk3iGUDGetAvIcsNo4Kpf9qpSReZSU/MgsJnhfCfi0bD2I9ZM3OB1Tf+isNe1 lS5Q== X-Gm-Message-State: APjAAAUZLbGPzGcZB1H9uthl9buSzs5kyXm/TU5fKm/f67yncxv+QRnQ n+DXSZBvJx5dlGxlin/sVAX1qgwsR8eb5K6zOD7gyYH1vT8= X-Google-Smtp-Source: APXvYqyrW0vlGEMdrC+JcTL5urcjlpUaqh/v3A/ueyzCuFmFwgyqqVIi3qt2LGPvPUVuAjxZ3LYlyiGbSBRfGE0YWO0= X-Received: by 2002:ac2:46f8:: with SMTP id q24mr8616165lfo.51.1569771353095; Sun, 29 Sep 2019 08:35:53 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: John Fleming Date: Sun, 29 Sep 2019 11:35:41 -0400 Message-ID: Subject: Re: Question about bottle neck in storage To: Warner Losh Cc: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 46h8kh0sCSz4dvL X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=spikefishsolutions.com header.s=google header.b=D2XC9hvo; dmarc=none; spf=pass (mx1.freebsd.org: domain of john@spikefishsolutions.com designates 2a00:1450:4864:20::132 as permitted sender) smtp.mailfrom=john@spikefishsolutions.com X-Spamd-Result: default: False [-5.42 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[spikefishsolutions.com:s=google]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[spikefishsolutions.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[spikefishsolutions.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-2.92)[ip: (-9.48), ipnet: 2a00:1450::/32(-2.90), asn: 15169(-2.17), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Sep 2019 15:35:57 -0000 On Tue, Sep 24, 2019 at 1:09 PM Warner Losh wrote: > > > > On Tue, Sep 24, 2019 at 5:46 PM John Fleming wrote: >> >> Is there anyway to see how busy a SAS/Sata controller is vs disks? I >> have a R720 with 14 Samsung 860 EVOs in it (its a lab server) in raid >> 10 ZFS. >> >> When firing off a dd I (bs=3D1G count=3D10) seems like the disks never g= o >> above %50 busy. I'm trying to figure out if i'm maxing out SATA 3 BW >> or if its something else (like terrible dd options). > > > Two points to consider here. First, NVMe has lots of queues and needs lot= s of concurrent transactions to saturate, so the 50% busy means you are no = where close to saturating the drives. Schedule more I/O too fix that. It's = better to do lots and lots of concurrent DD to different parts of the drive= , or to use fio with the aio kernel option and posixaio I/O scheduling meth= od. > > I use the following script, but often need to increase the number of thre= ads / jobs to saturate. > > ; SSD testing: 128k I/O 64 jobs 32 deep queue > > [global] > direct=3D1 > rw=3Drandread > refill_buffers > norandommap > randrepeat=3D0 > bs=3D128k > ioengine=3Dposixaio > iodepth=3D32 > numjobs=3D64 > runtime=3D60 > group_reporting > thread > > [ssd128k] > I didn't catch what utilty was using that. I started poking around an iozone and bonnie++ BTW these are SATA not nvme. > Second, the system's % busy statistics are misleading. They are the %of t= he time that a command is outstanding on the drive. 100% busy can be a tiny= percentage of the total bandwidth you can get from the drive. > >> >> my setup is Dell R720 with 2 x LSI 9361 cards. Each card is going to a >> dedicated 8 drive board inside the front of the R720. Basically i'm >> just saying its not a single SAS cable to 14 drives. >> >> Don't have cpu info hand.. zeon something. DDR3-1600 (128GB) >> >> Both controllers are in 8x slots running PCIe gen 3. >> >> BTW i'm sure this has been asked a million times but what would be >> some decent benchmark tests while i'm at it? > > > See above... :) > > Warner So my UPS got angry and shut everything down. I figured this was a good change to look at iostats again. This is while the array is being scrubbed. I'm very happy with these numbers! BTW da0 and 8 are OS drives and not raid 10 members. extended device statistics device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b da0 0 0 0.0 0.0 0 0 0 0 0 0 da1 4003 7 505202.5 207.6 0 0 1 0 2 100 da2 3980 10 508980.2 265.5 0 0 0 0 2 100 da3 3904 8 499675.8 183.1 0 0 0 0 2 99 da4 3850 8 488870.5 263.9 0 0 0 0 2 100 da5 4013 11 513640.6 178.8 0 0 1 0 2 100 da6 3851 13 489035.8 286.4 0 0 1 0 2 100 da7 3931 12 503197.6 271.6 0 0 0 0 2 100 da8 0 0 0.0 0.0 0 0 0 0 0 0 da9 4002 8 505164.1 207.6 0 0 1 0 2 100 da10 3981 10 509133.8 265.5 0 0 0 0 2 100 da11 3905 8 499791.0 183.1 0 0 0 0 2 100 da12 3851 9 488985.6 263.9 0 0 0 0 2 100 da13 4012 11 513576.6 178.8 0 0 1 0 2 100 da14 3850 14 488971.8 286.4 0 0 0 0 2 100 da15 3930 12 503108.0 271.6 0 0 0 0 2 100 From owner-freebsd-stable@freebsd.org Sun Sep 29 20:11:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 51A10129363 for ; Sun, 29 Sep 2019 20:11:40 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46hGrp5rd3z3QlB for ; Sun, 29 Sep 2019 20:11:38 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [2a02:b90:3002:411::6] (helo=dilbert.ingresso.co.uk) by constantine.ingresso.co.uk with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92.2 (FreeBSD)) (envelope-from ) id 1iEfXV-000K07-Sk for freebsd-stable@freebsd.org; Sun, 29 Sep 2019 20:11:29 +0000 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.92.2 (FreeBSD)) (envelope-from ) id 1iEfXV-0005xc-Pf for freebsd-stable@freebsd.org; Sun, 29 Sep 2019 21:11:29 +0100 To: freebsd-stable@freebsd.org Subject: Running iperf3 as a server drops all connections to a machine Message-Id: From: Pete French Date: Sun, 29 Sep 2019 21:11:29 +0100 X-Rspamd-Queue-Id: 46hGrp5rd3z3QlB X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-3.54 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:31.24.6.74]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-3.04)[ip: (-9.65), ipnet: 31.24.0.0/21(-4.83), asn: 16082(-0.65), country: GB(-0.08)]; DMARC_POLICY_ALLOW(0.00)[ingresso.co.uk,none]; URIBL_XBL(1.50)[ingresso.co.uk]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Sep 2019 20:11:40 -0000 This is odd - I have six FreeBSD boxes on the local ether here. Five are HP Microservers, one is an AMD Ryzen. They are all running 12.1 r352847 built yesterday - and its one compilation I did which is rsynced out to all the machines, so an identical build. I was running iperf3 -s on one machine and iperf3 -c on another to measure TCp/IP rates as I am experimenting with the RACK version of the stack and using CUBIC as a congetion control algorithm. All goes well between the Intel boxes, and if I use the Ryzen box as a client ot the Intel boxes. But if I run the iperf3 server on the Ryzemn box and use one of the Intel boxes as a client then after a few seconds all the TCP connection on the Ryzen box are dropped. There is nothing in dmesg, no indication that I can find of anything going wrong, and I can immediately ssh back in. On the client side it looks like this: [webadmin@dogbert ~]$ iperf3 -c dilbert Connecting to host dilbert, port 5201 [ 5] local 10.64.50.50 port 41134 connected to 10.64.50.6 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 112 MBytes 938 Mbits/sec 1 78.2 KBytes [ 5] 1.00-2.00 sec 112 MBytes 936 Mbits/sec 2 54.8 KBytes [ 5] 2.00-3.02 sec 14.7 MBytes 122 Mbits/sec 3 1.43 KBytes [ 5] 3.02-4.00 sec 0.00 Bytes 0.00 bits/sec 2 1.43 KBytes [ 5] 4.00-5.02 sec 0.00 Bytes 0.00 bits/sec 1 1.43 KBytes [ 5] 5.02-6.02 sec 0.00 Bytes 0.00 bits/sec 1 1.43 KBytes [ 5] 5.02-6.02 sec 0.00 Bytes 0.00 bits/sec 1 1.43 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-6.02 sec 238 MBytes 332 Mbits/sec 10 sender [ 5] 0.00-6.02 sec 0.00 Bytes 0.00 bits/sec receiver iperf3: error - the server has terminated [webadmin@dogbert ~]$ ...and on the server side, this... petefrench@dilbert ~]$ iperf3 -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 10.64.50.50, port 41133 [ 5] local 10.64.50.6 port 5201 connected to 10.64.50.50 port 41134 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 105 MBytes 883 Mbits/sec [ 5] 1.00-2.00 sec 112 MBytes 936 Mbits/sec [ 5] 2.00-3.01 sec 21.3 MBytes 177 Mbits/sec Connection to dilbert.ingresso.co.uk closed by remote host. Connection to dilbert.ingresso.co.uk closed. This only happens on the Ryzen machine. It happens whether I am using the RACk stack or the default one, and whether I am using CUBIC or the default too. The system is build with CPUTYPE set to 'core2' and some modules skipped, but apart from that nothing special. Its a bit worrying to me that this can happen, especially in userland. Any opinions or things people would ike me to check ? Indeed, can anyone reproduce this ? cheers, -pete. From owner-freebsd-stable@freebsd.org Sun Sep 29 20:30:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9A1061297C5 for ; Sun, 29 Sep 2019 20:30:44 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46hHGp5sjLz3wgx for ; Sun, 29 Sep 2019 20:30:42 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [IPv6:2a03:3100:c:13:0:0:0:5]) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x8TKUSYu013402 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 29 Sep 2019 20:30:29 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: petefrench@ingresso.co.uk Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id x8TKUOQA026748 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 30 Sep 2019 03:30:24 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Running iperf3 as a server drops all connections to a machine To: Pete French , freebsd-stable@freebsd.org References: From: Eugene Grosbein Message-ID: <59a037a5-03df-e143-41e9-0cfe648a3f22@grosbein.net> Date: Mon, 30 Sep 2019 03:30:17 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 46hHGp5sjLz3wgx X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.66 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-1.56)[ip: (-3.98), ipnet: 2a01:4f8::/29(-2.01), asn: 24940(-1.79), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Sep 2019 20:30:44 -0000 30.09.2019 3:11, Pete French wrote: > Its a bit worrying to me that this can happen, especially in userland. > > Any opinions or things people would ike me to check ? netstat -p tcp -ss tcpdump -i $interface -npvs0 icmp or 'tcp[tcpflags] & (tcp-rst) != 0' From owner-freebsd-stable@freebsd.org Tue Oct 1 19:10:59 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A70CB13DE3B for ; Tue, 1 Oct 2019 19:10:59 +0000 (UTC) (envelope-from info_accounts.outlook.com@bitcoin-trading.site) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 46jTPv2vfnz4FRB for ; Tue, 1 Oct 2019 19:10:59 +0000 (UTC) (envelope-from info_accounts.outlook.com@bitcoin-trading.site) Received: by mailman.nyi.freebsd.org (Postfix) id 63A4413DE3A; Tue, 1 Oct 2019 19:10:59 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6363F13DE39 for ; Tue, 1 Oct 2019 19:10:59 +0000 (UTC) (envelope-from info_accounts.outlook.com@bitcoin-trading.site) Received: from server-69-r69.ipv4.per01.ds.network (server-69-r69.ipv4.per01.ds.network [103.67.234.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.ds.network", Issuer "USERTrust RSA Organization Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46jTPs1SbFz4FR6 for ; Tue, 1 Oct 2019 19:10:57 +0000 (UTC) (envelope-from info_accounts.outlook.com@bitcoin-trading.site) Received: from EXCH06-AU.premium.exchange (10.21.24.90) by Edge01-au.internal.dreamscapenetworks.com (10.21.24.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_DHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Wed, 2 Oct 2019 02:56:08 +0800 Received: from EXCH06-AU.premium.exchange (10.21.24.90) by EXCH06-AU.premium.exchange (10.21.24.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_DHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Wed, 2 Oct 2019 02:56:07 +0800 Received: from EXCH06-AU.premium.exchange ([fe80::39b3:9a19:82da:c178]) by EXCH06-AU.premium.exchange ([fe80::39b3:9a19:82da:c178%12]) with mapi id 15.01.1713.004; Wed, 2 Oct 2019 02:56:07 +0800 From: =?utf-8?B?b3V0bG9vay5jb20gwq4=?= To: "stable@freebsd.org" Subject: Account unusual sign-in activity Thread-Topic: Account unusual sign-in activity Thread-Index: AQHVeInUQlu1yqdF/02UXWMG5S2U2g== Date: Tue, 1 Oct 2019 18:55:45 +0000 Message-ID: <1569946398d992a0349e1e1d0a1428cf448340ae00_6357@bitcoin-trading.site> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [52.32.73.207] MIME-Version: 1.0 X-Rspamd-Queue-Id: 46jTPs1SbFz4FR6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=bitcoin-trading.site; spf=pass (mx1.freebsd.org: domain of info_accounts.outlook.com@bitcoin-trading.site designates 103.67.234.225 as permitted sender) smtp.mailfrom=info_accounts.outlook.com@bitcoin-trading.site X-Spamd-Result: default: False [-2.89 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:103.67.234.0/24:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-0.19)[ipnet: 103.67.234.0/23(-4.34), asn: 38719(3.39), country: AU(0.01)]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(-0.50)[bitcoin-trading.site,none]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:38719, ipnet:103.67.234.0/23, country:AU]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Oct 2019 19:10:59 -0000 RGVhciAgc3RhYmxlQGZyZWVic2Qub3JnLA0KDQpZb3VyIEVtYWlsIEFjY291bnQgd2FzIGp1c3Qg c2lnbmVkIGluIG9uIGEgbmV3IFdpbmRvd3MgZGV2aWNlIGZyb20gdGhpcyBJUCAxODUuMjQyLjE4 MC4xMjAuIFlvdSdyZSBnZXR0aW5nIHRoaXMgZW1haWwgdG8gbWFrZSBzdXJlIGl0IHdhcyB5b3Uu DQoNCklmIHlvdSBkb24ndCByZWNvZ25pemUgdGhpcyBhY3Rpdml0eSwgaXQgbWVhbnMgc29tZW9u ZSBlbHNlIG1heSBoYXZlIGFjY2Vzc2VkIHlvdXIgRW1haWwgQWNjb3VudC4NCg0KRm9yIHlvdXIg cHJvdGVjdGlvbiwgcGxlYXNlIHJldmlldyB5b3VyIGFjY291bnQgYWN0aXZpdHkgYmVsb3cuDQoN ClJldmlldyByZWNlbnQgYWN0aXZpdHk8aHR0cHM6Ly9vcGh0YWxtaWNjZW50ZXIuY29tL2ltYWdl cy9zbGlkZXIvb3dkL2luZGV4LnBocD9lbWFpbD1zdGFibGVAZnJlZWJzZC5vcmc+DQoNCk9uY2Ug dGhlIGluZm9ybWF0aW9uIHByb3ZpZGVkIG1hdGNoZXMgd2hhdCBpcyBvbiBvdXIgcmVjb3JkLCB5 b3VyIGFjY291bnQgd2lsbCB3b3JrIGVmZmVjdGl2ZWx5IHByb3RlY3RlZCBhZnRlciB0aGUgdmVy aWZpY2F0aW9uIGlzIHByb2Nlc3NlZC4NClRoYW5rcywNClRoZSBNYWlsIHNlcnZpY2UgdGVhbQ0K From owner-freebsd-stable@freebsd.org Wed Oct 2 16:49:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7214C133998 for ; Wed, 2 Oct 2019 16:49:16 +0000 (UTC) (envelope-from celina.clarke@inforesearchsolutions.com) Received: from IND01-BO1-obe.outbound.protection.outlook.com (mail-eopbgr1390131.outbound.protection.outlook.com [40.107.139.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46k2Cv1GXWz3CD5 for ; Wed, 2 Oct 2019 16:49:14 +0000 (UTC) (envelope-from celina.clarke@inforesearchsolutions.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QeOmQ4FJVPCKwAOsb+dDuX+mHthOn8yBrYnAtM12Heh7EeEeYoA5VaYVhVgTy0T21vVN00d4nEOqh0brtmrUiRAxYlEAjhC4w/E+HLw8SadH/LQMmRagZfBVQh8sW0AC/VH/X89oxSaZ2T06yHJ5nlJrg5uo6quUGBk2cNxsRtqoNk8g/I1Zauo4NPLtM/BrlTEQjSPsLk/GX6yGMs0rzqHH8s6B3+WywyHVcin75z1PzZRTNuh9oTK4Iv1Bz1wO36BMF2ml5U7o51/VgZ7YtKfE7fh1jMQ/sCCBZ0lT9IQU8Xy3K424acrQwaPfQslEOV6GADPVFCEfKR2nRM6aEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MCugwiAP/ZDIN7H8JTasgRODh4ZKHhR2qG/9Tudz2Lw=; b=N5N9RbKl9IdCD0KgqKiXQPi/5+IRVT0Y3EkhnVneB645fK8tGPTQ3lAbtpw1pbvuPUSCbIatnUj+THnZ92fJFiqEbL/mFSzdsybwUD6Y5lbq6r7CublUZCd7BTmsvjEVnmZxWsOVODSdW8NJQsaitkDLhpDkvW0lAQut5/LK/01EfmqYmggRlPp5mym53hohrKkfA2RuMUc8bhcTeX3Z+/McsEcPUekd3dZrYbkToYGacGfI4D+9e9zp20R+QlN7expk8QPTExGW9OIGySVsO0SlLNwaGoarWVy1x5OSSMglc8Kbooh0PpaA/kdsRX5BN8UuHiHN5k9dJKTm8p1gYg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=inforesearchsolutions.com; dmarc=pass action=none header.from=inforesearchsolutions.com; dkim=pass header.d=inforesearchsolutions.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=NETORGFT3453886.onmicrosoft.com; s=selector2-NETORGFT3453886-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MCugwiAP/ZDIN7H8JTasgRODh4ZKHhR2qG/9Tudz2Lw=; b=iAhwWlFFZeNDLgRiTbt522i5ys995x6oXhSlRSeTXSx7VAUYd1P2O4I02EfIPTJ/P3CX1B6jQsLBn8wBqOVERqgTfbbK33KS3J1XegFqd2H3VWfpxnX8O1g7jkMAyUZtXryYBB9eEfDYSf7yaHIByoM7TcU1mxQUSiwIVQSpcbM= Received: from MAXPR01MB3453.INDPRD01.PROD.OUTLOOK.COM (52.134.156.82) by MAXPR01MB3678.INDPRD01.PROD.OUTLOOK.COM (10.255.152.215) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2305.20; Wed, 2 Oct 2019 16:49:11 +0000 Received: from MAXPR01MB3453.INDPRD01.PROD.OUTLOOK.COM ([fe80::30b5:8a85:ec5b:eaaa]) by MAXPR01MB3453.INDPRD01.PROD.OUTLOOK.COM ([fe80::30b5:8a85:ec5b:eaaa%5]) with mapi id 15.20.2305.023; Wed, 2 Oct 2019 16:49:11 +0000 From: Celina Clarke To: "freebsd-stable@freebsd.org" Subject: Ruckus Networks Users Contact List Thread-Topic: Ruckus Networks Users Contact List Thread-Index: AdV5QUOKralf0UmRTQGdVbRTkuo1/g== Date: Wed, 2 Oct 2019 16:48:50 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [49.206.228.230] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e39bcf03-cbff-48a0-5797-08d747587369 x-ms-traffictypediagnostic: MAXPR01MB3678: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-forefront-prvs: 0178184651 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(396003)(136003)(39840400004)(376002)(346002)(366004)(189003)(199004)(71190400001)(7696005)(6916009)(5640700003)(26005)(9686003)(6436002)(54896002)(6306002)(486006)(186003)(55016002)(476003)(44832011)(99286004)(4744005)(74316002)(25786009)(5660300002)(7736002)(2501003)(2906002)(256004)(102836004)(45080400002)(508600001)(33656002)(6506007)(8936002)(14454004)(52536014)(8676002)(2351001)(9326002)(81166006)(81156014)(316002)(66476007)(66946007)(76116006)(626008)(55236004)(71200400001)(66066001)(86362001)(66556008)(64756008)(66446008)(6666004)(790700001)(3846002)(6116002)(26710200004); DIR:OUT; SFP:1102; SCL:1; SRVR:MAXPR01MB3678; H:MAXPR01MB3453.INDPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: inforesearchsolutions.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: oDd/fQhC6pKW+8SrL27lZrQ3zUN//L041JLszAx7eGYlVzrD/jCGgKMOmEGYnNXiG/kUPyRbmLcM5cnqgkHkf9z4TC3t1O/uZEyXycQtxdbaOlVDCT/zxE8ozByUg8UJnewtpjSyzq64/TJeReL0SSQb5ea2k9OQohFdNxGS6ZhEkDWSmvfHUsGuBxhjaqNiZ/ojGyrK07mNdAJtbbPszwBE3TEVWjs0Y7uLD+ligmN7Yt7WfI5CQVKuAMy57mD82LX3WhEuy2ZzUmDfCo2boYzKnWyXlDDT2bo3UJvgmRfaJdG9e+I2wCGibz7P9DrtLmoHgDH0Z789juhcPMNZZa15IIUfiA0LjrowqpHBqc3bgweGp6IuwPYKh5he31970yuTXsr4igcpcnZguVv2Za7NN7QChLomF3vmUjA1REw= x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: inforesearchsolutions.com X-MS-Exchange-CrossTenant-Network-Message-Id: e39bcf03-cbff-48a0-5797-08d747587369 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Oct 2019 16:48:50.8411 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 13babfe8-560f-48bf-afee-d8a0e58d7ebc X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: t0cjXE70vZqkQJ6M1djjAOPe9wMsW28GRZhvsPXN5TEmvboWel9BYSu+mi4RTXb+0wVhHt2GsY6ioXHqY0PyMWSI2wFpns2m1tGIBr4yOaLb70kWTgDtLQcTHZEGhIPV X-MS-Exchange-Transport-CrossTenantHeadersStamped: MAXPR01MB3678 X-Rspamd-Queue-Id: 46k2Cv1GXWz3CD5 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=NETORGFT3453886.onmicrosoft.com header.s=selector2-NETORGFT3453886-onmicrosoft-com header.b=iAhwWlFF; dmarc=none; spf=pass (mx1.freebsd.org: domain of celina.clarke@inforesearchsolutions.com designates 40.107.139.131 as permitted sender) smtp.mailfrom=celina.clarke@inforesearchsolutions.com X-Spamd-Result: default: False [-4.79 / 15.00]; IP_SCORE(-1.29)[ipnet: 40.64.0.0/10(-3.84), asn: 8075(-2.56), country: US(-0.05)]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[NETORGFT3453886.onmicrosoft.com:s=selector2-NETORGFT3453886-onmicrosoft-com]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[inforesearchsolutions.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[NETORGFT3453886.onmicrosoft.com:+]; RCVD_IN_DNSWL_NONE(0.00)[131.139.107.40.list.dnswl.org : 127.0.3.0]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.64.0.0/10, country:US]; ARC_ALLOW(-1.00)[i=1] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Oct 2019 16:49:16 -0000 Hello, I would like to know if you are interested in acquiring Ruckus Networks Use= rs Contact List. Information fields: Names, Title, Email, Phone, Company Name, Company URL, = Company physical address, SIC Code, Industry, Company Size (Revenue and Emp= loyee), LinkedIn profile link and kind of technology using/solution in plac= e. Please let me know your target geography so that I can provide you with the= counts, pricing and more information for reference. Await your response! Regards, Celina Clarke Marketing Executive If you do not wish to receive further emails, please respond with Remove me= . From owner-freebsd-stable@freebsd.org Thu Oct 3 10:49:20 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D0A9E130F80 for ; Thu, 3 Oct 2019 10:49:20 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46kVB7689Bz3wqF for ; Thu, 3 Oct 2019 10:49:19 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [2a02:b90:3002:411::6] (helo=dilbert.ingresso.co.uk) by constantine.ingresso.co.uk with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92.2 (FreeBSD)) (envelope-from ) id 1iFyfW-000Kow-Uq; Thu, 03 Oct 2019 10:49:10 +0000 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.92.3 (FreeBSD)) (envelope-from ) id 1iFyfW-0000uH-Q8; Thu, 03 Oct 2019 11:49:10 +0100 To: eugen@grosbein.net, freebsd-stable@freebsd.org, petefrench@ingresso.co.uk Subject: Re: Running iperf3 as a server drops all connections to a machine In-Reply-To: <59a037a5-03df-e143-41e9-0cfe648a3f22@grosbein.net> Message-Id: From: Pete French Date: Thu, 03 Oct 2019 11:49:10 +0100 X-Rspamd-Queue-Id: 46kVB7689Bz3wqF X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-5.87 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:31.24.6.74]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; IP_SCORE(-3.07)[ip: (-9.68), ipnet: 31.24.0.0/21(-4.84), asn: 16082(-0.75), country: GB(-0.08)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Oct 2019 10:49:20 -0000 > > Any opinions or things people would ike me to check ? > > netstat -p tcp -ss > tcpdump -i $interface -npvs0 icmp or 'tcp[tcpflags] & (tcp-rst) !=3D 0' I havent had a chnace to look at this for a couple of days, but I thought I would give it mor testing now (doing the above). Updated to the latest -STABLE as I always do before checking things, and the problem has now gone away. Theres nothing I can see in the commits over the last few days which touches this - the only thign I can see is the changes to ixgbe, and mine is igb so it cant be that. The test I did at the time showed the network rate slowing to zero, but staying up if I didnt use cubic - the disconnect only happened using cubic, if thats a useful data point. I will try and find time to go back to the older ernel and see if it still does it. If not then I guess I can say it was a hardware fault, which seems to have fixed itself, but it didnt feel like it at the time. So, thats the update - will follow up to this thread if I can get it to happen again, and will make the suggested tests. (I assume you mean to run that inside 'screen' on the server side, yes?) thanks, -pete. From owner-freebsd-stable@freebsd.org Thu Oct 3 11:54:01 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0D1E013255A for ; Thu, 3 Oct 2019 11:54:01 +0000 (UTC) (envelope-from freebsd-stable=freebsd.org@mail51x10.eol-001.com) Received: from mail51x10.eol-001.com (mail51x10.eol-001.com [51.38.225.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "serveur-56.localdomain", Issuer "serveur-56.localdomain" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46kWck4yCzz412h for ; Thu, 3 Oct 2019 11:53:58 +0000 (UTC) (envelope-from freebsd-stable=freebsd.org@mail51x10.eol-001.com) Date: Thu, 3 Oct 2019 11:53:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eol-001.com; s=trust; t=1570103630; bh=WuUeXcmqxP4udtmZ6V6ZYADQKqY1pehpTeVT9a7gyTQ=; h=Date:To:From:Reply-To:Subject:Message-ID:List-Unsubscribe: List-Unsubscribe-Post:List-ID:MIME-Version:Content-Type; b=HsOoPlUeB7YJ4xf1+xdbKQlVMC3A2iSYXKN3trodtLifrPZj8dgCzlaHvY2RLfFrp bGf4FdFUmQ3f/NHNiWmBiQDgR5XhAFIWfA65CGZQQojTE5alxOZ+hqV3RSjSLDt1pl 1QQ5ae7aOVYVqtnpTMFHeAWlL5/oXvWEvbZsgWl8= To: freebsd-stable@freebsd.org From: Grossiste Super-Aliments Reply-To: contact@grossiste-superfood.ch Subject: =?UTF-8?Q?La_baie_d'a=C3=A7a=C3=AD_arrive_en_Suisse!?= Message-ID: X-Mailer: eTarget Dream Clouder - **CIDB6dfrpE8t6N3Kucn** X-eTa-campaign: cxheX8iW X-eTa-User: LRdDHgWUbChjvaP6 List-Unsubscribe-Post: List-Unsubscribe=One-Click MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; s=trust; d=grossiste-superfood.ch; h=From:Reply-To:To:Date:Message-ID:List-ID:List-Unsubscribe-Post:List-Unsubscribe:Subject:Content-Type:MIME-Version; i=contact@grossiste-superfood.ch; bh=5A/u7ZOiC1CS8PLierclPJAbuWafHJAZk67lCURG62E=; b=ptb2gG27Lu6JSFVCJ6ibc9BF2Z/5ANaaHwfbGZK4pw9fM2fdvBQ6us37j7cOEPttUhNctSfuTEJnMczFMoeQD730DU1mZWxvbBGLYN7q77/zcj/0W8/BCygo86o7lK7Gty5J4IqB76KGlu4VoSz3tAoqk+su+19EztAg2G+Wb3c= X-Rspamd-Queue-Id: 46kWck4yCzz412h X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=eol-001.com header.s=trust header.b=HsOoPlUe; dkim=pass header.d=grossiste-superfood.ch header.s=trust header.b=ptb2gG27; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-stable=freebsd.org@mail51x10.eol-001.com designates 51.38.225.10 as permitted sender) smtp.mailfrom=freebsd-stable=freebsd.org@mail51x10.eol-001.com X-Spamd-Result: default: False [5.10 / 15.00]; HAS_REPLYTO(0.00)[contact@grossiste-superfood.ch]; R_SPF_ALLOW(-0.20)[+ip4:51.38.225.10]; TO_DN_NONE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; URI_COUNT_ODD(1.00)[9]; DKIM_TRACE(0.00)[eol-001.com:+,grossiste-superfood.ch:+]; SUBJECT_ENDS_EXCLAIM(0.00)[]; FORGED_SENDER(0.00)[contact@grossiste-superfood.ch,freebsd-stable=freebsd.org@mail51x10.eol-001.com]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+,1:+,2:~]; AUTOGEN_PHP_SPAMMY(1.00)[]; ASN(0.00)[asn:16276, ipnet:51.38.0.0/16, country:FR]; FROM_NEQ_ENVFROM(0.00)[contact@grossiste-superfood.ch,freebsd-stable=freebsd.org@mail51x10.eol-001.com]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[eol-001.com:s=trust,grossiste-superfood.ch:s=trust]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_PHPMAILER_SIG(0.00)[]; DMARC_NA(0.00)[grossiste-superfood.ch]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; FORGED_SENDER_VERP_SRS(0.00)[]; HTML_SHORT_LINK_IMG_2(1.00)[]; MANY_INVISIBLE_PARTS(0.05)[1]; NEURAL_SPAM_MEDIUM(0.89)[0.889,0]; NEURAL_SPAM_LONG(0.97)[0.971,0]; ENVFROM_VERP(0.00)[]; IP_SCORE(0.70)[ipnet: 51.38.0.0/16(2.00), asn: 16276(1.48), country: FR(-0.00)] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Oct 2019 11:54:01 -0000 Si vous ne visualisez pas correctement ce message, cliquez ici Soyez tendance!!=C2=A0 Faites d=C3=A9couvrir la saveur de l'a=C3=A7a=C3=AD =C3=A0 vos clients !! A=C3=A7a=C3=AD Delight : Fournisseur d'a=C3=A7a=C3=AD en Suisse La baie d=E2=80=99a=C3=A7a=C3=AD est consid=C3=A9r=C3=A9e comme un super al= iment en raison de ses nombreuses vertus! Chez Aca=C3=AD Delight, nous comm= ercialisons cette baie congel=C3=A9e aupr=C3=A8s des professionnels de la r= estauration et de l=E2=80=99h=C3=B4tellerie. Visitez notre site web =C2=A0 Un go=C3=BBt unique et inimitable, encore m=C3=A9connu en Europe. Offrez un= e exp=C3=A9rience in=C3=A9dite =C3=A0 vos clients! En savoir plus =09 La baie d'a=C3=A7a=C3=AD fait partie de ce qu' on appelle un "super-aliment= " en raison de ses multiples vertus. En savoir plus =C2=A0 A qui s'adresse l'a=C3=A7ai ? Hotels / Restaurents Surprenez vos clients en leur proposant de d=C3=A9licieuses pr=C3=A9paratio= ns =C3=A0 base d'A=C3=A7ai Delight! =09 Magasins Bio La baie d'a=C3=A7a=C3=AD est aujourd'hui tr=C3=A8s recherch=C3=A9e par les = adeptes d'une alimentation saine! =09 Fitness Ce fruit est recommand=C3=A9 dans le r=C3=A9gime alimentaire de n=E2=80= =99importe quel sportif qui souhaite augmenter ses performances. =C2=A0 A=C3=A7a=C3=AD Delight=C2=A0 Rue du Grand-Bureau 20, 1227 Gen=C3=A8ve, Suisse=C2=A0 contact@acai-delight.com =09 +41 78 848 72 99 Pour vous d=C3=A9sinscrire, cliquer ici From owner-freebsd-stable@freebsd.org Fri Oct 4 14:05:31 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5CDE112F174; Fri, 4 Oct 2019 14:05:31 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46lBV238tjz4PMk; Fri, 4 Oct 2019 14:05:30 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf1-f43.google.com with SMTP id u3so4552022lfl.10; Fri, 04 Oct 2019 07:05:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:openpgp:autocrypt:message-id :date:user-agent:mime-version:content-language :content-transfer-encoding; bh=wH5qzAlOJ0J+wMqIfzZGPuZb5TnbcfYtxIkN47/AHUU=; b=Brlqi+n+eV5KZ9rjSsr93BxsP4sIOXIwOmx9c9P6f7Ckn3Kbo322Y2LlIf+HGrjPpF tu+jYc2unTpCopzdQeS56/oJpEygb7sKIddgBkv9DkYWtOuWA+mIOl1emDwzZkdm3bvS HnOIMyUrYxSvLmsNT1KoXnLUF/Vfj7QcX1Exv2vPHsLisPJ+6WtBZL1AL6gYmdcZvYJd vDG0x1HPt/NqOP3UHtakrQPgoLk2HrStmkMzt35UdU3GasFo6MnYGlSbpZmZsBO80vhP AKO5jlxMPcDfTf5J6CC3XeXBIuoQRVY4f1tKlLUC+RRBjBrhhPf0kdRYA+LylG5UUT58 ZUtw== X-Gm-Message-State: APjAAAVs3TcaTXZAiBwuZ9smDNywVX7eHCCviY3K3kjXL5H66wsaKbZi sR5KZgoBuuuhfMso498Y2DGyDGrkIP8= X-Google-Smtp-Source: APXvYqzzgow72OLLUJmov3d/4mP6fgl2/MIcRksh0em+MpG8BZH5Ja+bEEVKAimYxkRXvtEEHb/ndg== X-Received: by 2002:a19:f617:: with SMTP id x23mr8889057lfe.97.1570197928220; Fri, 04 Oct 2019 07:05:28 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id x30sm1303977ljd.39.2019.10.04.07.05.26 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 04 Oct 2019 07:05:26 -0700 (PDT) To: freebsd-current@FreeBSD.org, freebsd-stable@FreeBSD.org, freebsd-fs@FreeBSD.org From: Andriy Gapon Subject: ZFS with 32-bit, non-x86 kernel Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= mQINBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABtB5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz6JAlQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryLkCDQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAYkCPAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: Date: Fri, 4 Oct 2019 17:05:25 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46lBV238tjz4PMk X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.167.43 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-3.17 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[FreeBSD.org]; RCVD_TLS_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-1.17)[ip: (-0.38), ipnet: 209.85.128.0/17(-3.27), asn: 15169(-2.16), country: US(-0.05)]; RCVD_IN_DNSWL_NONE(0.00)[43.167.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[43.167.85.209.rep.mailspike.net : 127.0.0.17]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[96.151.72.93.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Oct 2019 14:05:31 -0000 Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? If you do, could you please let me know? Along with uname -rmp output. Thank you! -- Andriy Gapon From owner-freebsd-stable@freebsd.org Fri Oct 4 17:17:03 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EF01D133B16 for ; Fri, 4 Oct 2019 17:17:03 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46lGl25bpyz4b57 for ; Fri, 4 Oct 2019 17:17:02 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id x94HGxaR016522 for ; Fri, 4 Oct 2019 19:16:59 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 8463FAC7 for ; Fri, 4 Oct 2019 19:16:59 +0200 (CEST) To: freebsd-stable From: Harry Schmalzbauer Subject: =?UTF-8?Q?ZVOLs_volmode/sync_performance_influence_=e2=80=93_affect?= =?UTF-8?Q?ing_windows_guests_via_FC_RDM.vmdk?= Organization: OmniLAN Message-ID: Date: Fri, 4 Oct 2019 19:16:59 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Fri, 04 Oct 2019 19:16:59 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-Rspamd-Queue-Id: 46lGl25bpyz4b57 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd@omnilan.de designates 2a00:e10:2800::a130 as permitted sender) smtp.mailfrom=freebsd@omnilan.de X-Spamd-Result: default: False [-5.82 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[omnilan.de]; RCPT_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-3.52)[ip: (-9.27), ipnet: 2a00:e10:2800::/38(-4.61), asn: 25074(-3.70), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:25074, ipnet:2a00:e10:2800::/38, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Oct 2019 17:17:04 -0000 Hello, I noticed a significant guest write performance drop with volmode=dev during my 12.1 fibre channel tests. I remember having heard of such reports by some people oaccasionally during the last years, so I decided to see how far I can track it down. Unfortunately, I found no way to demonstrate the effect with in-box tools, even not utilizing fio(1) (from ports/benchmarks/fio). Since I don't know how zvols/ctl work under the hood, I'd need help from the experts, how/why volmode seems to affect sync property/behaviour. The numbers I see let me think that setting volmode=geom will cause the same ZFS _zvol_-behaviour as setting the sync property to "disabled". Why? Shortest summary: Performance on Windows guests writing files onto a NTFS filesystem drops by factor ~8 with volmode=dev, but · After setting sync=disabled with vomode=dev ZVOLs, I see the same write rate as I get with volmode=geom. · Also, disabling write cache flush on windows has exactly the same effect, while leaving sync=standard. Here's a little more background information. The windows guest uses the zvol-backed-FC-target as mapped raw device from a virtual SCSI controller (ZVOL->ctl(4)->isp(4)->qlnativefc(ESXi-Initiator)->RDM.vmdk->paravirt-SCSI->\\.\PhysicalDrive1->GPT...NTFS) The initiator is ESXi6.7, but I'm quiet sure I saw the same effect with iSCSI (windows software iSCSI initiator) instead of FC some time ago, while I haven't falsified this run. Here's what I've done trying to reproduce the issue, leaving windows/ESXi out of the game: I'm creating a ZVOL block backend for ctl(4):     zfs create -V 10G -o compression=off -o volmode=geom -o sync=standard MyPool/testvol     ctladm create -b block -d guest-zvol -o file=/dev/zvol/MyPool/testvol The first line creates the ZVOL with default values.  If the pool or parent dataset hasn't set local values for the compression, volmode or snyc properties, defining the 3 "-o"s can be omitted.     ctladm port -p `ctladm port -l | grep "camsim.*naa" | cut -w -f 1` -o on Now I have a "FREEBSD CTLDISK 0001", available as geom "daN". To simulate even better, I'm using the second isp(4) port as initiator (to be precise, I use 2 ports in simultanious target/initiator role, so I have the ZVOL backed block device available with and without real FC link in the path) Utilizing dd(1) on the 'da' connected to the FC-initiator, I get _exactly_ the same numbers as I get in my windows guest along all the different block sizes!!! E.g., for the 1k test, I'm running     dd if=/dev/zero bs=1k of=/dev/da11 count=100k status=progress (~8MB/s) For those wanting to follow the experiment – remove the "volmode=geom"-zvol:     ctladm port -p `ctladm port -l | grep "camsim.*naa" | cut -w -f 1` -o off     ctladm remove -b block -l 0  (<– only if you don't have LUN 0 in use otherwise)     zfs destroy MyPool/testvol "volmode" property can be altered at runtime, but won't have any effect!  Either you would have to reboot or re-import the pool. For my test I can simply create a new, identical ZVOL, this time with volmode=dev (instead of geom like before).     zfs create -V 10G -o compression=off -o volmode=dev -o sync=standard MyPool/testvol     ctladm create -b block -d guest-zvol -o file=/dev/zvol/MyPool/testvol     ctladm port -p `ctladm port -l | grep "camsim.*naa" | cut -w -f 1` -o on Now the same Windows filesystem write test drops throughput rate by factor 8 up for .5-32k block sizes and still about factor 3 for larger block sizes. (at this point you'll most likely have noticed a panic with 12.1-BETA3; see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240917 ) Unfortunately, I can't see any performance drop with the dd line from above. Since fio(8) has an parameter to issue fsync(3) any N written blocks, I also tried to reproduce with:     echo "[noop]" | fio --ioengine=sync --filename=/dev/da11 --bs=1k --rw=write --io_size=80m --fsync=1 - To my surprise, I still do _not_ see any performance drop, while I reproducably see the big factor 8 penalty on the windows guest. Can anybody tell me, which part I'm missing to simulate the real-world issue? Like mentioned, either disabling disk's write chache flush in windows, or alternatively setting sync=disabled restore the windows write throughput to the same numbers as with volmode=geom. fio(1) has the not usable ioengine "sg", which I know nothing about.  Maybe somebody has any hint in that direction? Thanks -harry From owner-freebsd-stable@freebsd.org Fri Oct 4 21:05:57 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E4C35139486; Fri, 4 Oct 2019 21:05:57 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46lMq96FcCz3ML0; Fri, 4 Oct 2019 21:05:57 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 70E4FEDCE; Fri, 4 Oct 2019 21:05:57 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Fri, 4 Oct 2019 21:05:55 +0000 From: Glen Barber To: freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org Cc: FreeBSD Release Engineering Team Subject: FreeBSD 12.1-BETA3 Now Available Message-ID: <20191004210555.GM4030@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed User-Agent: Mutt/1.12.1 (2019-06-15) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Oct 2019 21:05:58 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 The third BETA build of the 12.1-RELEASE release cycle is now available. Installation images are available for: o 12.1-BETA3 amd64 GENERIC o 12.1-BETA3 i386 GENERIC o 12.1-BETA3 powerpc GENERIC o 12.1-BETA3 powerpc64 GENERIC64 o 12.1-BETA3 powerpcspe MPC85XXSPE o 12.1-BETA3 sparc64 GENERIC o 12.1-BETA3 armv6 RPI-B o 12.1-BETA3 armv7 BANANAPI o 12.1-BETA3 armv7 BEAGLEBONE o 12.1-BETA3 armv7 CUBIEBOARD o 12.1-BETA3 armv7 CUBIEBOARD2 o 12.1-BETA3 armv7 CUBOX-HUMMINGBOARD o 12.1-BETA3 armv7 RPI2 o 12.1-BETA3 armv7 PANDABOARD o 12.1-BETA3 armv7 WANDBOARD o 12.1-BETA3 armv7 GENERICSD o 12.1-BETA3 aarch64 GENERIC o 12.1-BETA3 aarch64 RPI3 o 12.1-BETA3 aarch64 PINE64 o 12.1-BETA3 aarch64 PINE64-LTS Note regarding arm SD card images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root. It is strongly recommended to change the password for both users after gaining access to the system. Installer images and memory stick images are available here: https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.1/ The image checksums follow at the end of this e-mail. If you notice problems you can report them through the Bugzilla PR system or on the -stable mailing list. If you would like to use SVN to do a source based update of an existing system, use the "releng/12.1" branch. A summary of changes since 12.1-BETA2 includes: o An issue with imx6-based arm boards had been fixed. o An issue with 64-bit long double types leading to link failures had been fixed. o An overflow logic error had been fixed in fsck_msdosfs(8). o An issue in destruction of robust mutexes had been fixed. o Support for the '-vnP' flags to the zfs send subcommand had been added for bookmarks. o The ixgbe(4) driver had been updated to prevent a potential system crash with certain 10Gb Intel NICs. o A regression with the zfs send subcommand when using the '-n', '-P', and '-i' flags had been fixed. o The freebsd-update(8) utility had been updated to include two new subcommands, updatesready and showconfig. o Support for 'ps -H' had been added to kvm(3). o An issue when compiling certain ports targeting Intel Atom CPUs had been fixed. o A use-after-free in SCTP had been fixed. o A regression that could lead to a system crash when using vmxnet3 had been fixed. A list of changes since 12.0-RELEASE is available in the releng/12.1 release notes: https://www.freebsd.org/releases/12.1R/relnotes.html Please note, the release notes page is not yet complete, and will be updated on an ongoing basis as the 12.1-RELEASE cycle progresses. === Virtual Machine Disk Images === VM disk images are available for the amd64, i386, and aarch64 architectures. Disk images may be downloaded from the following URL (or any of the FreeBSD download mirrors): https://download.freebsd.org/ftp/releases/VM-IMAGES/12.1-BETA3/ The partition layout is: ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label) ~ 1 GB - freebsd-swap GPT partition type (swapfs GPT label) ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label) The disk images are available in QCOW2, VHD, VMDK, and raw disk image formats. The image download size is approximately 135 MB and 165 MB respectively (amd64/i386), decompressing to a 21 GB sparse image. Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI loader file is needed for qemu-system-aarch64 to be able to boot the virtual machine images. See this page for more information: https://wiki.freebsd.org/arm64/QEMU To boot the VM image, run: % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt \ -bios QEMU_EFI.fd -serial telnet::4444,server -nographic \ -drive if=none,file=VMDISK,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -device virtio-net-device,netdev=net0 \ -netdev user,id=net0 Be sure to replace "VMDISK" with the path to the virtual machine image. === Amazon EC2 AMI Images === FreeBSD/amd64 EC2 AMIs are available in the following regions: eu-north-1 region: ami-07085de4e26071c9e ap-south-1 region: ami-095bd806d8acfffb1 eu-west-3 region: ami-0314542b8d7579bdd eu-west-2 region: ami-06ec921eb87ef4d7b eu-west-1 region: ami-0f0051c800be4091e ap-northeast-2 region: ami-0f109258a463177bb ap-northeast-1 region: ami-0224a1cb8e19333b8 sa-east-1 region: ami-0536a86bff5f33356 ca-central-1 region: ami-06709921360dccfa3 ap-east-1 region: ami-0142af9336f6e529c ap-southeast-1 region: ami-0c439e0bc0c567dd3 ap-southeast-2 region: ami-0fa770b7f07583b48 eu-central-1 region: ami-0dfca49cf2ba89c43 us-east-1 region: ami-06884b4e2e511590f us-east-2 region: ami-06c687665309d8b17 us-west-1 region: ami-0dce597e8b07a6c6d us-west-2 region: ami-0e1f5ccdd2221b1d6 FreeBSD/aarch64 EC2 AMIs are available in the following regions: eu-north-1 region: ami-0914805810f9fcca2 ap-south-1 region: ami-0862409434d8089f2 eu-west-3 region: ami-08e6dc501b060fa7c eu-west-2 region: ami-000f2362fef121710 eu-west-1 region: ami-0c7a18e2b216a1b0c ap-northeast-2 region: ami-047bc72d91ab47b95 ap-northeast-1 region: ami-082dcca6b9ac52f3f sa-east-1 region: ami-0a9fd8ffffc889430 ca-central-1 region: ami-0d513e584801de3fa ap-east-1 region: ami-089a6b231886f692f ap-southeast-1 region: ami-09c6f305a761c8712 ap-southeast-2 region: ami-0ee5e02b85aecbef3 eu-central-1 region: ami-08321cbac28d28d71 us-east-1 region: ami-0a9c2fdd733536b50 us-east-2 region: ami-0a5b46f4260ed9ca5 us-west-1 region: ami-01aca4de517a623fe us-west-2 region: ami-0ac8b561fb3597d89 === Vagrant Images === FreeBSD/amd64 images are available on the Hashicorp Atlas site, and can be installed by running: % vagrant init freebsd/FreeBSD-12.1-BETA3 % vagrant up === Upgrading === The freebsd-update(8) utility supports binary upgrades of amd64 and i386 systems running earlier FreeBSD releases. Systems running earlier FreeBSD releases can upgrade as follows: # freebsd-update upgrade -r 12.1-BETA3 During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly. # freebsd-update install The system must be rebooted with the newly installed kernel before continuing. # shutdown -r now After rebooting, freebsd-update needs to be run again to install the new userland components: # freebsd-update install It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example, FreeBSD 11.x. Alternatively, the user can install misc/compat11x and other compatibility libraries, afterwards the system must be rebooted into the new userland: # shutdown -r now Finally, after rebooting, freebsd-update needs to be run again to remove stale files: # freebsd-update install == ISO CHECKSUMS == o 12.1-BETA3 amd64 GENERIC: SHA512 (FreeBSD-12.1-BETA3-amd64-bootonly.iso) = 3ca05e8bd48ac420621b1cb86842de3f0ea157a3d1051bf43354726c41f4c5a29421dd48729d7ab00cc5939e0f795b66d138fdb05abf2a9cc8fb83d7e2f57749 SHA512 (FreeBSD-12.1-BETA3-amd64-bootonly.iso.xz) = 4a7ded42cc283565226848cd8cc7ef1b409de001297f5c5a55e48c62badf4af3be3cca2606bb1ceb1fd4188b11268bb2b8c7ada96d94ba235c3e766bcd2b1f6a SHA512 (FreeBSD-12.1-BETA3-amd64-disc1.iso) = 71b413a52b0d8a4ad9fd9d0d1708fc9349b6d29d34ff15b9013b474de6a32034419c23bb30c0c1131d71957918d3f2fdd5d2825a01cf92b54a19af06fb4d2fc6 SHA512 (FreeBSD-12.1-BETA3-amd64-disc1.iso.xz) = 99db62096b415501cb186e83610f599bd3f36e02b7aac4c99658586c99441339892809f447e09a7256588b5b29fd3c1632626df59589c831548dff7877c463a0 SHA512 (FreeBSD-12.1-BETA3-amd64-dvd1.iso) = 2b36635c032ca7ca0396b74908e91f5a5ed9b1911dae42f87a611f3456ea9195bc6f0ef740703822bade4d61d8ee67fdb32f8d0ff81953e64d60a27f313a3444 SHA512 (FreeBSD-12.1-BETA3-amd64-dvd1.iso.xz) = 73610161093bce3a2c7951d3f5b6cce09b7fa8fdb4066515ff95efe99084c188d2899fe622793af5dc4ec0a5788e81ad8b515866b67ac3f9f8c0fd2c038476c3 SHA512 (FreeBSD-12.1-BETA3-amd64-memstick.img) = e3615ab92a5ec8fbd503b016ca5ef1af5d34de8021b8606e1e41a85abc60a42616cb74407d46ee665c6255d82d49441d3e976776c20e9bc0127525a57c98bc12 SHA512 (FreeBSD-12.1-BETA3-amd64-memstick.img.xz) = 0d7a384b1654ff7652b30c4b6a6d89dadbe59cbfe5c5b3f172420af134de19ff84f7e753a0d7b71019fa875a68fdaa0dc876046815e21bf666c93c19b24039aa SHA512 (FreeBSD-12.1-BETA3-amd64-mini-memstick.img) = 6ca8e687b46b5b2f8e35fb81e2f3daf788c9bd1b469afd4aba19aa81573f6cd2f337f009092541cd827657ad99843d3043f9573ac53bd4d49b086a97ff60f185 SHA512 (FreeBSD-12.1-BETA3-amd64-mini-memstick.img.xz) = be9facc283071e753b007d0a4a13cd044a3db9c187bf83507fd116428a1773eeda36397ffd3477b559d5f4a1c848285722867120a5b08b958a1f1edce58cf406 SHA256 (FreeBSD-12.1-BETA3-amd64-bootonly.iso) = b0d796f23c33f411dbd0eeaf8f5787318ff9c794568ecb30836cfdbc9b5577a0 SHA256 (FreeBSD-12.1-BETA3-amd64-bootonly.iso.xz) = 8accc9060948c692b6845e2ae6a698f7adf1c5fb6f9a45a24f1049877dabf32d SHA256 (FreeBSD-12.1-BETA3-amd64-disc1.iso) = 3e50474412efc961664d3cced6a24a1adb9f9fac9dca7c43c1034b5dbf62c9e6 SHA256 (FreeBSD-12.1-BETA3-amd64-disc1.iso.xz) = 6e82c48bff4a0001f111652dd9e79e46b76ab75b0329668e80ed5505e5e2b147 SHA256 (FreeBSD-12.1-BETA3-amd64-dvd1.iso) = 861951dd4b7afccccf50c574d74fc6a21eb0f9ce3dcbfc992bdffdaa418fc548 SHA256 (FreeBSD-12.1-BETA3-amd64-dvd1.iso.xz) = 12bc51f796554a00d3233a5313e77d0c6a7b787146d13782b97c424581ca9b8d SHA256 (FreeBSD-12.1-BETA3-amd64-memstick.img) = a5320a4febc26429bb8bf81d3d3ae59b339014c7a7bf7d093cfa81319a887c47 SHA256 (FreeBSD-12.1-BETA3-amd64-memstick.img.xz) = ed2947dcaeed7d088d7df97ae97eaa3cf1e0f0ae6354bc0f57f017356e7adbbf SHA256 (FreeBSD-12.1-BETA3-amd64-mini-memstick.img) = 79b63ce5af0d292903d00cc97fd58a4e25a4e8a20c0f830945ee8d3523909bd6 SHA256 (FreeBSD-12.1-BETA3-amd64-mini-memstick.img.xz) = ce3bf17c7339d6985d3cba543214fb441a77bcf77908006ac52790f143e96ff4 o 12.1-BETA3 i386 GENERIC: SHA512 (FreeBSD-12.1-BETA3-i386-bootonly.iso) = 3f72623e9e40934ba7d7bcc1bfd7c543f9369eaeeb649bdadba22f4ac628f9d7a341bfd31dfdc126629127cbd05d27443ed4481a8e8f122266b8b0f8e135771c SHA512 (FreeBSD-12.1-BETA3-i386-bootonly.iso.xz) = 1e7641017b00f73e60368257c5ed04fdc686b1fb3df8ac6544f4b8ee72b8fd3cba97d7695f11dada2c1d71906106adbfde2f491152f84daac1f05e24a22b0575 SHA512 (FreeBSD-12.1-BETA3-i386-disc1.iso) = 76f0c3a1f8419c7bec8c93fe435e66fb41d66a242ff6ac94a17a94e977794a21179eecdddd85a0db62f85d94a2356309ab03b1bec55d6d355e8cf394977fbabd SHA512 (FreeBSD-12.1-BETA3-i386-disc1.iso.xz) = 69957c30a44c676e46274565aae0eeaaf9f1cb4584476b139ddad2e0fb33d83b6da09c7a99bf7b596ab544e5ac02fed7aa9830e3bff67167342bbcfdbf118c9c SHA512 (FreeBSD-12.1-BETA3-i386-dvd1.iso) = b6eee5fda39f2a774f0273bb895f6183157a51e35d755eb27f07c9b4179dd0990c20223139cbd6c5bb2e509c59b2d816057a8b0d8792077eb470bc9647e3e3f1 SHA512 (FreeBSD-12.1-BETA3-i386-dvd1.iso.xz) = 622f5ba0ab93c78151a59a03ebc75059ad14a4cb6cf131c6c1c68b7d797ca712b5a5bdca0ec6c496b62dae2fab5234da0bbfccd2de737908f171f654ee7a239a SHA512 (FreeBSD-12.1-BETA3-i386-memstick.img) = 22b2a225020a4fcba13e6b2c4069adaa6352c92a4dfc96a1070d35084c2384b3a938aa3ed27e3daf449abf1e0a4713f88136080ab954b78783ee7720fca2098c SHA512 (FreeBSD-12.1-BETA3-i386-memstick.img.xz) = 7d2831e20a800537659b0139d9c754e98410891c866296321d4ebfd444f4a4aa0bda6d4c87665744cec46909ff4f1c4a91c564f1a5cc7319be2cae4c5913688d SHA512 (FreeBSD-12.1-BETA3-i386-mini-memstick.img) = 50b2eab8c72c632598cb6fb696dab0fc7c4fffdb566c9218fffd0a02ff9e279d4eae481c3af16aee6fa42f02fe15c2ce6b0856d42b74d998df396978de39616a SHA512 (FreeBSD-12.1-BETA3-i386-mini-memstick.img.xz) = 38ebfaa7a86e6d11e85cbde373d2e732195aab81f1eef7826efa47bb7202a9a888444ea87ca7b077962e0017389abed7e344d52f2af4c67a3d2cd4437de6d684 SHA256 (FreeBSD-12.1-BETA3-i386-bootonly.iso) = 7b184295d78d78d1faedfb182e18ee089cc82260bf6d81ca0f9202a251d0cc1b SHA256 (FreeBSD-12.1-BETA3-i386-bootonly.iso.xz) = a7bd02b0b513866626a0f89778d3ae77729bced260d0e0f5fed50a9f30f111bf SHA256 (FreeBSD-12.1-BETA3-i386-disc1.iso) = e5025d61f51e88e732bd319ab7a2884390c0762e63792c43504b3e1557a6b5c1 SHA256 (FreeBSD-12.1-BETA3-i386-disc1.iso.xz) = 452fd4eccf1bc20cefa7cab6705ce91364b06633b22753a45a3231071798f818 SHA256 (FreeBSD-12.1-BETA3-i386-dvd1.iso) = f3c3654c307933018c6fb312f11f18cbf6f484c1ffbaf219c3e15c1ee4df54d7 SHA256 (FreeBSD-12.1-BETA3-i386-dvd1.iso.xz) = 1dc4cb2eb5f46adb55609d293b2d423cc9d22c10a393695751987ba18b432450 SHA256 (FreeBSD-12.1-BETA3-i386-memstick.img) = e6275d7b14eb92cc97de7944730fafa61644c724bc9d6104f3169adee278138f SHA256 (FreeBSD-12.1-BETA3-i386-memstick.img.xz) = c30bd315f0691d8726093004f30404a614c87dfa6b97cad62784d95148fd2a34 SHA256 (FreeBSD-12.1-BETA3-i386-mini-memstick.img) = 65f86cf20aecd5059fbad9688c55dc0df96bf41c72ad160eb1ae496b4a03f418 SHA256 (FreeBSD-12.1-BETA3-i386-mini-memstick.img.xz) = 303ba32bc4cf4913956d0ec2d2d64a75be18fdac61fe00a17ee5979884aa791e o 12.1-BETA3 powerpc GENERIC: SHA512 (FreeBSD-12.1-BETA3-powerpc-bootonly.iso) = 826b2f5731764dfa0114291e2ed3f4b0d385938e767e8b92778fb258a45217e7578adb704dedd2b5d69ee5af86c527f637282d5542429502866ebe168b7c8b49 SHA512 (FreeBSD-12.1-BETA3-powerpc-bootonly.iso.xz) = 84cb81c0c6dc54d86438be739731ad6db485875c7881bf80f4a83d9422bf7f88ae60ae340d3a9aeaf1be43d96d74c93e208c0b336cfcc88ade484c117f4f45dc SHA512 (FreeBSD-12.1-BETA3-powerpc-disc1.iso) = 48753d7fd757aed35e6fcec783df57694791aadfcda4000eb220665b232d66e27bbb88c602e99921af081b4770ae3cfeb4b9df821045c7db7b3cd76ed3064416 SHA512 (FreeBSD-12.1-BETA3-powerpc-disc1.iso.xz) = 3f8b86fdc883c0a9f990577f15c28f89c6675c309283ff2908d1d9aec9bffe31fca8d1d6e8733f41bbb21ffec16461f9af15cddaa0be536059a17618a80fa240 SHA512 (FreeBSD-12.1-BETA3-powerpc-dvd1.iso) = 0ee962613c4175ac0d30fa9a996ea886c891353192e17e053844c7db9f26b7e3175efaec40e781c6e382ea78b2eeace6179272153e4847f9b5528950bdcc6bfb SHA512 (FreeBSD-12.1-BETA3-powerpc-dvd1.iso.xz) = 083fbd6f024a4c170da6ff52165863514dd786d95490968dee21de39b65450afb00bf134c7510559b22832a2f69ad0d913b0b7a2b10a6e9a47e46a48bb028c38 SHA512 (FreeBSD-12.1-BETA3-powerpc-memstick.img) = be289271d7f6da1ea189b316617349fddb38e42e4e9424b854fecd8db83ae0497ecf84ec2db505018c644b27a2d851acb8e81bd8071d5d442e75cef4e9343b36 SHA512 (FreeBSD-12.1-BETA3-powerpc-memstick.img.xz) = bf6987498ccf61c99f9315eef2c9424fe104a77d864447ed8149c360f02a56feb0398a4779776aa5d1d19effb3e43ae5d0735adca565dabe3c12320ab7f8c3d5 SHA512 (FreeBSD-12.1-BETA3-powerpc-mini-memstick.img) = 2033bc3dfbb33d99073e40920e15c0e1b5a580e5114e0598a2cadad9e9a207184b5f7903371383fb5e618f9b743e4704ae177318e6a166a76b559b83ab08fc09 SHA512 (FreeBSD-12.1-BETA3-powerpc-mini-memstick.img.xz) = 4e400c9f6e5e61209bd2bcc0d00a2090c5d055c81c485656524dbde5ecdac94fa1ba85fe975247817ef8dc13da16a54f6b79e4e5ec463538fd2a60208e388466 SHA256 (FreeBSD-12.1-BETA3-powerpc-bootonly.iso) = d5814bda886fab5101b70a751053f86f4090de21a1384e145967a922b156acbd SHA256 (FreeBSD-12.1-BETA3-powerpc-bootonly.iso.xz) = 6fdc8a4a83913d3b9d1ad76e6b159062126f6fa46924cfafc975e41fe3c4eb64 SHA256 (FreeBSD-12.1-BETA3-powerpc-disc1.iso) = a9424007d65686038dde8103db86adcce410c8244f1bdd0dca2409290d786329 SHA256 (FreeBSD-12.1-BETA3-powerpc-disc1.iso.xz) = 04bd2f794068525a1d9627767027667f9ce9a20254e591a25bed9bfd9ffb32e0 SHA256 (FreeBSD-12.1-BETA3-powerpc-dvd1.iso) = f5a1be285c2c71ea4b5c15df955372c7a066ea764750e4076af3a3699b45937c SHA256 (FreeBSD-12.1-BETA3-powerpc-dvd1.iso.xz) = e525d25243fea1dc0a5048e5a1edbb8996e90a27cb8a83c4187b31ce7bfcb76c SHA256 (FreeBSD-12.1-BETA3-powerpc-memstick.img) = b8b54bebc767f5e5b6fa3956d0fa11b25c3ac343c867833de05d53b898138cd7 SHA256 (FreeBSD-12.1-BETA3-powerpc-memstick.img.xz) = f992a86b981207c2a2ec6febbecbf9587488168f0da5a9964222881954c7aa1d SHA256 (FreeBSD-12.1-BETA3-powerpc-mini-memstick.img) = 363046b7f40381ab18db9975a222e2437d792a27fa79717389475588a9dff423 SHA256 (FreeBSD-12.1-BETA3-powerpc-mini-memstick.img.xz) = 60fd85d15c07676ca0b99ff6bd775f1a44b35795805c001b45e6408351829514 o 12.1-BETA3 powerpc64 GENERIC64: SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-bootonly.iso) = 18a3599a6ae60d8e42548df2913981857e87feccec4a3580ab4d68076ff9547d7b2955a2d5c66c58bcd5fe9367617e46292fc3fa34b9171f45559451b12984f3 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-bootonly.iso.xz) = 1f5d973702c814d1249311df80f6ab07f7069bb657876d3dd9bc6269024896ecc47c80f38d5eaaa544f8202d9aa725b070cebe987eccd86e453aad5cba209268 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-disc1.iso) = 9072f5a44a2cbcbc7981cdead05c79bb7147e2ba088e57d190ae3734297ff6ba265137036da71566744f6cdceac6c0d2f058115939e7fec0dc105c0a9d0fab12 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-disc1.iso.xz) = 17f738550a5eb4848dda552eb9a2491b36edbeae666e89ad0f66211108e1372d1845ef38275fb7da141fe939ecfd51d9337de558434b49acbec1b5e8b54fde6d SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-dvd1.iso) = b097f16ca49228088a8de29bde3bbcfe5d2fb07d3a8af4027db85118b215dbbecebf875d283c85db0e852a3e72c89c08bb0b8f6985d82b4b61f07ee20b6a1aa6 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-dvd1.iso.xz) = 4731e4ed5b529f3171eec15e32415af929eeb2d026f6a7f1be463e777be84ce4b3d2c149726ac7a93bc74fee7526798a57980b4d85465ae525496d21321487ae SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-memstick.img) = 8d56a40643c81c65993cf7a2535d6b4bdc5e46d3f24c4f33b82ce183d7169a0369f3d369b209fd74ec7dbdd11cf8dd52b7a40932ecc3962068b18fdde22d44a3 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-memstick.img.xz) = 614bfce766c86ffd6d544ece8cde8879f8ccb331d4162b6486988c04c049e76dec83be7ee316ab7811ee5ea092a9e9a355e9fb1c3f42952ae41fb982e1ab14cd SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-mini-memstick.img) = 2419bf1539caa95b4ee7893f6fdbabbd8c9e815e82cc32fff7eecbc66c4702e4108df55b9eeab23d6f13293e81a815f1dda2d1435607306561b04f131c597430 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpc64-mini-memstick.img.xz) = 9bbaafe83f3abb8b7de58e59dc40e22becfd9b39ce816e3d3b3df26eb5cf532c87d7b3219858e9588eca08aa2f92a6a5a8f5b0c69df4b7aa2d1b4556e4c7217d SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-bootonly.iso) = 1c13c4ca190e043c029feb6cc9b0afa5edf6622b92b2253f60470ccb538d1d7e SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-bootonly.iso.xz) = 64146727bc6ded2cc1d1a518210113c14227bfcef51653c6a16132bf2444724a SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-disc1.iso) = 40e08752ad6c0f7b0d51d82d80cec36d84cb54da39a49ea4988140894249521e SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-disc1.iso.xz) = c5db77505d2b8f466be0ecbda617b12f1b0de0fd2b7296f65607a5b2a2c9d124 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-dvd1.iso) = 4996cc4700407f5efa8dbbf3c630e5e31366bb98661fd632c13570ae92c6ac03 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-dvd1.iso.xz) = 5b426b3e7249aa4ba41b2f305ef82bba0eb2b688ed1f09faed3fd05f5c16b9ab SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-memstick.img) = 751a220eb49e3318b7c23bc22e2f80f06e97a830e859c3e626aef3154bf57679 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-memstick.img.xz) = 59b308e67f904bdd97349acccf6339fd136ab1472db9a415b112713be88ef713 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-mini-memstick.img) = 4af15e36c979dddbcec6e3327262046ca672dd499898165a430550a30e6b7175 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpc64-mini-memstick.img.xz) = a16119722c5afad406601c502d6775ada21de094c5ad378f134d2d1227a91184 o 12.1-BETA3 powerpcspe MPC85XXSPE: SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-bootonly.iso) = 12eeafbd4a31368f4727d165f04c208be2548b7d58198ef940906de8cefc42766945a03a086cd3dbb3d889bab310a2e64042d5a3a40f3070a38bec7a6f78099d SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-bootonly.iso.xz) = 79d1bd2d963c91f633b0e2b8eee9e8a6417ac4b537fd4e5ab5c1925f50a90789c4bc4add3308f44a8c1e7b120e12acd376df391c4187a8213c4253fb17c418cb SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-disc1.iso) = fb588190748e0ebd44190d27b9bb655cb09cfae823ed3df57f37fbc71a37189e7bbc556461dd7f83ee480d9e6ee276bb4df1db67d4d8a53c7fb3dd5dbbeabcea SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-disc1.iso.xz) = 35550ffc17bba5c1e92c68b849004a54f24920a56f54708ad30ebe75806f8cb596de2ffa474fa1be0fb5a64321fee8ca22587f8e60bff3a751cf74b7e4dbcde5 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-dvd1.iso) = 69f985372278ab6aec4489f9d215b546a864c3f61035ba61159e2149a388785adf1374bd48ccfc00ebb247456ff764493d9b0bf141652919b9c52e5caadb10bf SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-dvd1.iso.xz) = 2c46b077fb29fbb11c476a53e39b1191623241b0a94cc5f59d205584fa021b80398a12a92da46622c8198a36a0b52c960a01a13f2c71a6638790041f0d46e336 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-memstick.img) = 5ed242364e064e6b8d9e9eaa1b3bdcbf55a53f4d48bc67f2a88716ed5fd2ee5451ff5e8a00de49ada2fd89e6502c7af201719bf3728608cf60ccda15655dfd8b SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-memstick.img.xz) = c2104bc0bf2b1b645d868881398e429a22e2cacdccb06e9f7878d6b2ab96ddc09e71b32931657d627ee01db3edb9d914b9d8013b41a842e07aeefcd94ff3fd2a SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-mini-memstick.img) = 257c01d531d95bffada9fce5cd3fdd838ea5f660e8341bdd91ba2518b884ddbad7de5772a4ed11266ddf603e75152f994c54fb5858f94496349c28d4fe1f00b7 SHA512 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-mini-memstick.img.xz) = 2c7a19b093abbfbc19579d0c77a1aadbd3af7f958e8da7137bb4fbf69e66c7bbd51601c25bb701e5f1918d5420eb5628d0e49ededc73553cd1fd91d5d31b60bb SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-bootonly.iso) = e632e6f80da69be824cfab7d1bcceacfca8e08d4324707c1eea4007e55feaf8e SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-bootonly.iso.xz) = 07d04b38c5bd087f3728040ed6f4a187ea8e0fdd7cb1b3b401400d224458e9fa SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-disc1.iso) = faa0c6a0451b9ac527688a0cab6c5cee5e39324e85a837dbb74bb7c64dc2ed94 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-disc1.iso.xz) = e349496c4efce3fb082cc81749b8e1e78e219ec2150b4d77f3b38cea6dab96bc SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-dvd1.iso) = 6d3199dec39eccca9e1660f6ab5337c0efd716b3224dfd689e5c97b41cebb903 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-dvd1.iso.xz) = 64a7f976973f0cd1d8ba759136365991065ae4c3c5713c1fd72cda59854219bb SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-memstick.img) = 4cf585da2d94c61f917fe1d51f3e6d56f5b6f52494bd178611181d9158ca5ef1 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-memstick.img.xz) = 6d9c71534f90168b0c9e7831432265db7697859cfe4eb969246457ab55b996fc SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-mini-memstick.img) = 57822c24a2cc84848fb96d276373a1397333cbfb956939c92d5ba07e81bfd111 SHA256 (FreeBSD-12.1-BETA3-powerpc-powerpcspe-mini-memstick.img.xz) = bdc70aa7762a451baf390105ae29d691fe68f6b76cef17f3eeaac001db1eae16 o 12.1-BETA3 sparc64 GENERIC: SHA512 (FreeBSD-12.1-BETA3-sparc64-bootonly.iso) = da3f7703dacef804b445fbd1b51a96b3763ea0b48fc33316ec0688a70c3641afd0fc987cc78d33af5735d26baa5c0f72b2a2d9a848d68aa261c32fb97a0479af SHA512 (FreeBSD-12.1-BETA3-sparc64-bootonly.iso.xz) = 810dd132f5d03d7a125d684365005ba27c25d6279325e40df1d5f635ee207616fca658a6559d0febf15a30f7ae1fd7c58c10061aca76dafc6a9e78e50317fac2 SHA512 (FreeBSD-12.1-BETA3-sparc64-disc1.iso) = c22f35132c290266f9340710df04eca3ede310366392a4c1c0ee1f226a9cee5eb2802eedf248f8cf26d6d4f3cad047d8ba635c61baee8dfcfd3d585d9cb6a3c5 SHA512 (FreeBSD-12.1-BETA3-sparc64-disc1.iso.xz) = 16bb4a5007e3136c390a276bdf7a44555141cbb710e6831d883a1d28f8f752458d2032817e627631f877a950380c48a06e80e99263d7cef27de23cc1adfeaf0a SHA512 (FreeBSD-12.1-BETA3-sparc64-dvd1.iso) = 5aca1dcc9f813fd38c1a0bb08ec73308788b62eaf3983e514aa3229fb541d9dd574ea3926c35de70c3eb933579c320ee732721f8e864aa5e902b7f5e0aeff713 SHA512 (FreeBSD-12.1-BETA3-sparc64-dvd1.iso.xz) = 79b03461dbce9336b02632887d9af6f010ad0bf2054edbbadc4b544b617da595f363b413b4879796f72bd7c29222c466f62e79d605560cc6f9301348f4ac244f SHA256 (FreeBSD-12.1-BETA3-sparc64-bootonly.iso) = 3891e4520f3a04349350207ef6f1ccfb6e83350ff088269b7decf7b051f31ad3 SHA256 (FreeBSD-12.1-BETA3-sparc64-bootonly.iso.xz) = f308452758fac3115e101ad8d2d318f2cfc5a520525ac9b8ca330e203d1bfa4f SHA256 (FreeBSD-12.1-BETA3-sparc64-disc1.iso) = e850231cf68dbb615f4826a16dd26b7583cd83a1884cd5f3d3d33b7ab7c31891 SHA256 (FreeBSD-12.1-BETA3-sparc64-disc1.iso.xz) = 036fdb7ddc7ef90308870e0d0ad269347d88a24baed424c1f7630131edc02fa7 SHA256 (FreeBSD-12.1-BETA3-sparc64-dvd1.iso) = 3ff374ce2f40501db65a20bc33e6987bc88ea74f8e163e9109512e31d2697d71 SHA256 (FreeBSD-12.1-BETA3-sparc64-dvd1.iso.xz) = 0b2a27e504f28dd18c9c5ee99cd1e3e6648c551cea9f630f716b898207a981db o 12.1-BETA3 armv6 RPI-B: SHA512 (FreeBSD-12.1-BETA3-arm-armv6-RPI-B.img.xz) = 63ba7ab6fedad1ec71aaa53bcefa5e8729c9356ab2a0f160632e423ee6eaf6baab5f12474bd09f3c5c227cc80ed0b35d5f7f9dce81cd0b49fbab25748b330687 SHA256 (FreeBSD-12.1-BETA3-arm-armv6-RPI-B.img.xz) = 1e02d1475753a0c4aa1e619c4d5e5b3cea458e79d31a0f920868a7a63f312d9c o 12.1-BETA3 armv7 BANANAPI: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-BANANAPI.img.xz) = 3b2b005679fa258ad4b5255ce32768ee93742bf7eaa54f51122b18cf95ef915cad8008b8b782e83f9b55ce66a241b52a2876ad3d72b42beac283ef2589bd911e SHA256 (FreeBSD-12.1-BETA3-arm-armv7-BANANAPI.img.xz) = 9f85a67a66b3c5a0739d54cbb52c3b244bb7c2ca20834b811fb14b8b89f8bee6 o 12.1-BETA3 armv7 BEAGLEBONE: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-BEAGLEBONE.img.xz) = 84ef6e8ca92c4c6d7dc99e2425dbab332809cc782e8c510516c89c0331760f664d485b01a7290a799811f54857894d3bd6ee29090cb329fe3c654b331a0055d8 SHA256 (FreeBSD-12.1-BETA3-arm-armv7-BEAGLEBONE.img.xz) = dd8453de2fa3d0239d4226f2a58bcfa5fd1678a64ee486d32e994c0bbc4d9d48 o 12.1-BETA3 armv7 CUBIEBOARD: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-CUBIEBOARD.img.xz) = b77b19bae60f2645c13ba4ad2110fea68c296f9a6ec33fb6b69517767c4f5b3194fff47b5b019777534bc856aefdecabf731c9c6b0c0c7f653cd178b5b24178c SHA256 (FreeBSD-12.1-BETA3-arm-armv7-CUBIEBOARD.img.xz) = b0ff6be9c8b6dc6aae32cf6cc70581593fd529710c88a0fa24bbcaaffaa234c8 o 12.1-BETA3 armv7 CUBIEBOARD2: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-CUBIEBOARD2.img.xz) = cb5e59d1b950f07aeda0e98b7a25b1ca0f92a9ef7e21b8ca8e5c68dadc33d3c2d683afa33bd6b85a8f19c40010a2b0bc576ca0fdc9cc67535528d137ebd8e8e8 SHA256 (FreeBSD-12.1-BETA3-arm-armv7-CUBIEBOARD2.img.xz) = 16a1158a60212d94463dd95e9bb5561ae88fcb6c39724d879ad9d707eb43f2c7 o 12.1-BETA3 armv7 CUBOX-HUMMINGBOARD: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-CUBOX-HUMMINGBOARD.img.xz) = 6c5a9ea150626647ea03b2c7f72299466aa47ba17eff307a7cc3bbcc4ff704c2e163cbf2b2e86ef6d4cdb7a7444ffcbdad9af722e49149a928fc7cddedfdb9d7 SHA256 (FreeBSD-12.1-BETA3-arm-armv7-CUBOX-HUMMINGBOARD.img.xz) = 473f05565450e78e92be4ce29f741c51e13b0b38d8c700d96fe1fa3fd1155e00 o 12.1-BETA3 armv7 RPI2: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-RPI2.img.xz) = c4ed518217cafb114032c6569bf12925df134378ab4093da8dd8053f934442062bbba5bbc4f6b6d34a8185c42ad0fd2f9efb205962390b1e91cab84415a694fb SHA256 (FreeBSD-12.1-BETA3-arm-armv7-RPI2.img.xz) = 944f6649e22945362338fefddd8bcea8a23ce893d8b16c95b704398deae4f57b o 12.1-BETA3 armv7 PANDABOARD: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-PANDABOARD.img.xz) = b74a839e389ec2c22c7b43b1ab5cff1d957836009d25326f57be49608f0e4048ab912b474ff921f366fefef3aefca9091af65df7a762c12f71e807b2dcbd02de SHA256 (FreeBSD-12.1-BETA3-arm-armv7-PANDABOARD.img.xz) = dd2a825aaa1f3ae1b5b2c025f89925222f65358529d9ebfe8248d787c9af888d o 12.1-BETA3 armv7 WANDBOARD: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-WANDBOARD.img.xz) = 1ff0a71e4cd4a84c03ef6cd6b65b2d15ed937f28956f22b9ad5443a69625f01d51aa6afcc7f9b24f521a4c6e8f04016b0a44acc266bac29ef480d7bc70803914 SHA256 (FreeBSD-12.1-BETA3-arm-armv7-WANDBOARD.img.xz) = b255b832e458871e717afa592a8269c4c9fb2454b829ccc5451c8e5ceb362c0a o 12.1-BETA3 armv7 GENERICSD: SHA512 (FreeBSD-12.1-BETA3-arm-armv7-GENERICSD.img.xz) = cc9d907316a0f6ec949f2ac90d92e5a9bfdc6884cb519b83a87d6e29cd8a843ba07a7d64e60e3ca15d8cb616d963125a2fd20d9b0f4a1c21068f8c7d342a1326 SHA256 (FreeBSD-12.1-BETA3-arm-armv7-GENERICSD.img.xz) = fd32b855275e5d541af0832f63f9f41c9eccc90f89e7d63c939e8a3df4196abe o 12.1-BETA3 aarch64 GENERIC: SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64-memstick.img) = 5a04be5c40411a83233b5d473aeb1000999184e3e910b256a916122d762315a878c8178df2907690c6a2de024414ee97946c38da8f4cdf5f6b3a9a1a99b3d2b3 SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64-memstick.img.xz) = f4d8771bdd47069e79c4e23fd59bab7720540e0cc51efebd922bb67d1d2eb188a0da6a80f96030f4a9b3b8f7bcbd3f25d16113a4351024f4bb94cb8977e0bf29 SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64-mini-memstick.img) = 5e938ed2e5ca3f33f26e91630fc96229bd7985bda1a9e736ec202ee1235d3df0c078c67694dc0266657d162d596a786fc8e6a5a97ac124daea472c35f93d6e06 SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64-mini-memstick.img.xz) = cf19e7adf8f5d1f2366579b3905353ef55bfe15c34adbe01ce4bed89f771e7d0b557b4acaa57536362546bfb539572541b83c0145516e9a2f6602a77e63ed225 SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64-memstick.img) = bb22cb94f97d527582f8d4d129f89913a5db099d5e918992d7bf7ba19d5174cd SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64-memstick.img.xz) = fdbbc449d78072723b0979e894267f96471ce523b1030b35ce5f89afc69b9505 SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64-mini-memstick.img) = 446d820e253f18bba5f56379186bb923a48847ff05631223513b46d2d7b0b7ed SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64-mini-memstick.img.xz) = 5e3385e29b1734ae0ccd1231bcdbc00aae254a0da06a7f2edf8560a63a663786 o 12.1-BETA3 aarch64 RPI3: SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64-RPI3.img.xz) = 6fdbff949f88d74bc815d5e58660c729a24bcde5109ee70b7ef39e679f6d6d44c843387ae861dbb1b80e9a8e5582f964b0cb1bae73e06cc9b4f9c752982d0251 SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64-RPI3.img.xz) = 003d8df1060f5115f95897aa41ee380b9f00e2fe0d8822682dc449e100daae35 o 12.1-BETA3 aarch64 PINE64: SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64-PINE64.img.xz) = e69ab83119ca222b5e5e51ea2efd68363b5f2181fd998e6831634ce2d7983286c022165febd0634dfd27958992e9a8c3fa37d72100d2b49dc32c53c8463af66b SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64-PINE64.img.xz) = 8442cb648d40db0abfa940f84c66072f0c3aa71e8e4b889cafb7996e9c73078d o 12.1-BETA3 aarch64 PINE64-LTS: SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64-PINE64-LTS.img.xz) = 970aa2819465f8716f715dbd6b1123757a82ae0e062fa6d180ebeeb5623f7698ce8af98ac45e59262881e8434e943030fc04ab259502fa946a9ae5f4a113ce8e SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64-PINE64-LTS.img.xz) = 3913c8d7a08089b951dc622643f215d99569919c47f90814967ce1cb01917547 == VM IMAGE CHECKSUMS == o 12.1-BETA3 amd64: SHA512 (FreeBSD-12.1-BETA3-amd64.qcow2.xz) = 0b4810463baacab3c417ba4a1b17426f3c8c1bdd9b994bb54cf8a82b4c378db9825dd74e17db8731c8a2a87dd5e4d021c71f4b4f8ca7c18eb376b279e1516160 SHA512 (FreeBSD-12.1-BETA3-amd64.raw.xz) = 9f4b279f05251449b960ef452d717938c6dac9dda5520b203be1f770259bb9df687b11ffc96067f419bcc0011f9726f9aa57676d6d34ddc522c1a9827569e14a SHA512 (FreeBSD-12.1-BETA3-amd64.vhd.xz) = cdeefc9c962b244b9d81c135c9f4441da1e894d98ae887b588f3b56db8fe40ddaa59675eb5790391e41543eee809dd66ff9a3b44efbe0ab456ccb8db8a1605d0 SHA512 (FreeBSD-12.1-BETA3-amd64.vmdk.xz) = f4b0fff2d405e366963f64db13bdad3e2c2243fa107a99ecc2081df8674af651cd35b75303cbb3a0f9aec277587a5ac35c2459deb51fc8ea87d47879ca4c0c98 SHA256 (FreeBSD-12.1-BETA3-amd64.qcow2.xz) = 34ec582d0c18ec4d0420889b3bdd8ad113c11e74c0de39ecb277e2017a7df379 SHA256 (FreeBSD-12.1-BETA3-amd64.raw.xz) = b88b5faec7f028e3bafabf1f41541f44ea9d295dca13dc50ecc7fa88c56f2bd3 SHA256 (FreeBSD-12.1-BETA3-amd64.vhd.xz) = 9943c4e5a8efd2c86e56d1c9551f35d71544ca5632df332fa51628716f0bc331 SHA256 (FreeBSD-12.1-BETA3-amd64.vmdk.xz) = 456becb5f3e078a8bf9fdaf8880b7dacdd2caeda644767de0f73a2663872c841 o 12.1-BETA3 i386: SHA512 (FreeBSD-12.1-BETA3-i386.qcow2.xz) = a3f24b2efdbb53af4b8f5d2e6309bfee694f28ba7dce2fc7bf363df2a6ec67c10c886dea4406cc4fb7f93dd7c3f3dd97bcf21ef455771f7b1fa549cc6f03b8f2 SHA512 (FreeBSD-12.1-BETA3-i386.raw.xz) = deec1191e64d7b6ced2129bc8d5ccc5e8e6c721c37559eddbc76504f8b1d7268649cd966ba98f09ba96f361a2e2cc2e7a8f384d900634f0c69caf79e66429e2a SHA512 (FreeBSD-12.1-BETA3-i386.vhd.xz) = 6efa2df596a34e7518bda6c524e0fc96512660125a90bb8b452ea57009e1b5e273a4124dbff6d7f13311997dc764c735b70545b363b70677eb32c60dfee20fb9 SHA512 (FreeBSD-12.1-BETA3-i386.vmdk.xz) = 03b853fae009344610e2c3ef77e247b669bd47244fe2440e57925d289184aaa1dcb886eb02e47cd68c2866c0c3bb64bf39e4856333d031a4ec007afda2f51792 SHA256 (FreeBSD-12.1-BETA3-i386.qcow2.xz) = 3585e7e17aadb87160ba2e093b0ff5869695805ecad1e26281ab9888ab91f5a3 SHA256 (FreeBSD-12.1-BETA3-i386.raw.xz) = 5406e6b510a76ebd71bca47eccfae68be05368b6f24a9548837c3cc32f6a6cfc SHA256 (FreeBSD-12.1-BETA3-i386.vhd.xz) = 3b9a771fbcdb637ffa7c233a148099c231e6ca5bb5618b1839cc73b1ec93ec13 SHA256 (FreeBSD-12.1-BETA3-i386.vmdk.xz) = 071c3bb7f73f2f2cb0c39b7a09bbab0f7409da0c4c1e87535ac6cb8f2373e98a o 12.1-BETA3 aarch64: SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64.qcow2.xz) = 08ae23f7cb18084de7bc0b93e5d5ef70896100732508a86b07bfaeb2bb05691dbda2f9f18dccce34e5fd8539423070ef9c2bef87594486b52d73a160c144127a SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64.raw.xz) = de834cf1bcc9e85e524008274edb28e46014423940cb004a206980f48216973c5f3d7e254c11f990e9921774974bcf421e5131806071dbab6795e42b2ce1676c SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64.vhd.xz) = 2a120dfee1d5a0906f7132a99c994c213690a85432d543a45015e4256041d67db4fee2df1c5bea918776077a01f5d5bc815a960f8c8ef0f520bd25aca49eff3a SHA512 (FreeBSD-12.1-BETA3-arm64-aarch64.vmdk.xz) = 2ad95b1332d5e593cd3faf32c15c8da9a3b43b62416db6e9045babb53943d96c72225fb1a808d0b7dc32e536768ea09d43340ef6069998355294a4fd312ebdb0 SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64.qcow2.xz) = 1be7959f2b443615e20ab315949188821f63cbf9ce3b319e2e65cb06ba818c87 SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64.raw.xz) = dd78843d1954e17d16bfd1a72e5427c7c3381dd97d611891c19e502862460124 SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64.vhd.xz) = 2741b008834552e18349d06f0eff047cd9e8a947a0f702ea589b18dbce977904 SHA256 (FreeBSD-12.1-BETA3-arm64-aarch64.vmdk.xz) = e2d00062ce8dbe2b4488cca49000ee890b0a14251387a120c91ab15a56b6ed2c Regards, Glen Brought to you by Rubicon Communications, LLC (Netgate). Love FreeBSD? Support this and future releases with a donation to the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAl2XtDMACgkQAxRYpUeP 4pOmqA/9GOYW7P/GO4zdWdSoSTO11BE1KKfND1/nXK5rwU1b5Xe243IEckT68jcJ xk9urwlJtsr7ME1DbokrU6/zouWIzzhbqzixzB5PdlgrmqBIDGr6dArJEWrG5Slv mxi2Nu2v/iC1X0VX5ZUD5+5sgk4nHRPn42VxwwlV431BTANoM5r0BiODbGtlIHAj 4Ou0+8h6aQaUiq8aKsfdvfumCyo+jH6hgrowNAYNsdM8AkG0AtzFk1xj5ZHUx9q6 L+qGOhj21TIM82K/gnt4Oy104Z0IfHpmg/TheeAFjrovuI2bhG79IWW49Z8sb5SN eEwQbMiVM/DgCbLyE2X5105CRkRxG5xhR1vVQ/o7Y3F/suYvyZ9FR2AONob7HXqv SkBPuWrDFD0E/tJUcRsT1xCl7e7JOFFWHS/IuFBjwGEfuP2rmISehNj9LrBMiRch 7rH2qDDlUv+PjTBLWx/Ezq5c/qmfynGi9EAeRDDBN4QWNvnnpQIMLU8y9jVd5IkN QZTHk8Hh6OS2E6vLhbZCGTUki3Tl9yJ2GPh7JfZqQrRXZKe3rfjNNdQrZQZ9wKwN eJ8i0IOzlDbyv03UjaxxZSR4Tg73mXJYF3n+220VFnMJ5hL7uG0mTn4IOB4QnelP LqM8IpccXCaKBJAA3qHO21JMra3XFJnGgMJQpb58MX56WJcANlg= =ZBTs -----END PGP SIGNATURE----- From owner-freebsd-stable@freebsd.org Sat Oct 5 05:40:21 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B373EFFB93; Sat, 5 Oct 2019 05:40:21 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46lbDj4HWzz4Kt9; Sat, 5 Oct 2019 05:40:21 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1129) id 8A92A14F46; Sat, 5 Oct 2019 05:40:21 +0000 (UTC) Date: Sat, 5 Oct 2019 05:40:21 +0000 From: Li-Wen Hsu To: freebsd-testing@freebsd.org Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: FreeBSD CI Weekly Report 2019-09-29 Message-ID: <20191005054021.GA73816@freefall.freebsd.org> Reply-To: freebsd-testing@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.11.4 (2019-03-13) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Oct 2019 05:40:21 -0000 (Please send the followup to freebsd-testing@ and note Reply-To is set.) FreeBSD CI Weekly Report 2019-09-29 =================================== Here is a summary of the FreeBSD Continuous Integration results for the period from 2019-09-23 to 2019-09-29. During this period, we have: * 2159 builds (98.6% (-0.4) passed, 1.4% (+0.4) failed) of buildworld and buildkernel (GENERIC and LINT) were executed on aarch64, amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, sparc64 architectures for head, stable/12, stable/11 branches. * 340 test runs (71.5% (-5.7) passed, 20% (-2.8) unstable, 8.5% (+8.5) exception) were executed on amd64, i386, riscv64 architectures for head, stable/12, stable/11 branches. * 22 doc builds (100% passed) Test case status (on 2019-09-29 23:59): | Branch/Architecture | Total | Pass | Fail | Skipped | | ------------------- | ---------- | ---------- | ------ | -------- | | head/amd64 | 7588 (+21) | 7525 (+21) | 0 (0) | 63 (0) | | head/i386 | 7586 (+21) | 7514 (+21) | 0 (0) | 72 (0) | | 12-STABLE/amd64 | 7474 (0) | 7430 (0) | 0 (0) | 44 (0) | | 12-STABLE/i386 | 7472 (0) | 7421 (0) | 0 (0) | 51 (+3) | | 11-STABLE/amd64 | 6849 (0) | 6805 (0) | 0 (0) | 44 (0) | | 11-STABLE/i386 | 6847 (0) | 6767 (-3) | 34 (0) | 46 (+3) | (The statistics from experimental jobs are omitted) If any of the issues found by CI are in your area of interest or expertise please investigate the PRs listed below. The latest web version of this report is available at https://hackmd.io/@FreeBSD-CI/report-20190929 and archive is available at https://hackmd.io/@FreeBSD-CI/, any help is welcome. ## News * [FCP 20190401-ci_policy: CI policy](https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md) is in "feedback" state, please check and provide comments on -fcp@ and -hackers@ mailing lists. ## Fixed Tests * lib.libc.sys.mmap_test.mmap_truncate_signal * https://svnweb.freebsd.org/changeset/base/352807 * https://svnweb.freebsd.org/changeset/base/352869 ## Failing Tests * https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/ * local.kyua.* (31 cases) * local.lutok.* (3 cases) ## Failing and Flaky Tests (from experimental jobs) * https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/ * cddl.usr.sbin.dtrace.common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237641 * cddl.usr.sbin.dtrace.amd64.arrays.t_dtrace_contrib.tst_uregsarray_d * https://bugs.freebsd.org/240358 * Fixed in head https://svnweb.freebsd.org/changeset/base/353107 * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/ * There are ~60 failing cases, including flakey ones, see https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/ for more details ## Disabled Tests * sys.fs.tmpfs.mount_test.large https://bugs.freebsd.org/212862 * sys.fs.tmpfs.link_test.kqueue https://bugs.freebsd.org/213662 * sys.kqueue.libkqueue.kqueue_test.main https://bugs.freebsd.org/233586 * sys.kern.ptrace_test.ptrace__PT_KILL_competing_stop https://bugs.freebsd.org/220841 * lib.libc.regex.exhaust_test.regcomp_too_big (i386 only) https://bugs.freebsd.org/237450 * sys.netinet.socket_afinet.socket_afinet_bind_zero (new) https://bugs.freebsd.org/238781 * sys.netpfil.pf.names.names * sys.netpfil.pf.synproxy.synproxy https://bugs.freebsd.org/238870 * sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger https://bugs.freebsd.org/239292 * sys.netpfil.pf.forward.v4 (i386 only) * sys.netpfil.pf.forward.v6 (i386 only) * sys.netpfil.pf.set_tos.v4 (i386 only) https://bugs.freebsd.org/239380 * sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger https://bugs.freebsd.org/239397 * sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger https://bugs.freebsd.org/239399 * sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger https://bugs.freebsd.org/239425 * lib.libc.gen.getmntinfo_test.getmntinfo_test https://bugs.freebsd.org/240049 * sys.sys.qmath_test.qdivq_s64q https://bugs.freebsd.org/240219 * sys.kern.ptrace_test.ptrace__getppid https://bugs.freebsd.org/240510 * lib.libc.sys.stat_test.stat_socket https://bugs.freebsd.org/240621 * sys.netpfil.common.tos.pf_tos (i386 only) https://bugs.freebsd.org/240086 * lib.libarchive.functional_test.test_write_filter_zstd https://bugs.freebsd.org/240683 ## Issues ### Cause build fails * https://bugs.freebsd.org/233735 Possible build race: genoffset.o /usr/src/sys/sys/types.h: error: machine/endian.h: No such file or directory * https://bugs.freebsd.org/233769 Possible build race: ld: error: unable to find library -lgcc_s ### Cause kernel panics * https://bugs.freebsd.org/238870 sys.netpfil.pf.names.names and sys.netpfil.pf.synproxy.synproxy cause panic Patch exists: * https://reviews.freebsd.org/D20868 * https://reviews.freebsd.org/D20869 ### Open * https://bugs.freebsd.org/237403 Tests in sys/opencrypto should be converted to Python3 * https://bugs.freebsd.org/237641 Flakey test case: common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237656 "Freed UMA keg (rtentry) was not empty (18 items). Lost 1 pages of memory." seen when running sys/netipsec tests * https://bugs.freebsd.org/238781 sys.netinet.socket_afinet.socket_afinet_bind_zero does not work when mac_portacl(4) loaded * https://bugs.freebsd.org/239292 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger * https://bugs.freebsd.org/239380 sys.netpfil.pf.forward.{v4,v6} and sys.netpfil.pf.set_tos.v4 fail on i386 * https://bugs.freebsd.org/239397 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger * https://bugs.freebsd.org/239399 Flakey test case: sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger * https://bugs.freebsd.org/239425 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger * https://bugs.freebsd.org/240085 Failing test: sys.netpfil.common.forward.pf_v4 on i386 * https://bugs.freebsd.org/240086 Failing test: sys.netpfil.common.tos.pf_tos on i386 ### Others * [Tickets related to testing@](https://preview.tinyurl.com/y9maauwg) From owner-freebsd-stable@freebsd.org Sat Oct 5 09:15:05 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A27B712D331; Sat, 5 Oct 2019 09:15:05 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [195.149.99.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "raven.bwct.de", Issuer "raven.bwct.de" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46lh0T33nPz4VlZ; Sat, 5 Oct 2019 09:15:04 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.15.2/8.15.2) with ESMTPS id x959FcBY051755 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Sat, 5 Oct 2019 11:15:39 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cicely.de; s=default; t=1570266940; bh=ea+lUXaG9fOO7saG4krqNkaj44ZHf2nUDmOAZJyZhJw=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To; b=kkYX3QSjd06F+AmNfhMUC9ZRpf/V3iQ9Xn+cvQFudcsLlFQhgBiqohelEcgZ8PyAv 7x9VQUj1gDzJbNIuA3Cy2zkxqr6T36PG7C0L3FoZbMiIkYuOZ1RaPlge0kT+1Yq+Dn BXeu/1sM4KFagHrofqFyl9xsb6DrTcC6uQ5sotpE= Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id x959EvAs051387 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 5 Oct 2019 11:14:57 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.15.2/8.15.2) with ESMTP id x959EvhU075809; Sat, 5 Oct 2019 11:14:57 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.15.2/8.15.2/Submit) id x959Ev0p075808; Sat, 5 Oct 2019 11:14:57 +0200 (CEST) (envelope-from ticso) Date: Sat, 5 Oct 2019 11:14:57 +0200 From: Bernd Walter To: Andriy Gapon Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-fs@freebsd.org Subject: Re: ZFS with 32-bit, non-x86 kernel Message-ID: <20191005091457.GI31607@cicely7.cicely.de> Reply-To: ticso@cicely.de References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD cicely7.cicely.de 12.0-STABLE amd64 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=4.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de X-Rspamd-Queue-Id: 46lh0T33nPz4VlZ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Oct 2019 09:15:05 -0000 On Fri, Oct 04, 2019 at 05:05:25PM +0300, Andriy Gapon wrote: > > Does anyone use ZFS with a 32-bit kernel, that is also not i386 ? > If you do, could you please let me know? Along with uname -rmp output. > Thank you! [51]wb1# uname -rmp 12.0-RELEASE arm armv7 It is a wandboard qaud with iMX6 and 2G RAM. I'm using two uSD cards with zroot. I'm also using the same setup on a raspberry Pi1: [13]time1# uname -rmp 12.0-RELEASE arm armv6 But 512MB RAM are to low for zroot. It technically does work, but hoks up the CPU in arc_reclaim_thread when a scrub runs and since it takes forever it always runs. last pid: 80786; load averages: 6.56, 5.86, 5.75 up 185+05:51:55 11:10:04 372 threads: 5 running, 349 sleeping, 18 waiting CPU: 0.1% user, 0.0% nice, 86.9% system, 3.8% interrupt, 9.2% idle Mem: 4960K Active, 38M Inact, 128M Wired, 259M Free ARC: 24M Total, 8591K MFU, 8896K MRU, 34K Anon, 251K Header, 6888K Other 2676K Compressed, 74M Uncompressed, 28.31:1 Ratio Swap: PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND 8 root -8 - 0 96K arc_re 1597.2 53.94% zfskern{arc_reclaim_thread} 10 root 155 ki31 0 8192 RUN 1765.3 9.26% idle 0 root -8 - 0 2064K - 75.7H 2.63% kernel{dp_sync_taskq} 11 root -80 - 0 144K WAIT 67.8H 1.88% intr{intc0,28: bcm_dma0} 20 root -8 - 0 8192 mmcreq 39.0H 1.31% mmcsd0: mmc/sd card 12 root -8 - 0 24K - 27.4H 0.87% geom{g_down} 11 root -60 - 0 144K WAIT 28.6H 0.74% intr{swi4: clock (0)} 11 root -88 - 0 144K WAIT 21.1H 0.66% intr{intc0,70: +} 12 root -8 - 0 24K - 19.3H 0.64% geom{g_up} [16]time1# zpool status pool: zroot state: ONLINE scan: scrub in progress since Mon Jun 10 03:58:19 2019 34.0G scanned at 3.52K/s, 867M issued at 89/s, 1.77G total 0 repaired, 47.78% done, no estimated completion time config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 diskid/DISK-0000081Cs2a ONLINE 0 0 0 da0s2a ONLINE 0 0 0 errors: No known data errors -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.