From owner-freebsd-current@freebsd.org  Sun Sep  8 22:03:13 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5F63E4C9D
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sun,  8 Sep 2019 22:03:13 +0000 (UTC)
 (envelope-from clhamilto@gmail.com)
Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com
 [IPv6:2607:f8b0:4864:20::82c])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RQKD0wP9z4BS0
 for <freebsd-current@freebsd.org>; Sun,  8 Sep 2019 22:03:11 +0000 (UTC)
 (envelope-from clhamilto@gmail.com)
Received: by mail-qt1-x82c.google.com with SMTP id l22so13872902qtp.10
 for <freebsd-current@freebsd.org>; Sun, 08 Sep 2019 15:03:11 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=subject:references:to:from:message-id:date:user-agent:mime-version
 :in-reply-to:content-language;
 bh=m4Sra4RNGt8sikLxxC6a17KbXN6WPyDa/itdJkosKSc=;
 b=OGWSFg6avt3L0rZi12OJRpXR9oAdGy9cdPURIFzeW0O6AtXDHUKjgz7FD4We2VsGnr
 ZS40coktkWjqzVZnxuCtIROB8dycpuEJ7NCHe5RwP5l024SnH/PHGa51tar+tsJbsL6P
 AayvRNk2ibVN336kShsQM2W7pvOz5/7HnpEjLaj5eMD4itkY51w3cCxPFJg2SR+iNOiI
 +JBnp01F/cSmMk+WVL3CMk/U8ZqIZ26Jp4WrRTgOrUvvqReBkFtE1giPmgWe4RFNANSs
 WvfC2wxftyvTZtjj31/ML4QxMG6VnPRmbm3NamZhJHsat/n/Izo9ObZ1uW1j0A6qKMT8
 52Ig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:subject:references:to:from:message-id:date
 :user-agent:mime-version:in-reply-to:content-language;
 bh=m4Sra4RNGt8sikLxxC6a17KbXN6WPyDa/itdJkosKSc=;
 b=m6ZM/F85oGz9xvJ3VDaLQK1FL8eaAhevRpd8HC2I6lXtCEzQr3vyUTBziN1txmqvh1
 H9wVXaYdVHACK1LEIa/cMTbTikj2y4jbZS45KoDgW9WbNOZ6URiegiiOVR/nG746F2uh
 DOPhn8Oj2TK48b1XYwf5E/NljHz1zk8fUMUxxK2J1uILuNwSd9YIzaXHDbgZc+GJlN2H
 ePqRbeKcu4reUNg+pVoHjf0AJkZVXGHAuQvm0F5/iFNZZet7UgN0ITEGo/it/FpHkRj1
 noRUCcDmuRdTWmVAzGqX2+CkgQPkBFvIN9EKJgbAjSuccbtuxhnuMty+yj0nNAXn/4al
 Q7+Q==
X-Gm-Message-State: APjAAAUQxEFr6LsIunnOUUIzIfcuKPuk/OFtwaZP58WDzogJe9FEJFYQ
 p1VvRUJW7/k2teos5WhE8alZGzka
X-Google-Smtp-Source: APXvYqxMLImzgazZBT9h5E56ffu794avfGNwCjf3m8DDfamQCr9PiZbdIaCmFLMwv04bKr+mt9Gv9g==
X-Received: by 2002:ac8:5357:: with SMTP id d23mr12246069qto.266.1567980191041; 
 Sun, 08 Sep 2019 15:03:11 -0700 (PDT)
Received: from lenoil8.lenoil.net (c-76-106-45-221.hsd1.va.comcast.net.
 [76.106.45.221])
 by smtp.gmail.com with ESMTPSA id 60sm6044382qta.77.2019.09.08.15.03.10
 for <freebsd-current@freebsd.org>
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Sun, 08 Sep 2019 15:03:10 -0700 (PDT)
Subject: Panic with Current 351901 ISO image
References: <D970EBBF-E1B5-4CEA-98DD-B9C661F894B1@verizon.net>
To: freebsd-current@freebsd.org
From: Curtis Hamilton <clhamilto@gmail.com>
X-Forwarded-Message-Id: <D970EBBF-E1B5-4CEA-98DD-B9C661F894B1@verizon.net>
Message-ID: <01378a14-471d-31b4-c60a-98a7a059748b@gmail.com>
Date: Sun, 8 Sep 2019 18:03:09 -0400
User-Agent: Mozilla/5.0 (X11; FreeBSD powerpc; rv:52.0) Gecko/20100101
 Thunderbird/52.8.0
MIME-Version: 1.0
In-Reply-To: <D970EBBF-E1B5-4CEA-98DD-B9C661F894B1@verizon.net>
Content-Language: en-US
X-Rspamd-Queue-Id: 46RQKD0wP9z4BS0
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=OGWSFg6a;
 dmarc=pass (policy=none) header.from=gmail.com;
 spf=pass (mx1.freebsd.org: domain of clhamilto@gmail.com designates
 2607:f8b0:4864:20::82c as permitted sender) smtp.mailfrom=clhamilto@gmail.com
X-Spamd-Result: default: False [-3.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36];
 FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[];
 RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+];
 DMARC_POLICY_ALLOW(-0.50)[gmail.com,none];
 RECEIVED_SPAMHAUS_PBL(0.00)[221.45.106.76.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net
 : 127.0.0.10]; FROM_EQ_ENVFROM(0.00)[];
 MIME_TRACE(0.00)[0:+,1:+,2:~];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1];
 IP_SCORE(0.00)[ip: (-9.28), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27),
 country: US(-0.05)]; 
 RCVD_IN_DNSWL_NONE(0.00)[c.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; RCVD_TLS_ALL(0.00)[]
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 08 Sep 2019 22:03:14 -0000

I'm encountering (randomly) the below error when trying to install 
351901 from CD/DVD.


timeout stopping cpus
panic: acquiring blockable sleep lock with spinlock or critical section 
held (sleep mutex) vm map (system) @ /usr/src/sys/vm/vm_map.c:4066
cpuid = 0
time = 1
KDB: stack backtrace:
0xe00000008879d4f0: at .kdb_backtrace+0x5c
0xe00000008879d620: at .vpanic+0x1b4
0xe00000008879d6e0: at .panic+0x38
0xe00000008879d770: at .witness_checkorder+0xf4
0xe00000008879d860: at .__mtx_lock_flags+0xfc
0xe00000008879d910: at ._vm_map_lock_read+0x34
0xe00000008879d990: at .vm_map_lookup+0x94
0xe00000008879dad0: at .vm_fault_hold+0x158
0xe00000008879dcf0: at .vm_fault+0x94
0xe00000008879dda0: at .cpu_fetch_syscall_args+0x404
0xe00000008879de50: at .trap+0xf28
0xe00000008879e010: at .powerpc_interrupt+0x290
0xe00000008879e0b0: kernel DSI read trap @ 0x392 by 
.sched_relinquish+0x290: srr1=0x9000000000001032
r1=0xe00000008879e360 cr=0x20009032 xer=0 ctr=0xc000000002ea198c 
r2=0xc00000000393f7e0 sr=0x40000000
0xe00000008879e360: at .sched_tdname+0xbc
0xe00000008879e3f0: at .sched_switch+0x414
0xe00000008879e530: at .mi_switch+0x2d4
0xe00000008879e5c0: at .sched_bind+0xd8
0xe00000008879e650: at .taskqgroup_config_gtask_init+0x108
0xe00000008879e700: at .gtaskqueue_cancel+0x2ac
0xe00000008879e7c0: at .taskqgroup_adjust+0xb0c
0xe00000008879e850: at .fork_exit+0xd0
0xe00000008879e8f0: at .fork_trampoline+0x10
0xe00000008879e920: at -0x4
KDB: enter: panic
[ thread pid 0 tid 100144 ]
Stopped at .kdb_enter+0x60:        ld      r2, r1, 0x28
db>

From owner-freebsd-current@freebsd.org  Sun Sep  8 22:59:57 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 28B8DE6117
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sun,  8 Sep 2019 22:59:57 +0000 (UTC)
 (envelope-from clay.daniels.jr@gmail.com)
Received: from mail-vk1-xa2d.google.com (mail-vk1-xa2d.google.com
 [IPv6:2607:f8b0:4864:20::a2d])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RRZh0zvRz4DDM
 for <freebsd-current@freebsd.org>; Sun,  8 Sep 2019 22:59:55 +0000 (UTC)
 (envelope-from clay.daniels.jr@gmail.com)
Received: by mail-vk1-xa2d.google.com with SMTP id t136so2335231vkt.9
 for <freebsd-current@freebsd.org>; Sun, 08 Sep 2019 15:59:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:from:date:message-id:subject:to;
 bh=1gjYTLptPfjmAxzBg3/wHMYmTZrxSbmmoXk1938i4Q0=;
 b=aYRf9+NyW6/0IZ8AUYSBiWUszGDwYuuHPAFjEDgxHEqj7FJRxURGFwVMk+y+zj/BpJ
 YZKVGxQzHpSjBtioMhRRfqOIVfdq9xGTbDMP3GiJ6AT5+fhlzBf4eX89DjN7a6HqYU9c
 7HZtZB/IdYNIjUzSsbs7HO5YjFnWG4kCbWL65EMAA5RXo8jbKVoViYNOAbpFaa+FW+Ai
 f0z3SlvYRRZX3v6kk3S0HXvq3rqBuJdrh/o3r/PNcrGz0le0bPQTemP1QIP0CQc3z5PW
 hpxI6XYocM3PjhETqGxSqsRVwb73dijQnInvUWyinqusy8vBvq3w4pYOl9IWr7Evwnu9
 fdKg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=1gjYTLptPfjmAxzBg3/wHMYmTZrxSbmmoXk1938i4Q0=;
 b=fqOp0/irDy6tK4BlNMJYNfRY4vdO4IOfbESkenYBXZTg+Ufd+Wd3y0O9f6jGgNVVMa
 +yyW3XBksBSUbEC/iwTXEW6DJBO9K361/DxHvwNKUUZCgePFUtXRKdQkuPTRajMGafe9
 g1TJX8cH4sy1uuuo1oHclW3cn5RQEXE5mS9VRnkS2dQTZcNVzd8LjoeSc+f1UPQi46jt
 9rjYedMbsuVMIpMHR1MkDLcG2Yhjdrzw4vD+vk63qgtcTiH0mid7dPC/7Zar3YKG7yXO
 42QUwNgxX3RLLEGiV2SJ2CnnoJE+p2xZY2F6n/DnMaEzLMNMtgU3omhrQDS0WvW0GWXC
 NSIA==
X-Gm-Message-State: APjAAAWDG9NTHHYpalsoMSrU5V26vTq+NUp3f/6EL+8RCwlkC8wBT5g7
 NzEnzGjitCXAWvPoX7hcdsHO2xBIFSCfXZSw0K8A5Ow=
X-Google-Smtp-Source: APXvYqwIeSLSlGPha7T9K9Zxpg2T2XlP6r8SYPYxwb9IHLhIcV8K1h7onUhY8fP05cIlpLMB5tCZ2CCWwE3Gs1iK3Qs=
X-Received: by 2002:a1f:78c5:: with SMTP id t188mr9490715vkc.87.1567983593979; 
 Sun, 08 Sep 2019 15:59:53 -0700 (PDT)
MIME-Version: 1.0
From: "Clay Daniels Jr." <clay.daniels.jr@gmail.com>
Date: Sun, 8 Sep 2019 17:59:41 -0500
Message-ID: <CAGLDxTXT4Dh4D_FuFNoMgsqNTbx03sY=yYAPixpddqim3P_LpA@mail.gmail.com>
Subject: fusefs & ntfs-3g
To: "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>
X-Rspamd-Queue-Id: 46RRZh0zvRz4DDM
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=aYRf9+Ny;
 dmarc=pass (policy=none) header.from=gmail.com;
 spf=pass (mx1.freebsd.org: domain of claydanielsjr@gmail.com designates
 2607:f8b0:4864:20::a2d as permitted sender)
 smtp.mailfrom=claydanielsjr@gmail.com
X-Spamd-Result: default: False [-2.00 / 15.00];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c];
 FREEMAIL_FROM(0.00)[gmail.com]; URI_COUNT_ODD(1.00)[3];
 DKIM_TRACE(0.00)[gmail.com:+];
 DMARC_POLICY_ALLOW(-0.50)[gmail.com,none];
 FROM_EQ_ENVFROM(0.00)[];
 IP_SCORE(0.00)[ip: (-9.25), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27),
 country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 TAGGED_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1];
 RCVD_IN_DNSWL_NONE(0.00)[d.2.a.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; TO_DN_EQ_ADDR_ALL(0.00)[];
 RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 08 Sep 2019 22:59:57 -0000

I want to view my Windows C: drive on ata0p4.
This is what I have:
FreeBSD bsd13 13.0-CURRENT FreeBSD 13.0-CURRENT r351901 GENERIC  amd64
clay@bsd13:~ % pkg info -r fusefs-libs
fusefs-libs-2.9.9:
clay@bsd13:~ % pkg info -r fusefs-libs3
fusefs-libs3-3.6.2:
clay@bsd13:~ % cat /boot/loader.conf
hw.syscons.disable=1
fusefs_load="YES"
clay@bsd13:~ % kldstat
Id Refs Address                Size Name
 1   72 0xffffffff80200000  2336f80 kernel
 2    1 0xffffffff82538000    27990 fusefs.ko
 3    1 0xffffffff82720000   2519c4 amdgpu.ko
 4    2 0xffffffff82972000    77bd0 drm.ko
 5    5 0xffffffff829ea000    12470 linuxkpi.ko
 6    3 0xffffffff829fd000    12f30 linuxkpi_gplv2.ko
 7    2 0xffffffff82a10000      8e0 lindebugfs.ko
 8    1 0xffffffff82a11000     f281 ttm.ko
 9    1 0xffffffff82a21000     23f7 radeon_kabini_pfp_bin.ko
10    1 0xffffffff82a24000     23f5 radeon_kabini_me_bin.ko
11    1 0xffffffff82a27000     23f5 radeon_kabini_ce_bin.ko
12    1 0xffffffff82a2a000     43f7 radeon_kabini_mec_bin.ko
13    1 0xffffffff82a2f000     2a77 radeon_kabini_rlc_bin.ko
14    1 0xffffffff82a32000     12e9 radeon_kabini_sdma_bin.ko
15    1 0xffffffff82a34000     12eb radeon_kabini_sdma1_bin.ko
16    1 0xffffffff82a36000    38ea7 radeon_kabini_uvd_bin.ko
17    1 0xffffffff82a6f000    18c47 radeon_kabini_vce_bin.ko
18    1 0xffffffff82a88000     2538 intpm.ko
19    1 0xffffffff82a8b000      a50 smbus.ko
20    1 0xffffffff82a8c000     1820 uhid.ko
21    1 0xffffffff82a8e000     2928 ums.ko
22    1 0xffffffff82a91000     1b00 wmt.ko
23    1 0xffffffff82a93000      acf mac_ntpd.ko
24    1 0xffffffff82a94000     a9f8 tmpfs.ko

clay@bsd13:~ % pkg info fusefs-ntfs-2017.3.23
fusefs-ntfs-2017.3.23
Name           : fusefs-ntfs
Version        : 2017.3.23
Installed on   : Sun Sep  8 17:24:36 2019 CDT
Origin         : sysutils/fusefs-ntfs
Architecture   : FreeBSD:13:amd64
Prefix         : /usr/local
Categories     : sysutils
Licenses       : GPLv2+
Maintainer     : freebsd@dussan.org
WWW            : https://www.tuxera.com/community/open-source-ntfs-3g/
Comment        : Mount NTFS partitions (read/write) and disk images
Options        :
DOCS           : on
LOCK           : on
UBLIO          : on
Shared Libs required:
libfuse.so.2
libuuid.so.1
libublio.so.1
Shared Libs provided:
libntfs-3g.so.88
Annotations    :
FreeBSD_version: 1300044
repo_type      : binary
repository     : FreeBSD
Flat size      : 1.93MiB
Description    :
The ntfs-3g driver is an open source, freely available read/write NTFS
driver, which provides safe and fast handling of the Windows XP, Windows
Server 2003, Windows 2000, Windows Vista, Windows Server 2008, Windows 7
and Windows 8 NTFS file systems. Almost the full POSIX filesystem
functionality is supported, the major exceptions are changing the file
ownerships and the access rights.

root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt
* Windows is hibernated, refused to mount.
The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
Falling back to read-only mount because the NTFS partition is in an unsafe
state. Please resume and shutdown Windows fully (no hibernation
or fast restarting.

I'm having a roadblock here. Maybe someone knows the answer.

From owner-freebsd-current@freebsd.org  Sun Sep  8 23:10:01 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5DCFFE6587
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sun,  8 Sep 2019 23:10:01 +0000 (UTC)
 (envelope-from asomers@gmail.com)
Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com
 [209.85.167.42])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RRpJ2dzyz4DhG
 for <freebsd-current@freebsd.org>; Sun,  8 Sep 2019 23:10:00 +0000 (UTC)
 (envelope-from asomers@gmail.com)
Received: by mail-lf1-f42.google.com with SMTP id w6so9025830lfl.2
 for <freebsd-current@freebsd.org>; Sun, 08 Sep 2019 16:10:00 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=iU5nryxKj0AfQ2MVWdRu/mZ43Zn9UoK0XQIJAazpRnU=;
 b=WFirmc4Vim8kKyj93wDyfNH4U5/bRpkLNxhaK8+6oWMtqwzOSitWe7paks8bV67r1t
 8aeJYIGpXVvD6qQL8InVyrX97Fkt+3tWGz55Nx691B+RxFW5Ie6A8SeuBGOG07aPYc9w
 pfabyJDu3FjCp9VyQXK5EBG2uVLdx+T9UK2wJMBK/nKInqmzVUqgZ/PXGEw8Aq28It6c
 ZNYwIbHsZSdI3Rw4YIom1yxA/VVlD52z4NnoQB4bdieYAIbLF0yzeOS36IQGkn9UsIyh
 xH8epYooFIlshDzAEeM1n03QgpEgq4TR8x4P7iN/URJzhIrQSNHSHdUZ0WwN+6HM2inb
 ef1Q==
X-Gm-Message-State: APjAAAVVvM46lhnrVjGVuHhMSw5eycQV662Tg48RxpfGNQGVjkP6GiIM
 ins8MBdfiMVWm5B1Ba3q/D/dBJpmrtTHK/Cr+TmJow==
X-Google-Smtp-Source: APXvYqxNYkOzBKZxvHslnIQGiUdzoU2UDfEhvGV/x69tG7xOW5XnaLIIpY5eoQSjQU3zuS6S0ytOSXv4uM1YDLAsvDk=
X-Received: by 2002:ac2:4352:: with SMTP id o18mr14487042lfl.164.1567984198554; 
 Sun, 08 Sep 2019 16:09:58 -0700 (PDT)
MIME-Version: 1.0
References: <CAGLDxTXT4Dh4D_FuFNoMgsqNTbx03sY=yYAPixpddqim3P_LpA@mail.gmail.com>
In-Reply-To: <CAGLDxTXT4Dh4D_FuFNoMgsqNTbx03sY=yYAPixpddqim3P_LpA@mail.gmail.com>
From: Alan Somers <asomers@freebsd.org>
Date: Sun, 8 Sep 2019 17:09:46 -0600
Message-ID: <CAOtMX2jQ9KwbZJYnds+7mqD8-jv_wn3SJsPS4HvS64jJQV3YDw@mail.gmail.com>
Subject: Re: fusefs & ntfs-3g
To: "Clay Daniels Jr." <clay.daniels.jr@gmail.com>
Cc: "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>,
 freebsd@dussan.org
X-Rspamd-Queue-Id: 46RRpJ2dzyz4DhG
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates
 209.85.167.42 as permitted sender) smtp.mailfrom=asomers@gmail.com
X-Spamd-Result: default: False [-2.24 / 15.00]; ARC_NA(0.00)[];
 TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3];
 R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[freebsd.org];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 URI_COUNT_ODD(1.00)[3]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[42.167.85.209.list.dnswl.org : 127.0.5.0];
 TO_DN_SOME(0.00)[];
 IP_SCORE(-1.24)[ip: (-0.55), ipnet: 209.85.128.0/17(-3.32), asn: 15169(-2.27),
 country: US(-0.05)]; FREEMAIL_TO(0.00)[gmail.com];
 FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com];
 R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
 FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com];
 RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 08 Sep 2019 23:10:01 -0000

On Sun, Sep 8, 2019 at 5:00 PM Clay Daniels Jr. <clay.daniels.jr@gmail.com>
wrote:

> I want to view my Windows C: drive on ata0p4.
> This is what I have:
> FreeBSD bsd13 13.0-CURRENT FreeBSD 13.0-CURRENT r351901 GENERIC  amd64
> clay@bsd13:~ % pkg info -r fusefs-libs
> fusefs-libs-2.9.9:
> clay@bsd13:~ % pkg info -r fusefs-libs3
> fusefs-libs3-3.6.2:
> clay@bsd13:~ % cat /boot/loader.conf
> hw.syscons.disable=1
> fusefs_load="YES"
> clay@bsd13:~ % kldstat
> Id Refs Address                Size Name
>  1   72 0xffffffff80200000  2336f80 kernel
>  2    1 0xffffffff82538000    27990 fusefs.ko
>  3    1 0xffffffff82720000   2519c4 amdgpu.ko
>  4    2 0xffffffff82972000    77bd0 drm.ko
>  5    5 0xffffffff829ea000    12470 linuxkpi.ko
>  6    3 0xffffffff829fd000    12f30 linuxkpi_gplv2.ko
>  7    2 0xffffffff82a10000      8e0 lindebugfs.ko
>  8    1 0xffffffff82a11000     f281 ttm.ko
>  9    1 0xffffffff82a21000     23f7 radeon_kabini_pfp_bin.ko
> 10    1 0xffffffff82a24000     23f5 radeon_kabini_me_bin.ko
> 11    1 0xffffffff82a27000     23f5 radeon_kabini_ce_bin.ko
> 12    1 0xffffffff82a2a000     43f7 radeon_kabini_mec_bin.ko
> 13    1 0xffffffff82a2f000     2a77 radeon_kabini_rlc_bin.ko
> 14    1 0xffffffff82a32000     12e9 radeon_kabini_sdma_bin.ko
> 15    1 0xffffffff82a34000     12eb radeon_kabini_sdma1_bin.ko
> 16    1 0xffffffff82a36000    38ea7 radeon_kabini_uvd_bin.ko
> 17    1 0xffffffff82a6f000    18c47 radeon_kabini_vce_bin.ko
> 18    1 0xffffffff82a88000     2538 intpm.ko
> 19    1 0xffffffff82a8b000      a50 smbus.ko
> 20    1 0xffffffff82a8c000     1820 uhid.ko
> 21    1 0xffffffff82a8e000     2928 ums.ko
> 22    1 0xffffffff82a91000     1b00 wmt.ko
> 23    1 0xffffffff82a93000      acf mac_ntpd.ko
> 24    1 0xffffffff82a94000     a9f8 tmpfs.ko
>
> clay@bsd13:~ % pkg info fusefs-ntfs-2017.3.23
> fusefs-ntfs-2017.3.23
> Name           : fusefs-ntfs
> Version        : 2017.3.23
> Installed on   : Sun Sep  8 17:24:36 2019 CDT
> Origin         : sysutils/fusefs-ntfs
> Architecture   : FreeBSD:13:amd64
> Prefix         : /usr/local
> Categories     : sysutils
> Licenses       : GPLv2+
> Maintainer     : freebsd@dussan.org
> WWW            : https://www.tuxera.com/community/open-source-ntfs-3g/
> Comment        : Mount NTFS partitions (read/write) and disk images
> Options        :
> DOCS           : on
> LOCK           : on
> UBLIO          : on
> Shared Libs required:
> libfuse.so.2
> libuuid.so.1
> libublio.so.1
> Shared Libs provided:
> libntfs-3g.so.88
> Annotations    :
> FreeBSD_version: 1300044
> repo_type      : binary
> repository     : FreeBSD
> Flat size      : 1.93MiB
> Description    :
> The ntfs-3g driver is an open source, freely available read/write NTFS
> driver, which provides safe and fast handling of the Windows XP, Windows
> Server 2003, Windows 2000, Windows Vista, Windows Server 2008, Windows 7
> and Windows 8 NTFS file systems. Almost the full POSIX filesystem
> functionality is supported, the major exceptions are changing the file
> ownerships and the access rights.
>
> root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt
> * Windows is hibernated, refused to mount.
> The disk contains an unclean file system (0, 0).
> Metadata kept in Windows cache, refused to mount.
> Falling back to read-only mount because the NTFS partition is in an unsafe
> state. Please resume and shutdown Windows fully (no hibernation
> or fast restarting.
>
> I'm having a roadblock here. Maybe someone knows the answer.
>

I'm assuming that you already ascertained that the Windows file system was
in fact clean?  If so, then this sounds like a problem with fusefs-ntfs,
not with fuse in general.  I've CC'd its maintainer.  He may be able to
help you.
-Alan

From owner-freebsd-current@freebsd.org  Sun Sep  8 23:46:29 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id D40E8E7031
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sun,  8 Sep 2019 23:46:29 +0000 (UTC)
 (envelope-from imb@protected-networks.net)
Received: from sarah.protected-networks.net (sarah.protected-networks.net
 [IPv6:2001:470:1f07:4e1::1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (1024 bits) client-digest SHA256)
 (Client CN "sarah.protected-networks.net",
 Issuer "Protected Networks CA" (not verified))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RScP52gwz4Fs3;
 Sun,  8 Sep 2019 23:46:29 +0000 (UTC)
 (envelope-from imb@protected-networks.net)
Received: from mail.protected-networks.net (mail.protected-networks.net
 [202.12.127.228])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 key-exchange X25519 server-signature RSA-PSS (1024 bits) server-digest SHA256
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "mail.protected-networks.net",
 Issuer "Protected Networks CA" (verified OK))
 by sarah.protected-networks.net (Postfix) with ESMTPS id E98E2D79A1;
 Sun,  8 Sep 2019 19:36:20 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=
 protected-networks.net; h=content-transfer-encoding
 :content-language:content-type:content-type:in-reply-to
 :mime-version:user-agent:date:date:message-id:from:from
 :references:subject:subject; s=201508; t=1567985765; bh=fN4bXXCL
 Sftnq6D/1d1gOxU4oCjWFZCxM1cYLnxpjYM=; b=gC0A7Qvk42ejZpC3sQqE0mk/
 XVCgXPQGk7gzMo1vT+fjzntA09zodT5WxERMel532A1ZHbQ86yxm6rdMR7MctK0t
 0anFk6Pj5b59rJtwBkU8HCAiyEChKRDIR0gsXwoQtLtmBChjt4eKMZh2H3CGjmBu
 fFfes3Ow0fuiThnB2Ok=
Received: from toshi.auburn.protected-networks.net
 (toshi.auburn.protected-networks.net [192.168.1.10])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
 (Client did not present a certificate)
 (Authenticated sender: imb@mail.protected-networks.net)
 by mail.protected-networks.net (Postfix) with ESMTPSA id B8C3D2DFA3;
 Sun,  8 Sep 2019 19:36:05 -0400 (EDT)
Subject: Re: fusefs & ntfs-3g
To: Alan Somers <asomers@freebsd.org>,
 "Clay Daniels Jr." <clay.daniels.jr@gmail.com>
Cc: "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>,
 freebsd@dussan.org
References: <CAGLDxTXT4Dh4D_FuFNoMgsqNTbx03sY=yYAPixpddqim3P_LpA@mail.gmail.com>
 <CAOtMX2jQ9KwbZJYnds+7mqD8-jv_wn3SJsPS4HvS64jJQV3YDw@mail.gmail.com>
From: Michael Butler <imb@protected-networks.net>
Message-ID: <f3f7050d-f94d-5c78-1b75-d27c52226a6d@protected-networks.net>
Date: Sun, 8 Sep 2019 19:36:05 -0400
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101
 Thunderbird/68.1.0
MIME-Version: 1.0
In-Reply-To: <CAOtMX2jQ9KwbZJYnds+7mqD8-jv_wn3SJsPS4HvS64jJQV3YDw@mail.gmail.com>
Content-Type: text/plain; charset=windows-1252
Content-Language: en-NZ
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46RScP52gwz4Fs3
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-6.00 / 15.00]; TAGGED_RCPT(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; REPLY(-4.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 08 Sep 2019 23:46:29 -0000

On 9/8/19 7:09 PM, Alan Somers wrote:
> On Sun, Sep 8, 2019 at 5:00 PM Clay Daniels Jr. <clay.daniels.jr@gmail.com>
> wrote:
> 
>> I want to view my Windows C: drive on ata0p4.
>> This is what I have:
>> FreeBSD bsd13 13.0-CURRENT FreeBSD 13.0-CURRENT r351901 GENERIC  amd64
>> clay@bsd13:~ % pkg info -r fusefs-libs
>> fusefs-libs-2.9.9:
>> clay@bsd13:~ % pkg info -r fusefs-libs3
>> fusefs-libs3-3.6.2:
>> clay@bsd13:~ % cat /boot/loader.conf
>> hw.syscons.disable=1
>> fusefs_load="YES"
>> clay@bsd13:~ % kldstat
>> Id Refs Address                Size Name
>>  1   72 0xffffffff80200000  2336f80 kernel
>>  2    1 0xffffffff82538000    27990 fusefs.ko
>>  3    1 0xffffffff82720000   2519c4 amdgpu.ko
>>  4    2 0xffffffff82972000    77bd0 drm.ko
>>  5    5 0xffffffff829ea000    12470 linuxkpi.ko
>>  6    3 0xffffffff829fd000    12f30 linuxkpi_gplv2.ko
>>  7    2 0xffffffff82a10000      8e0 lindebugfs.ko
>>  8    1 0xffffffff82a11000     f281 ttm.ko
>>  9    1 0xffffffff82a21000     23f7 radeon_kabini_pfp_bin.ko
>> 10    1 0xffffffff82a24000     23f5 radeon_kabini_me_bin.ko
>> 11    1 0xffffffff82a27000     23f5 radeon_kabini_ce_bin.ko
>> 12    1 0xffffffff82a2a000     43f7 radeon_kabini_mec_bin.ko
>> 13    1 0xffffffff82a2f000     2a77 radeon_kabini_rlc_bin.ko
>> 14    1 0xffffffff82a32000     12e9 radeon_kabini_sdma_bin.ko
>> 15    1 0xffffffff82a34000     12eb radeon_kabini_sdma1_bin.ko
>> 16    1 0xffffffff82a36000    38ea7 radeon_kabini_uvd_bin.ko
>> 17    1 0xffffffff82a6f000    18c47 radeon_kabini_vce_bin.ko
>> 18    1 0xffffffff82a88000     2538 intpm.ko
>> 19    1 0xffffffff82a8b000      a50 smbus.ko
>> 20    1 0xffffffff82a8c000     1820 uhid.ko
>> 21    1 0xffffffff82a8e000     2928 ums.ko
>> 22    1 0xffffffff82a91000     1b00 wmt.ko
>> 23    1 0xffffffff82a93000      acf mac_ntpd.ko
>> 24    1 0xffffffff82a94000     a9f8 tmpfs.ko
>>
>> clay@bsd13:~ % pkg info fusefs-ntfs-2017.3.23
>> fusefs-ntfs-2017.3.23
>> Name           : fusefs-ntfs
>> Version        : 2017.3.23
>> Installed on   : Sun Sep  8 17:24:36 2019 CDT
>> Origin         : sysutils/fusefs-ntfs
>> Architecture   : FreeBSD:13:amd64
>> Prefix         : /usr/local
>> Categories     : sysutils
>> Licenses       : GPLv2+
>> Maintainer     : freebsd@dussan.org
>> WWW            : https://www.tuxera.com/community/open-source-ntfs-3g/
>> Comment        : Mount NTFS partitions (read/write) and disk images
>> Options        :
>> DOCS           : on
>> LOCK           : on
>> UBLIO          : on
>> Shared Libs required:
>> libfuse.so.2
>> libuuid.so.1
>> libublio.so.1
>> Shared Libs provided:
>> libntfs-3g.so.88
>> Annotations    :
>> FreeBSD_version: 1300044
>> repo_type      : binary
>> repository     : FreeBSD
>> Flat size      : 1.93MiB
>> Description    :
>> The ntfs-3g driver is an open source, freely available read/write NTFS
>> driver, which provides safe and fast handling of the Windows XP, Windows
>> Server 2003, Windows 2000, Windows Vista, Windows Server 2008, Windows 7
>> and Windows 8 NTFS file systems. Almost the full POSIX filesystem
>> functionality is supported, the major exceptions are changing the file
>> ownerships and the access rights.
>>
>> root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt
>> * Windows is hibernated, refused to mount.
>> The disk contains an unclean file system (0, 0).
>> Metadata kept in Windows cache, refused to mount.
>> Falling back to read-only mount because the NTFS partition is in an unsafe
>> state. Please resume and shutdown Windows fully (no hibernation
>> or fast restarting.
>>
>> I'm having a roadblock here. Maybe someone knows the answer.
>>
> 
> I'm assuming that you already ascertained that the Windows file system was
> in fact clean?  If so, then this sounds like a problem with fusefs-ntfs,
> not with fuse in general.  I've CC'd its maintainer.  He may be able to
> help you.
> -Alan

Windows-10 has a changed default; it does the equivalent of hibernation
to facilitate "fast start". ntfs-3g won't touch a partition for writing
in that mode :-(

If I recall correctly, there is a setting you must change from in
Windows under control panel -> system -> power and sleep. From there you
should be able to disable the "fast start" option and, after shutting
down, ntfs-3g will allow a read/write mount,

	imb


From owner-freebsd-current@freebsd.org  Mon Sep  9 00:08:32 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id A758CE7BC8
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 00:08:32 +0000 (UTC)
 (envelope-from dclarke@blastwave.org)
Received: from atl4mhob01.registeredsite.com (atl4mhob01.registeredsite.com
 [209.17.115.39])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "*.registeredsite.com",
 Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RT5q3hN9z4Ghr
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 00:08:31 +0000 (UTC)
 (envelope-from dclarke@blastwave.org)
Received: from mailpod.hostingplatform.com
 (atl4qobmail01pod2.registeredsite.com [10.30.77.35])
 by atl4mhob01.registeredsite.com (8.14.4/8.14.4) with ESMTP id x8908SGa019894
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL)
 for <freebsd-current@freebsd.org>; Sun, 8 Sep 2019 20:08:29 -0400
Received: (qmail 49025 invoked by uid 0); 9 Sep 2019 00:08:28 -0000
X-TCPREMOTEIP: 99.253.177.25
X-Authenticated-UID: dclarke@blastwave.org
Received: from unknown (HELO ?172.16.35.3?)
 (dclarke@blastwave.org@99.253.177.25)
 by 0 with ESMTPA; 9 Sep 2019 00:08:28 -0000
Subject: Re: Panic with Current 351901 ISO image
To: freebsd-current@freebsd.org
References: <D970EBBF-E1B5-4CEA-98DD-B9C661F894B1@verizon.net>
 <01378a14-471d-31b4-c60a-98a7a059748b@gmail.com>
From: Dennis Clarke <dclarke@blastwave.org>
Message-ID: <f6bb2fc2-6af7-d88c-e8e6-ab3ad1fb2e44@blastwave.org>
Date: Sun, 8 Sep 2019 20:08:18 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101
 Thunderbird/69.0
MIME-Version: 1.0
In-Reply-To: <01378a14-471d-31b4-c60a-98a7a059748b@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46RT5q3hN9z4Ghr
X-Spamd-Bar: +++++
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=none (mx1.freebsd.org: domain of dclarke@blastwave.org has no SPF policy
 when checking 209.17.115.39) smtp.mailfrom=dclarke@blastwave.org
X-Spamd-Result: default: False [5.64 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1];
 RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[];
 RBL_VIRUSFREE_BOTNET(2.00)[39.115.17.209.bip.virusfree.cz : 127.0.0.2];
 NEURAL_SPAM_MEDIUM(1.00)[0.998,0];
 NEURAL_SPAM_LONG(1.00)[1.000,0];
 RCVD_IN_DNSWL_NONE(0.00)[39.115.17.209.list.dnswl.org : 127.0.5.0];
 R_SPF_NA(0.00)[]; DMARC_NA(0.00)[blastwave.org];
 FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:19871, ipnet:209.17.112.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 IP_SCORE(0.74)[ipnet: 209.17.112.0/21(2.08), asn: 19871(1.66),
 country: US(-0.05)]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 00:08:32 -0000

On 9/8/19 6:03 PM, Curtis Hamilton wrote:
> I'm encountering (randomly) the below error when trying to install 
> 351901 from CD/DVD.
> 

On what type of system ?



-- 
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken
GreyBeard and suspenders optional

From owner-freebsd-current@freebsd.org  Mon Sep  9 00:19:50 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3067AE8371
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 00:19:50 +0000 (UTC)
 (envelope-from kob6558@gmail.com)
Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com
 [IPv6:2607:f8b0:4864:20::32d])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RTLs237Fz4HJG;
 Mon,  9 Sep 2019 00:19:48 +0000 (UTC)
 (envelope-from kob6558@gmail.com)
Received: by mail-ot1-x32d.google.com with SMTP id 100so10847693otn.2;
 Sun, 08 Sep 2019 17:19:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=MWBP8BB3zcz/pq/7X8Sy5EluOTyL6O43XxnZzP9DrLs=;
 b=rhTwVhaRRk0TuKaKdyvjHBgR/t1xV9R29xpX5AiP7VXN/Ioe3zz1ycXLqItSlnpaxR
 7T8wP8BZ5zzs13bzKoBKAF+1j7crhDLRS9Z8LrWKEepfiQVhLQj/MLCzfTySpSIMmNhQ
 CT1u3H2VV0IxFYDtdGq51IH4KwVTuDj0J/ypY0I69BIk9A2n57AxSK+9VbhQXSArgdkt
 LxbAgAHBtbCac4VNmvlykeDm8noKFqTOFlLC12v22hZ1m57y/Z5sj628HdZU9CoFWHSI
 cNf1vgAf4YI1QOItapvnlEBjTFkR5T7EEbDTGTVaqDhMKISKT5dmeMEC69rkqYL7JVc1
 TT4Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=MWBP8BB3zcz/pq/7X8Sy5EluOTyL6O43XxnZzP9DrLs=;
 b=YO2gJOmMtNx7PmqD+Cxx/SphGR4GrAtoJm2oGNcVb/EkcNhLqVt0A7vQKiVbvl2v04
 zxF1SC8PET5sE0AUIPKdvtLKxAbux8LV2J5TxBlaD1CR5tEhTKEBL56/JOQ+6+Heaalr
 +iTfNjbTOwOjuFVIOm8n6nNMiV3g7pkHZjRTgcylxlPP/im1XNOHqHZgQb0GqqrJwtwu
 Q11S1BP9H5YeqqcU5XHPHjLZurin3qgDV1yNhcXS+hbljBjT3djOrb0mmP+SvFrnHV7I
 xonTOoI3lsVbboLlqaGoPfxflfRt2OaRS2H5lTNCDcHSroeIAqERGNQPczcRwIODpZwV
 wm2A==
X-Gm-Message-State: APjAAAWUT0lK6sNM8TtdqutA6/Z7BLcWgwhZQQ4xQb3JCoPwkG5Tourq
 WDhxqqL5e2uM5lJPUgcPdYcy5VWqP4P+k/ZBB/c=
X-Google-Smtp-Source: APXvYqyqo8tX9LVKgeDMeVlKifchCsUfr0BnjArZXtJXFqiNhTvwasXY4UpK581ckS2+FJx76DRwBWXaNIlV8zTYzIs=
X-Received: by 2002:a05:6830:138b:: with SMTP id
 d11mr15144259otq.199.1567988387516; 
 Sun, 08 Sep 2019 17:19:47 -0700 (PDT)
MIME-Version: 1.0
References: <CAGLDxTXT4Dh4D_FuFNoMgsqNTbx03sY=yYAPixpddqim3P_LpA@mail.gmail.com>
 <CAOtMX2jQ9KwbZJYnds+7mqD8-jv_wn3SJsPS4HvS64jJQV3YDw@mail.gmail.com>
 <f3f7050d-f94d-5c78-1b75-d27c52226a6d@protected-networks.net>
In-Reply-To: <f3f7050d-f94d-5c78-1b75-d27c52226a6d@protected-networks.net>
From: Kevin Oberman <rkoberman@gmail.com>
Date: Sun, 8 Sep 2019 17:19:30 -0700
Message-ID: <CAN6yY1t=CisCLx_8eU-=NGrPQ3PcRcaQQ1v=eao+LBO5XotPDQ@mail.gmail.com>
Subject: Re: fusefs & ntfs-3g
To: Michael Butler <imb@protected-networks.net>
Cc: Alan Somers <asomers@freebsd.org>,
 "Clay Daniels Jr." <clay.daniels.jr@gmail.com>, 
 "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>, freebsd@dussan.org
X-Rspamd-Queue-Id: 46RTLs237Fz4HJG
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=rhTwVhaR;
 dmarc=pass (policy=none) header.from=gmail.com;
 spf=pass (mx1.freebsd.org: domain of kob6558@gmail.com designates
 2607:f8b0:4864:20::32d as permitted sender) smtp.mailfrom=kob6558@gmail.com
X-Spamd-Result: default: False [-1.20 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[];
 TO_DN_SOME(0.00)[];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36];
 FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_FIVE(0.00)[5];
 DKIM_TRACE(0.00)[gmail.com:+];
 DMARC_POLICY_ALLOW(-0.50)[gmail.com,none];
 FORGED_SENDER(0.30)[rkoberman@gmail.com,kob6558@gmail.com];
 IP_SCORE(0.00)[ip: (-6.44), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27),
 country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[rkoberman@gmail.com,kob6558@gmail.com];
 DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[d.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[];
 SUSPICIOUS_RECIPS(1.50)[]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 00:19:50 -0000

On Sun, Sep 8, 2019 at 4:46 PM Michael Butler <imb@protected-networks.net>
wrote:

> On 9/8/19 7:09 PM, Alan Somers wrote:
> > On Sun, Sep 8, 2019 at 5:00 PM Clay Daniels Jr. <
> clay.daniels.jr@gmail.com>
> > wrote:
> >
> >> I want to view my Windows C: drive on ata0p4.
> >> This is what I have:
> [...]
> >> root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt
> >> * Windows is hibernated, refused to mount.
> >> The disk contains an unclean file system (0, 0).
> >> Metadata kept in Windows cache, refused to mount.
> >> Falling back to read-only mount because the NTFS partition is in an
> unsafe
> >> state. Please resume and shutdown Windows fully (no hibernation
> >> or fast restarting.
> >>
> >> I'm having a roadblock here. Maybe someone knows the answer.
> >>
> >
> > I'm assuming that you already ascertained that the Windows file system
> was
> > in fact clean?  If so, then this sounds like a problem with fusefs-ntfs,
> > not with fuse in general.  I've CC'd its maintainer.  He may be able to
> > help you.
> > -Alan
>
> Windows-10 has a changed default; it does the equivalent of hibernation
> to facilitate "fast start". ntfs-3g won't touch a partition for writing
> in that mode :-(
>
> If I recall correctly, there is a setting you must change from in
> Windows under control panel -> system -> power and sleep. From there you
> should be able to disable the "fast start" option and, after shutting
> down, ntfs-3g will allow a read/write mount,
>

I can confirm this. It is documented, but not obvious. You ave two choices:
1. Change the system setting, more or less as Michael suggests. I'm away
from my only W10 system, so I can't check the exact details.
2. Force a full shutdown by starting a command window and entering
"shutdown /s  /f /t 0". This is a one-time full shutdown.
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkoberman@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683

From owner-freebsd-current@freebsd.org  Mon Sep  9 00:41:49 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 62B45E8CA5
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 00:41:49 +0000 (UTC)
 (envelope-from clay.daniels.jr@gmail.com)
Received: from mail-vs1-xe42.google.com (mail-vs1-xe42.google.com
 [IPv6:2607:f8b0:4864:20::e42])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RTrD0JBWz4JLP;
 Mon,  9 Sep 2019 00:41:47 +0000 (UTC)
 (envelope-from clay.daniels.jr@gmail.com)
Received: by mail-vs1-xe42.google.com with SMTP id s18so7676055vsa.0;
 Sun, 08 Sep 2019 17:41:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=Cuz3UuG/o/Qscibhkn3FVXucBCZmcvUTW8pnbrB8M94=;
 b=CIWFMnsfutwPAOl1jOb/2XNS/FnV4NniyS3hpCxYbRA6o+NdEJjB+xLBTbQgA+EenI
 FpE3ZGcOE3MkzavUIYlBgSXMw/uRW0kRMQ29plh64wz+aveD1/BQ0YyWg0QP3RvkXEh6
 mVZFRxXpEkszexUyqcsNBQgkDBHRhtOVRDABkhvgRlwONhyc2OEAOFBAiEURR5BBUwvl
 xMEPuF6Xm/cpYv3P8rDq9IaySICsYlnHF8acLhMouKS19czhZoiSzRKAkTQbI9skEuOl
 TcQUVTmEe/X9MOP4pB0DFjTwBngNhcjgoYG3+ydYNHI1vg2S3cLHg/5gWIiMGdiloZkY
 D7KA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=Cuz3UuG/o/Qscibhkn3FVXucBCZmcvUTW8pnbrB8M94=;
 b=falakkH4hZ7do8GnqHCIRFLcFJUemoPh7huyCVWVIUR65YHSYgDgZYlXwaNVM7ZpOF
 HZk/WMF5smXC11gtR6i413L0cw10bNMmh9RWEaVREsfd26myUeh057KWtK4di9wnnblD
 /nATozzvu5Yy6HYnhAk0mzzX3MDRA2WMR8EyPlJodURRXOpNK3VixOHWijyMyhJ2Zind
 5QDzsEk8Z35I+S4snYjAnn7ZvARfhtLiBYBvqkZR57dfJeWc9244FKWuk9pKresCvpjz
 6gLngEiPjrzR4a2WzfiNCtmqABg36CGapVy1Xr4PQVzZmI7m7n4wj8mNllXwkx+sB2Tl
 TRjA==
X-Gm-Message-State: APjAAAVCvjyPG8DfGJEQy1V9EKf5XSFhMG8DT1GFxc2xsdTCB0zC++zW
 7SotH4u1qOwZ1CcDYLuLD7VnekeZZ1HdZ4ZLqA==
X-Google-Smtp-Source: APXvYqynZ2g2IrKGdIK/pKn4J6BUa5BnaREqpzcq++RnfPCc8BsiS0bG7EB57SFHoXhO5KMnS7I6EbOD6POu2KyO+6A=
X-Received: by 2002:a67:ce88:: with SMTP id c8mr7640334vse.221.1567989706398; 
 Sun, 08 Sep 2019 17:41:46 -0700 (PDT)
MIME-Version: 1.0
References: <CAGLDxTXT4Dh4D_FuFNoMgsqNTbx03sY=yYAPixpddqim3P_LpA@mail.gmail.com>
 <CAOtMX2jQ9KwbZJYnds+7mqD8-jv_wn3SJsPS4HvS64jJQV3YDw@mail.gmail.com>
 <f3f7050d-f94d-5c78-1b75-d27c52226a6d@protected-networks.net>
 <CAN6yY1t=CisCLx_8eU-=NGrPQ3PcRcaQQ1v=eao+LBO5XotPDQ@mail.gmail.com>
In-Reply-To: <CAN6yY1t=CisCLx_8eU-=NGrPQ3PcRcaQQ1v=eao+LBO5XotPDQ@mail.gmail.com>
From: "Clay Daniels Jr." <clay.daniels.jr@gmail.com>
Date: Sun, 8 Sep 2019 19:41:34 -0500
Message-ID: <CAGLDxTUkTxdGa9brrsvK3Dio4yXBZtsBcMyw9+1Ru8h-PdB6vw@mail.gmail.com>
Subject: Re: fusefs & ntfs-3g
To: Kevin Oberman <rkoberman@gmail.com>
Cc: Michael Butler <imb@protected-networks.net>,
 Alan Somers <asomers@freebsd.org>, 
 "freebsd-current@freebsd.org" <freebsd-current@freebsd.org>, freebsd@dussan.org
X-Rspamd-Queue-Id: 46RTrD0JBWz4JLP
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=CIWFMnsf;
 dmarc=pass (policy=none) header.from=gmail.com;
 spf=pass (mx1.freebsd.org: domain of claydanielsjr@gmail.com designates
 2607:f8b0:4864:20::e42 as permitted sender)
 smtp.mailfrom=claydanielsjr@gmail.com
X-Spamd-Result: default: False [-3.00 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[];
 TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c];
 RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[gmail.com:+];
 DMARC_POLICY_ALLOW(-0.50)[gmail.com,none];
 FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[];
 IP_SCORE(0.00)[ip: (2.16), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27),
 country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 TAGGED_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[2.4.e.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 00:41:49 -0000

Alan,  Michael, & Kevin, THANKS very much. This is kind of neat:

root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt
root@bsd13:/home/clay # cd /mnt
root@bsd13:/mnt # ls -al

total 2368276
drwxrwxrwx   1 root  wheel           0 Jul 17 16:40 $Recycle.Bin
drwxrwxrwx   1 root  wheel        4096 Aug  6 02:08 .
drwxr-xr-x  19 root  wheel        1024 Sep  8 19:25 ..
drwxrwxrwx   1 root  wheel           0 Aug 13 15:05 Config.Msi
lrwxrwxrwx   2 root  wheel          10 Jul 17 17:07 Documents and Settings
-> /mnt/Users
drwxrwxrwx   1 root  wheel           0 Mar 18 23:52 PerfLogs
drwxrwxrwx   1 root  wheel        4096 Aug  4 16:11 Planetdance
drwxrwxrwx   1 root  wheel        8192 Sep  3 18:55 Program Files
drwxrwxrwx   1 root  wheel        8192 Aug 11 02:20 Program Files (x86)
drwxrwxrwx   1 root  wheel        4096 Aug  6 02:01 ProgramData
drwxrwxrwx   1 root  wheel           0 Jul 17 15:42 Recovery
drwxrwxrwx   1 root  wheel        4096 Sep  8 13:59 System Volume
Information
drwxrwxrwx   1 root  wheel        4096 Jul 17 16:01 Users
drwxrwxrwx   1 root  wheel       16384 Aug 31 20:39 Windows
-rwxrwxrwx   1 root  wheel  1485533184 Sep  8 19:03 hiberfil.sys
-rwxrwxrwx   1 root  wheel   671088640 Sep  8 00:24 pagefile.sys
-rwxrwxrwx   1 root  wheel   268435456 Sep  8 00:24 swapfile.sys
root@bsd13:/mnt #

Clay


On Sun, Sep 8, 2019 at 7:19 PM Kevin Oberman <rkoberman@gmail.com> wrote:

> On Sun, Sep 8, 2019 at 4:46 PM Michael Butler <imb@protected-networks.net>
> wrote:
>
>> On 9/8/19 7:09 PM, Alan Somers wrote:
>> > On Sun, Sep 8, 2019 at 5:00 PM Clay Daniels Jr. <
>> clay.daniels.jr@gmail.com>
>> > wrote:
>> >
>> >> I want to view my Windows C: drive on ata0p4.
>> >> This is what I have:
>> [...]
>> >> root@bsd13:/home/clay # ntfs-3g -o default_permissons /dev/ada0p4 /mnt
>> >> * Windows is hibernated, refused to mount.
>> >> The disk contains an unclean file system (0, 0).
>> >> Metadata kept in Windows cache, refused to mount.
>> >> Falling back to read-only mount because the NTFS partition is in an
>> unsafe
>> >> state. Please resume and shutdown Windows fully (no hibernation
>> >> or fast restarting.
>> >>
>> >> I'm having a roadblock here. Maybe someone knows the answer.
>> >>
>> >
>> > I'm assuming that you already ascertained that the Windows file system
>> was
>> > in fact clean?  If so, then this sounds like a problem with fusefs-ntfs,
>> > not with fuse in general.  I've CC'd its maintainer.  He may be able to
>> > help you.
>> > -Alan
>>
>> Windows-10 has a changed default; it does the equivalent of hibernation
>> to facilitate "fast start". ntfs-3g won't touch a partition for writing
>> in that mode :-(
>>
>> If I recall correctly, there is a setting you must change from in
>> Windows under control panel -> system -> power and sleep. From there you
>> should be able to disable the "fast start" option and, after shutting
>> down, ntfs-3g will allow a read/write mount,
>>
>
> I can confirm this. It is documented, but not obvious. You ave two choices:
> 1. Change the system setting, more or less as Michael suggests. I'm away
> from my only W10 system, so I can't check the exact details.
> 2. Force a full shutdown by starting a command window and entering
> "shutdown /s  /f /t 0". This is a one-time full shutdown.
> --
> Kevin Oberman, Part time kid herder and retired Network Engineer
> E-mail: rkoberman@gmail.com
> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
>
>

From owner-freebsd-current@freebsd.org  Mon Sep  9 16:23:47 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 89560D7296
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 16:23:47 +0000 (UTC) (envelope-from ian@freebsd.org)
Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org
 [54.186.57.195])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Rtl716H5z4BWv
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 16:23:46 +0000 (UTC)
 (envelope-from ian@freebsd.org)
ARC-Seal: i=1; a=rsa-sha256; t=1568046225; cv=none;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 b=DMHRkqd0qMhzvhET7okL/oQvH8SjkRcvPx88TQRMVHB7aX64XlkD6FCrra7/rMpBBc799XwLUqQxT
 /OI7Rbcasg95iwquePimpd+KnC8jGfC5S4rB4PdWBoGh5PMHD8DRpg4Mhsn0FM0FC4M9lq/3HJ5yfa
 CHCsj08P10zev4Gwf9JhjaFTAkYywSowcmRtqg2qWoBXi37ytmTOj2zIqiEOzMin1Gpbvk0zSd7hAQ
 DKWPscSiTJdKYMZZiJIVcfFMoZWnBH+rZP2Z83Xfyw7158pgwsmo5h7Opd7oL+dRXKEWK0crxf9hu6
 2f0lFT16zDLjZRAZFJfYk8kzzlB4ooA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:dkim-signature:from;
 bh=ud7Q7xdQPRE4fkkD0wRKHXhnzBfclKNdJdmcyv1JiSk=;
 b=jNH1WQhvhkicmjFXaklmSha6Ha/KgCcAkmBy9FKNGZHcpDJDoVWPz1Cq+X0OT9g1QrSW/X7BN4dSN
 QdQuflvOOJrO6JnFCepPlcNsHyRcvBWOf/GlbAzkILTM0JBvvSgjROl4Ui9MMPddSGSRxzEFb3pabY
 rkMxvAUBQC0Gz99nZeIbeWDYPmrqY2j1oXe3SJwx0oVx64qer/3S2yvnjdK8zsyvw92lMtUuKr6X7P
 wotu78zdjBPAVrv0/GQmJuNzWl3Oo98NeTP/56UPKHPmZio3jizFpobKo+y7HgNerF3SJet9Z/XLYM
 kdlxyw34CUJmLO1fDA0ijHOhFCU0RrQ==
ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org;
 spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60;
 dmarc=none header.from=freebsd.org;
 arc=none header.oldest-pass=0;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=dkim-high;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:from;
 bh=ud7Q7xdQPRE4fkkD0wRKHXhnzBfclKNdJdmcyv1JiSk=;
 b=rkpLS3dwfbw0nskVY1CZJPx/JMpX9qtSDolcCFAU30LzVzMQy/LgJt7Bvs6WwsCBFzJpqqinz9z6X
 JjLec9EDDYORsxcooJHJ3SJyyizWILoj4+0DytMkzaCWdf8EDDysi++Z2GINmanQaKWsfDS/wBtRiH
 MQ7c2xTgjzw3tBL+oCaP1javiQvck8ON+Kl5Uj73KxLXdct3XdOSxKG3tLMJ+VC+hah7EoZg69zdVE
 cLprhVbXEem2gjq6FpdkOmbCnst984NzOjCjbq4+hAl+XH3JrqlqGNHKVmgs8DxcCdftWgkCiLpDqa
 8MkC6gJdNDeL80SNCT7PrIMecIIDz4A==
X-MHO-RoutePath: aGlwcGll
X-MHO-User: 2f71b0be-d31e-11e9-b67d-cdd75d6ce7a8
X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information
X-Originating-IP: 67.177.211.60
X-Mail-Handler: DuoCircle Outbound SMTP
Received: from ilsoft.org (unknown [67.177.211.60])
 by outbound3.ore.mailhop.org (Halon) with ESMTPSA
 id 2f71b0be-d31e-11e9-b67d-cdd75d6ce7a8;
 Mon, 09 Sep 2019 16:23:43 +0000 (UTC)
Received: from rev (rev [172.22.42.240])
 by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89GNeFc062698;
 Mon, 9 Sep 2019 10:23:40 -0600 (MDT) (envelope-from ian@freebsd.org)
Message-ID: <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org>
Subject: Re: ntpd segfaults on start
From: Ian Lepore <ian@freebsd.org>
To: Cy Schubert <Cy.Schubert@cschubert.com>, Konstantin Belousov
 <kostikbel@gmail.com>
Cc: Harlan Stenn <stenn@nwtime.org>, Vladimir Zakharov <zakharov.vv@gmail.com>,
 freebsd-current@freebsd.org
Date: Mon, 09 Sep 2019 10:23:40 -0600
In-Reply-To: <201909071628.x87GSFPV005827@slippy.cwsent.com>
References: <201909051307.x85D7MGs034053@slippy.cwsent.com>
 <20190905142817.GB2559@kib.kiev.ua>
 <201909060355.x863tRhP089169@slippy.cwsent.com>
 <201909060639.x866dJ7f090176@slippy.cwsent.com>
 <201909062356.x86NuKdk003780@slippy.cwsent.com>
 <156d1e7c-0dbb-8707-90b3-13ae97c87449@nwtime.org>
 <20190907075619.GG2559@kib.kiev.ua>
 <201909071309.x87D9GxZ089964@slippy.cwsent.com>
 <20190907153226.GI2559@kib.kiev.ua>
 <201909071545.x87FjLS6004448@slippy.cwsent.com>
 <20190907161749.GJ2559@kib.kiev.ua>
 <201909071628.x87GSFPV005827@slippy.cwsent.com>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46Rtl716H5z4BWv
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-2.00 / 15.00];
 local_wl_from(0.00)[freebsd.org];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[];
 ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 16:23:47 -0000

On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin Belousov writes:
> > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote:
> > > I've been able to set the memlock rlimit as low as 20 MB. The issue is 
> > > letting it default to 0 which allows ntp to mlockall() anything it wants. 
> > > ntpd on my sandbox is currently using 18 MB.
> > 
> > Default stack size on amd64 is 512M, and default stack gap percentage is
> > 3%. This means that the gap can be as large as ~17MB. If 3MB is enough
> > for the stack of the main thread of ntpd, then fine.
> 
> The default stack is 200K, which is also tuneable in ntp.conf.
> 
> [...]

I haven't seen anyone ask what I consider to be the crucial question
yet:  why are we locking ntpd into memory by default at all?

I have seen two rationales for ntpd using mlockall() and setrlimit(): 

   - There are claims that it improves timing performance.

   - Because ntpd is a daemon that can run for months at a time,
   setting limits on memory and stack growth can help detect and
   mitigate against memory leak problems in the daemon.

I am *highly* skeptical of claims that locking ntpd in memory will
improve timing performance on freebsd (to the point where I'm inclined
to dismiss them out of hand, but I'd be willing to look at any actual
evidence).

The second point has at least some validity, but would be better
implemented (IMO) by decoupling the address space limit from the act of
locking down memory, and allowing configuration of RLIMIT_AS separately
from RLIMIT_MEMLOCK.  If there's any interest in this, I could try to
put together a patchset and submit it upstream for that.

I would propose that for freebsd, the autoconf-generated value for
DFLT_RLIMIT_MEMLOCK should be -1 to avoid calling setrlimit() and
mlockall() by default.  Then in the ntp.conf we distribute we have a
section like:

   # To lock ntpd into memory, uncomment the following rlimit line.
   # This should not be necessary on most systems, but may improve
   # performance on heavily-loaded servers experiencing enough
   # memory pressure to cause ntpd to be paged out.
   # rlimit memlock <something> stacksize <something>

Then we would need to come up with reasonable values for <something>.

-- Ian


From owner-freebsd-current@freebsd.org  Mon Sep  9 16:30:57 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 13408D75A0
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 16:30:57 +0000 (UTC)
 (envelope-from freebsd-rwg@gndrsh.dnsmgr.net)
Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RtvN5YC0z4BwC;
 Mon,  9 Sep 2019 16:30:56 +0000 (UTC)
 (envelope-from freebsd-rwg@gndrsh.dnsmgr.net)
Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1])
 by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x89GUjvG044289;
 Mon, 9 Sep 2019 09:30:45 -0700 (PDT)
 (envelope-from freebsd-rwg@gndrsh.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
 by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x89GUjGX044288;
 Mon, 9 Sep 2019 09:30:45 -0700 (PDT) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>
Message-Id: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net>
Subject: Re: ntpd segfaults on start
In-Reply-To: <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org>
To: Ian Lepore <ian@freebsd.org>
Date: Mon, 9 Sep 2019 09:30:45 -0700 (PDT)
CC: Cy Schubert <Cy.Schubert@cschubert.com>,
 Konstantin Belousov <kostikbel@gmail.com>, Harlan Stenn <stenn@nwtime.org>,
 Vladimir Zakharov <zakharov.vv@gmail.com>, freebsd-current@freebsd.org
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-Rspamd-Queue-Id: 46RtvN5YC0z4BwC
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-6.00 / 15.00]; TAGGED_RCPT(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; REPLY(-4.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 16:30:57 -0000

> On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin Belousov writes:
> > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote:
> > > > I've been able to set the memlock rlimit as low as 20 MB. The issue is 
> > > > letting it default to 0 which allows ntp to mlockall() anything it wants. 
> > > > ntpd on my sandbox is currently using 18 MB.
> > > 
> > > Default stack size on amd64 is 512M, and default stack gap percentage is
> > > 3%. This means that the gap can be as large as ~17MB. If 3MB is enough
> > > for the stack of the main thread of ntpd, then fine.
> > 
> > The default stack is 200K, which is also tuneable in ntp.conf.
> > 
> > [...]
> 
> I haven't seen anyone ask what I consider to be the crucial question
> yet:  why are we locking ntpd into memory by default at all?
> 
> I have seen two rationales for ntpd using mlockall() and setrlimit(): 
> 
>    - There are claims that it improves timing performance.
> 
>    - Because ntpd is a daemon that can run for months at a time,
>    setting limits on memory and stack growth can help detect and
>    mitigate against memory leak problems in the daemon.

Doesn't locking this memory down also protect ntpd from OOM kills?
If so that is a MUST preserve functionality, as IMHO killing ntpd
on a box that has it configured is a total no win situation.

Regards,
Rod

> I am *highly* skeptical of claims that locking ntpd in memory will
> improve timing performance on freebsd (to the point where I'm inclined
> to dismiss them out of hand, but I'd be willing to look at any actual
> evidence).
> 
> The second point has at least some validity, but would be better
> implemented (IMO) by decoupling the address space limit from the act of
> locking down memory, and allowing configuration of RLIMIT_AS separately
> from RLIMIT_MEMLOCK.  If there's any interest in this, I could try to
> put together a patchset and submit it upstream for that.
> 
> I would propose that for freebsd, the autoconf-generated value for
> DFLT_RLIMIT_MEMLOCK should be -1 to avoid calling setrlimit() and
> mlockall() by default.  Then in the ntp.conf we distribute we have a
> section like:
> 
>    # To lock ntpd into memory, uncomment the following rlimit line.
>    # This should not be necessary on most systems, but may improve
>    # performance on heavily-loaded servers experiencing enough
>    # memory pressure to cause ntpd to be paged out.
>    # rlimit memlock <something> stacksize <something>
> 
> Then we would need to come up with reasonable values for <something>.
> 
> -- Ian
> 
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
> 

-- 
Rod Grimes                                                 rgrimes@freebsd.org

From owner-freebsd-current@freebsd.org  Mon Sep  9 18:13:34 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0815ADAA4E
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 18:13:34 +0000 (UTC) (envelope-from ian@freebsd.org)
Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org
 [54.149.155.156])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Rx9n489Tz4LkQ
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 18:13:33 +0000 (UTC)
 (envelope-from ian@freebsd.org)
ARC-Seal: i=1; a=rsa-sha256; t=1568052811; cv=none;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 b=dDMr0Boc3Th5HN+DoCayLzudhO8wvUQy5Pp/AJ0CbtX0Ai8WthQGLPGxwu4EWAZM3w8gkYzEqbzYT
 oYi81heJhcoXW7XrzXkS0AgAzfYjqRT2FrznG/mCD9uM1D2WA7SoVoqO/Pv+jVaJW5vedx2t9zzynv
 /sQSeHanBUzEiyd3aeEeYALLuz+WKvw7XBXNA/gYwgp6NnsSGGPn3E3tUCjx/nZKdU22yHP9K6tHfh
 n5I9ArPAx5WjFeAHSHtJI0AUD5EBnVji30fJFEqnGiw/UhvLPITV1Cn/0zArvsg+buyVj/Yfx3Rx7x
 AC3HhK+QmaGGMrSlxJ/ZbwCFtrXIgMQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:dkim-signature:from;
 bh=M6kE9IAaGxc8PIJ7D5sV1E8cnc9Ni9NeV7sCnyDluTw=;
 b=pdrTkgJp46fFHgVkw0Jxt9nYTocGQfYZLp5MQC9WfvUmrR14p+E4Pwtltz4aweclI6iBqvCq+qFMc
 cfZ/9wBgUHWsTpGXLaE3S/DzoKDt7DMbtNxe8X9I7QdZVNauEmwbHePte1B6k8rkb2HcWyKg+v/eKh
 lMQgo1V+OrAGu5LnyAl5qwgvyl12qDmhAHYa/OVpJHG79oCQGebxEzZvkiJevGNHIWDEGJXF6+QRO4
 2N4q+2IS/MbPXUhvcd4wPilni9AyJMFmXOON+91mSiLTLAMzZmVhgdPBxEmNmV7uQRS+VudqHwiEGS
 aoLHaI/Ju4Dm+PvMGrOi6ZCbwBsa9EQ==
ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org;
 spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60;
 dmarc=none header.from=freebsd.org;
 arc=none header.oldest-pass=0;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=dkim-high;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:from;
 bh=M6kE9IAaGxc8PIJ7D5sV1E8cnc9Ni9NeV7sCnyDluTw=;
 b=hbTbpjWqKPr2EHmVLGMW7XvABCQlKGkST9hj6rot0EQChgLnPMtRbC4MlEcjgIQ2jDcNWL1qz9aF+
 jvjj5HWygkBAYA0ezbRWq3ewIY5G8UpqQ4VyPIRUJEYhbO7txfuXMGXYn7Jn1+JJTx9RbRoMiAOp5d
 Lrpr/ffCqelB/ou0sDGhPwnL7Mkyj0EaYQrSnPa+ePOTTZxWM3jwfihNp8c0NGTTe6fruljSd3opcP
 r7aKUz+rvi6fSU8OYF/XyF6i0IjVTkCI+ga+uPEXsr+2wY+VcwCwRcGeCcFwJ6XexZ18L+fWUAdV47
 0ssc5zpZI4Wz4/K55kCcbjJvFNguOng==
X-MHO-RoutePath: aGlwcGll
X-MHO-User: 8375d43e-d32d-11e9-85ed-13b9aae3a1d2
X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information
X-Originating-IP: 67.177.211.60
X-Mail-Handler: DuoCircle Outbound SMTP
Received: from ilsoft.org (unknown [67.177.211.60])
 by outbound4.ore.mailhop.org (Halon) with ESMTPSA
 id 8375d43e-d32d-11e9-85ed-13b9aae3a1d2;
 Mon, 09 Sep 2019 18:13:30 +0000 (UTC)
Received: from rev (rev [172.22.42.240])
 by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89IDOXq063123;
 Mon, 9 Sep 2019 12:13:24 -0600 (MDT) (envelope-from ian@freebsd.org)
Message-ID: <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org>
Subject: Re: ntpd segfaults on start
From: Ian Lepore <ian@freebsd.org>
To: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>
Cc: Cy Schubert <Cy.Schubert@cschubert.com>, Konstantin Belousov
 <kostikbel@gmail.com>, Harlan Stenn <stenn@nwtime.org>, Vladimir Zakharov
 <zakharov.vv@gmail.com>, freebsd-current@freebsd.org
Date: Mon, 09 Sep 2019 12:13:24 -0600
In-Reply-To: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net>
References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46Rx9n489Tz4LkQ
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-2.00 / 15.00]; TAGGED_RCPT(0.00)[];
 local_wl_from(0.00)[freebsd.org];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 18:13:34 -0000

On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote:
> > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin
> > > Belousov writes:
> > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote:
> > > > > I've been able to set the memlock rlimit as low as 20 MB. The
> > > > > issue is 
> > > > > letting it default to 0 which allows ntp to mlockall()
> > > > > anything it wants. 
> > > > > ntpd on my sandbox is currently using 18 MB.
> > > > 
> > > > Default stack size on amd64 is 512M, and default stack gap
> > > > percentage is
> > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is
> > > > enough
> > > > for the stack of the main thread of ntpd, then fine.
> > > 
> > > The default stack is 200K, which is also tuneable in ntp.conf.
> > > 
> > > [...]
> > 
> > I haven't seen anyone ask what I consider to be the crucial
> > question
> > yet:  why are we locking ntpd into memory by default at all?
> > 
> > I have seen two rationales for ntpd using mlockall() and
> > setrlimit(): 
> > 
> >    - There are claims that it improves timing performance.
> > 
> >    - Because ntpd is a daemon that can run for months at a time,
> >    setting limits on memory and stack growth can help detect and
> >    mitigate against memory leak problems in the daemon.
> 
> Doesn't locking this memory down also protect ntpd from OOM kills?
> If so that is a MUST preserve functionality, as IMHO killing ntpd
> on a box that has it configured is a total no win situation.
> 

Does it have that effect?  I don't know.  But I would argue that that's
a separate issue, and we should make that happen by adding
ntpd_oomprotect=YES to /etc/defaults/rc.conf

Right now only syslogd has oomprotect set to YES by default.  Maybe
that's a good choice -- once we start declaring one daemon to be more
important than others, you'll discover there's a whole back lot full of
bikesheds that need painting.

So maybe we should just document ntpd_oomprotect=YES in some more-
prominent way.  If we were to add a comment block to ntp.conf
describing rlimit, that might be a good place to mention setting
ntpd_oomprotect in rc.conf.

-- Ian



From owner-freebsd-current@freebsd.org  Mon Sep  9 18:22:52 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id A38D2DB174
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 18:22:52 +0000 (UTC)
 (envelope-from freebsd-rwg@gndrsh.dnsmgr.net)
Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RxNW1lFtz4MYK
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 18:22:50 +0000 (UTC)
 (envelope-from freebsd-rwg@gndrsh.dnsmgr.net)
Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1])
 by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x89IMkAt044809;
 Mon, 9 Sep 2019 11:22:46 -0700 (PDT)
 (envelope-from freebsd-rwg@gndrsh.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
 by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x89IMkdK044808;
 Mon, 9 Sep 2019 11:22:46 -0700 (PDT) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>
Message-Id: <201909091822.x89IMkdK044808@gndrsh.dnsmgr.net>
Subject: Re: ntpd segfaults on start
In-Reply-To: <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org>
To: Ian Lepore <ian@freebsd.org>
Date: Mon, 9 Sep 2019 11:22:46 -0700 (PDT)
CC: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>,
 Cy Schubert <Cy.Schubert@cschubert.com>,
 Konstantin Belousov <kostikbel@gmail.com>, Harlan Stenn <stenn@nwtime.org>,
 Vladimir Zakharov <zakharov.vv@gmail.com>, freebsd-current@freebsd.org
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-Rspamd-Queue-Id: 46RxNW1lFtz4MYK
X-Spamd-Bar: +
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF
 policy when checking 69.59.192.140)
 smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net
X-Spamd-Result: default: False [1.93 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-0.64)[-0.636,0]; FROM_HAS_DN(0.00)[];
 TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TAGGED_RCPT(0.00)[];
 MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[dnsmgr.net];
 AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 IP_SCORE(0.04)[ip: (0.15), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.04),
 country: US(-0.05)]; NEURAL_SPAM_LONG(0.13)[0.125,0];
 RCPT_COUNT_SEVEN(0.00)[7]; R_SPF_NA(0.00)[];
 FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US];
 MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 18:22:52 -0000

> On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote:
> > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin
> > > > Belousov writes:
> > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote:
> > > > > > I've been able to set the memlock rlimit as low as 20 MB. The
> > > > > > issue is 
> > > > > > letting it default to 0 which allows ntp to mlockall()
> > > > > > anything it wants. 
> > > > > > ntpd on my sandbox is currently using 18 MB.
> > > > > 
> > > > > Default stack size on amd64 is 512M, and default stack gap
> > > > > percentage is
> > > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is
> > > > > enough
> > > > > for the stack of the main thread of ntpd, then fine.
> > > > 
> > > > The default stack is 200K, which is also tuneable in ntp.conf.
> > > > 
> > > > [...]
> > > 
> > > I haven't seen anyone ask what I consider to be the crucial
> > > question
> > > yet:  why are we locking ntpd into memory by default at all?
> > > 
> > > I have seen two rationales for ntpd using mlockall() and
> > > setrlimit(): 
> > > 
> > >    - There are claims that it improves timing performance.
> > > 
> > >    - Because ntpd is a daemon that can run for months at a time,
> > >    setting limits on memory and stack growth can help detect and
> > >    mitigate against memory leak problems in the daemon.
> > 
> > Doesn't locking this memory down also protect ntpd from OOM kills?
> > If so that is a MUST preserve functionality, as IMHO killing ntpd
> > on a box that has it configured is a total no win situation.
> > 
> 
> Does it have that effect?  I don't know.  But I would argue that that's
> a separate issue, and we should make that happen by adding
> ntpd_oomprotect=YES to /etc/defaults/rc.conf
> 
> Right now only syslogd has oomprotect set to YES by default.  Maybe
> that's a good choice -- once we start declaring one daemon to be more
> important than others, you'll discover there's a whole back lot full of
> bikesheds that need painting.
> 
> So maybe we should just document ntpd_oomprotect=YES in some more-
> prominent way.  If we were to add a comment block to ntp.conf
> describing rlimit, that might be a good place to mention setting
> ntpd_oomprotect in rc.conf.
>
> -- Ian

All very reasonable, thanks Ian.

-- 
Rod Grimes                                                 rgrimes@freebsd.org

From owner-freebsd-current@freebsd.org  Mon Sep  9 18:45:00 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id B7D5CDBCA2
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 18:45:00 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Rxt43bTMz4PGt;
 Mon,  9 Sep 2019 18:45:00 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from tom.home (kib@localhost [127.0.0.1])
 by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x89Iikci082709
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Mon, 9 Sep 2019 21:44:49 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x89Iikci082709
Received: (from kostik@localhost)
 by tom.home (8.15.2/8.15.2/Submit) id x89IikiX082708;
 Mon, 9 Sep 2019 21:44:46 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
 using -f
Date: Mon, 9 Sep 2019 21:44:46 +0300
From: Konstantin Belousov <kostikbel@gmail.com>
To: Ian Lepore <ian@freebsd.org>
Cc: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>,
 Cy Schubert <Cy.Schubert@cschubert.com>, Harlan Stenn <stenn@nwtime.org>,
 Vladimir Zakharov <zakharov.vv@gmail.com>, freebsd-current@freebsd.org
Subject: Re: ntpd segfaults on start
Message-ID: <20190909184446.GU2559@kib.kiev.ua>
References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net>
 <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM,
 NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home
X-Rspamd-Queue-Id: 46Rxt43bTMz4PGt
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-6.00 / 15.00];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[];
 REPLY(-4.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 18:45:00 -0000

On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote:
> On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote:
> > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin
> > > > Belousov writes:
> > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote:
> > > > > > I've been able to set the memlock rlimit as low as 20 MB. The
> > > > > > issue is 
> > > > > > letting it default to 0 which allows ntp to mlockall()
> > > > > > anything it wants. 
> > > > > > ntpd on my sandbox is currently using 18 MB.
> > > > > 
> > > > > Default stack size on amd64 is 512M, and default stack gap
> > > > > percentage is
> > > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is
> > > > > enough
> > > > > for the stack of the main thread of ntpd, then fine.
> > > > 
> > > > The default stack is 200K, which is also tuneable in ntp.conf.
> > > > 
> > > > [...]
> > > 
> > > I haven't seen anyone ask what I consider to be the crucial
> > > question
> > > yet:  why are we locking ntpd into memory by default at all?
> > > 
> > > I have seen two rationales for ntpd using mlockall() and
> > > setrlimit(): 
> > > 
> > >    - There are claims that it improves timing performance.
> > > 
> > >    - Because ntpd is a daemon that can run for months at a time,
> > >    setting limits on memory and stack growth can help detect and
> > >    mitigate against memory leak problems in the daemon.
> > 
> > Doesn't locking this memory down also protect ntpd from OOM kills?
> > If so that is a MUST preserve functionality, as IMHO killing ntpd
> > on a box that has it configured is a total no win situation.
> > 
> 
> Does it have that effect?  I don't know.  But I would argue that that's
> a separate issue, and we should make that happen by adding
> ntpd_oomprotect=YES to /etc/defaults/rc.conf
Wiring process memory has no effect on OOM selection. More, because
all potentially allocated pages are allocated for real after mlockall(),
the size of the vmspace, as accounted by OOM, is the largest possible
size from the whole lifetime.

On the other hand, the code execution times are not predictable if the
process's pages can be paged out. Under severe load next instruction
might take several seconds or even minutes to start. It is quite unlike
the scheduler delays. That introduces a jitter in the local time
measurements and their usage as done in userspace. Wouldn't this affect
the accuracy ?

> 
> Right now only syslogd has oomprotect set to YES by default.  Maybe
> that's a good choice -- once we start declaring one daemon to be more
> important than others, you'll discover there's a whole back lot full of
> bikesheds that need painting.
> 
> So maybe we should just document ntpd_oomprotect=YES in some more-
> prominent way.  If we were to add a comment block to ntp.conf
> describing rlimit, that might be a good place to mention setting
> ntpd_oomprotect in rc.conf.
> 
> -- Ian
> 

From owner-freebsd-current@freebsd.org  Mon Sep  9 19:21:48 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9258ADCB6B
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 19:21:48 +0000 (UTC)
 (envelope-from cy.schubert@cschubert.com)
Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.13])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "Client", Issuer "CA" (not verified))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RyhW3C97z4RbJ;
 Mon,  9 Sep 2019 19:21:47 +0000 (UTC)
 (envelope-from cy.schubert@cschubert.com)
Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA
 id 7PEKitZ6asAGk7PEMiQHYZ; Mon, 09 Sep 2019 13:21:44 -0600
X-Authority-Analysis: v=2.3 cv=WeVylHpX c=1 sm=1 tr=0
 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17
 a=kj9zAlcOel0A:10 a=J70Eh1EUuV4A:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8
 a=DdwSh0sFBUtyfsAMNqAA:9 a=0_iv0QpZzRoiHx9C:21 a=HJwVhIzJ0gusrvkh:21
 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22
Received: from slippy.cwsent.com (slippy8 [10.2.2.6])
 by spqr.komquats.com (Postfix) with ESMTPS id 97C18A99;
 Mon,  9 Sep 2019 12:21:39 -0700 (PDT)
Received: from slippy.cwsent.com (localhost [127.0.0.1])
 by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x89JLdLp051527;
 Mon, 9 Sep 2019 12:21:39 -0700 (PDT)
 (envelope-from Cy.Schubert@cschubert.com)
Received: from slippy (cy@localhost)
 by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x89JLcBv051522;
 Mon, 9 Sep 2019 12:21:38 -0700 (PDT)
 (envelope-from Cy.Schubert@cschubert.com)
Message-Id: <201909091921.x89JLcBv051522@slippy.cwsent.com>
X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs
X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1
Reply-to: Cy Schubert <Cy.Schubert@cschubert.com>
From: Cy Schubert <Cy.Schubert@cschubert.com>
X-os: FreeBSD
X-Sender: cy@cwsent.com
X-URL: http://www.cschubert.com/
To: Ian Lepore <ian@freebsd.org>
cc: Cy Schubert <Cy.Schubert@cschubert.com>,
 Konstantin Belousov <kostikbel@gmail.com>, Harlan Stenn <stenn@nwtime.org>,
 Vladimir Zakharov <zakharov.vv@gmail.com>, freebsd-current@freebsd.org
Subject: Re: ntpd segfaults on start
In-reply-to: <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org>
References: <201909051307.x85D7MGs034053@slippy.cwsent.com> 
 <20190905142817.GB2559@kib.kiev.ua>
 <201909060355.x863tRhP089169@slippy.cwsent.com>
 <201909060639.x866dJ7f090176@slippy.cwsent.com>
 <201909062356.x86NuKdk003780@slippy.cwsent.com>
 <156d1e7c-0dbb-8707-90b3-13ae97c87449@nwtime.org>
 <20190907075619.GG2559@kib.kiev.ua>
 <201909071309.x87D9GxZ089964@slippy.cwsent.com>
 <20190907153226.GI2559@kib.kiev.ua>
 <201909071545.x87FjLS6004448@slippy.cwsent.com>
 <20190907161749.GJ2559@kib.kiev.ua>
 <201909071628.x87GSFPV005827@slippy.cwsent.com>
 <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org>
Comments: In-reply-to Ian Lepore <ian@freebsd.org>
 message dated "Mon, 09 Sep 2019 10:23:40 -0600."
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Mon, 09 Sep 2019 12:21:38 -0700
X-CMAE-Envelope: MS4wfJtQIxh5ZXCHWA13c9h8NEFmOVKf05hF8UW4wVpcTjWQ+osI1B3orEhQr1RZqzteT6LpKEaU6LE/N72UrXmemDrdL1FVw7225J4NN0bMTgPpjpIo6p7z
 QD4IqtxawWrfG6z8ts+52npNW8x+49bVZg8V4E6wSZxzlEthzofuU1kDEwSkb3epeWptH8FNa8GWmy2XWOGInajVgDrf/idHj3skObBh1FziaMPkSdtr2MRD
 x+3zU4hiX8uuWMzK2uLk7g==
X-Rspamd-Queue-Id: 46RyhW3C97z4RbJ
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org; dkim=none;
 spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF
 policy when checking 64.59.134.13) smtp.mailfrom=cy.schubert@cschubert.com
X-Spamd-Result: default: False [-4.04 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5];
 HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com];
 FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[];
 HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_FIVE(0.00)[6];
 REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[13.134.59.64.list.dnswl.org : 127.0.5.0];
 R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA];
 RCVD_TLS_LAST(0.00)[];
 IP_SCORE(-2.44)[ip: (-6.65), ipnet: 64.59.128.0/20(-3.08), asn: 6327(-2.39),
 country: CA(-0.09)]; 
 RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net
 : 127.0.0.11]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 19:21:48 -0000

In message <66f80012757134b6317b673f9eeb24db66c996a2.camel@freebsd.org>, 
Ian Le
pore writes:
> On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin Belousov writes:
> > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote:
> > > > I've been able to set the memlock rlimit as low as 20 MB. The issue is 
> > > > letting it default to 0 which allows ntp to mlockall() anything it want
> s. 
> > > > ntpd on my sandbox is currently using 18 MB.
> > > 
> > > Default stack size on amd64 is 512M, and default stack gap percentage is
> > > 3%. This means that the gap can be as large as ~17MB. If 3MB is enough
> > > for the stack of the main thread of ntpd, then fine.
> > 
> > The default stack is 200K, which is also tuneable in ntp.conf.
> > 
> > [...]
>
> I haven't seen anyone ask what I consider to be the crucial question
> yet:  why are we locking ntpd into memory by default at all?
>
> I have seen two rationales for ntpd using mlockall() and setrlimit(): 
>
>    - There are claims that it improves timing performance.
>
>    - Because ntpd is a daemon that can run for months at a time,
>    setting limits on memory and stack growth can help detect and
>    mitigate against memory leak problems in the daemon.
>
> I am *highly* skeptical of claims that locking ntpd in memory will
> improve timing performance on freebsd (to the point where I'm inclined
> to dismiss them out of hand, but I'd be willing to look at any actual
> evidence).
>
> The second point has at least some validity, but would be better
> implemented (IMO) by decoupling the address space limit from the act of
> locking down memory, and allowing configuration of RLIMIT_AS separately
> from RLIMIT_MEMLOCK.  If there's any interest in this, I could try to
> put together a patchset and submit it upstream for that.

Our upstream is already cc'd on this thread.

diff --git a/usr.sbin/ntp/config.h b/usr.sbin/ntp/config.h
index 56dbfedba6e..b26dd417885 100644
--- a/usr.sbin/ntp/config.h
+++ b/usr.sbin/ntp/config.h
@@ -287,7 +287,7 @@
 #define DEFAULT_HZ 100
 
 /* Default number of megabytes for RLIMIT_MEMLOCK */
-#define DFLT_RLIMIT_MEMLOCK 32
+#define DFLT_RLIMIT_MEMLOCK -1
 
 /* Default number of 4k pages for RLIMIT_STACK */
 #define DFLT_RLIMIT_STACK 50

But I'd wait for Harlan to weigh in on this. I agree with kib@ that this 
may introduce unwanted jitter. I'd also want to understand why this 
defaults to -1 for Linux only.

>
> I would propose that for freebsd, the autoconf-generated value for

For the port but in base we control this directly.

> DFLT_RLIMIT_MEMLOCK should be -1 to avoid calling setrlimit() and
> mlockall() by default.  Then in the ntp.conf we distribute we have a
> section like:
>
>    # To lock ntpd into memory, uncomment the following rlimit line.
>    # This should not be necessary on most systems, but may improve
>    # performance on heavily-loaded servers experiencing enough
>    # memory pressure to cause ntpd to be paged out.
>    # rlimit memlock <something> stacksize <something>
>
> Then we would need to come up with reasonable values for <something>.

I haven't had a chance to look at this in depth yet but I suspect that 
<something> isn't in fact 32 as specified in config.h. It behaves as if 
this is set to 0 to mlockall() all it asks for.

>
> -- Ian


-- 
Cheers,
Cy Schubert <Cy.Schubert@cschubert.com>
FreeBSD UNIX:  <cy@FreeBSD.org>   Web:  http://www.FreeBSD.org

	The need of the many outweighs the greed of the few.



From owner-freebsd-current@freebsd.org  Mon Sep  9 19:28:43 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 08429DCF66
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 19:28:43 +0000 (UTC)
 (envelope-from cy.schubert@cschubert.com)
Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.138])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "Client", Issuer "CA" (not verified))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46RyrV0BJqz4S8k;
 Mon,  9 Sep 2019 19:28:41 +0000 (UTC)
 (envelope-from cy.schubert@cschubert.com)
Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA
 id 7PL2i1PpQIhW97PL3i2KAX; Mon, 09 Sep 2019 13:28:39 -0600
X-Authority-Analysis: v=2.3 cv=FcFJO626 c=1 sm=1 tr=0
 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17
 a=kj9zAlcOel0A:10 a=J70Eh1EUuV4A:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8
 a=dGt7y4mFooTMp67Xen0A:9 a=7Zwj6sZBwVKJAoWSPKxL6X1jA+E=:19 a=CjuIK1q_8ugA:10
 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22
Received: from slippy.cwsent.com (slippy8 [10.2.2.6])
 by spqr.komquats.com (Postfix) with ESMTPS id E8A6AAC0;
 Mon,  9 Sep 2019 12:28:35 -0700 (PDT)
Received: from slippy.cwsent.com (localhost [127.0.0.1])
 by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x89JSZ23062485;
 Mon, 9 Sep 2019 12:28:35 -0700 (PDT)
 (envelope-from Cy.Schubert@cschubert.com)
Received: from slippy (cy@localhost)
 by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x89JSZMm062482;
 Mon, 9 Sep 2019 12:28:35 -0700 (PDT)
 (envelope-from Cy.Schubert@cschubert.com)
Message-Id: <201909091928.x89JSZMm062482@slippy.cwsent.com>
X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs
X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1
Reply-to: Cy Schubert <Cy.Schubert@cschubert.com>
From: Cy Schubert <Cy.Schubert@cschubert.com>
X-os: FreeBSD
X-Sender: cy@cwsent.com
X-URL: http://www.cschubert.com/
To: Konstantin Belousov <kostikbel@gmail.com>
cc: Ian Lepore <ian@freebsd.org>,
 "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>,
 Cy Schubert <Cy.Schubert@cschubert.com>, Harlan Stenn <stenn@nwtime.org>,
 Vladimir Zakharov <zakharov.vv@gmail.com>, freebsd-current@freebsd.org
Subject: Re: ntpd segfaults on start
In-reply-to: <20190909184446.GU2559@kib.kiev.ua>
References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net> 
 <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org> 
 <20190909184446.GU2559@kib.kiev.ua>
Comments: In-reply-to Konstantin Belousov <kostikbel@gmail.com>
 message dated "Mon, 09 Sep 2019 21:44:46 +0300."
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Mon, 09 Sep 2019 12:28:35 -0700
X-CMAE-Envelope: MS4wfCNdLMMFu3T+xzAFxBXSvUKj0SzhkIIgldO5/QnxhzgxjV1wEHU17Yd6R+/KmL5Va0LZ3UcIP5BjQKQrofbyr2fZJiLiYiwEJaBkTpxZhzIUqBQS+Wqp
 LDmk5oNVllnFT3wYus0ZKaGHCramev9Qvdzu0dbLzYiBn0pyokfo5zBVnE20zs83lO4H3rTduhOMCkf9ZQSgcqKOTjZ9Tfn1iM64ZN1ymO/cuz6D9H2I2tr/
 2BAyflQHGM3LlnBS/VofA9mvzkv6GsLses6ss/RmcaquqNYRe5JGiE60mxVH+HCZ
X-Rspamd-Queue-Id: 46RyrV0BJqz4S8k
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org; dkim=none;
 spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF
 policy when checking 64.59.136.138) smtp.mailfrom=cy.schubert@cschubert.com
X-Spamd-Result: default: False [-2.42 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
 HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com];
 TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; HAS_XAW(0.00)[];
 RCPT_COUNT_SEVEN(0.00)[7]; FREEMAIL_TO(0.00)[gmail.com];
 FROM_EQ_ENVFROM(0.00)[];
 IP_SCORE(-2.32)[ip: (-6.06), ipnet: 64.59.128.0/20(-3.08), asn: 6327(-2.39),
 country: CA(-0.09)]; R_DKIM_NA(0.00)[];
 ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA];
 MIME_TRACE(0.00)[0:+];
 RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net
 : 127.0.0.11]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_COUNT_FIVE(0.00)[5];
 REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[];
 RCVD_TLS_LAST(0.00)[]; MIME_GOOD(-0.10)[text/plain];
 TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[138.136.59.64.list.dnswl.org : 127.0.5.0];
 R_SPF_NA(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 19:28:43 -0000

In message <20190909184446.GU2559@kib.kiev.ua>, Konstantin Belousov writes:
> On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote:
> > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote:
> > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin
> > > > > Belousov writes:
> > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert wrote:
> > > > > > > I've been able to set the memlock rlimit as low as 20 MB. The
> > > > > > > issue is 
> > > > > > > letting it default to 0 which allows ntp to mlockall()
> > > > > > > anything it wants. 
> > > > > > > ntpd on my sandbox is currently using 18 MB.
> > > > > > 
> > > > > > Default stack size on amd64 is 512M, and default stack gap
> > > > > > percentage is
> > > > > > 3%. This means that the gap can be as large as ~17MB. If 3MB is
> > > > > > enough
> > > > > > for the stack of the main thread of ntpd, then fine.
> > > > > 
> > > > > The default stack is 200K, which is also tuneable in ntp.conf.
> > > > > 
> > > > > [...]
> > > > 
> > > > I haven't seen anyone ask what I consider to be the crucial
> > > > question
> > > > yet:  why are we locking ntpd into memory by default at all?
> > > > 
> > > > I have seen two rationales for ntpd using mlockall() and
> > > > setrlimit(): 
> > > > 
> > > >    - There are claims that it improves timing performance.
> > > > 
> > > >    - Because ntpd is a daemon that can run for months at a time,
> > > >    setting limits on memory and stack growth can help detect and
> > > >    mitigate against memory leak problems in the daemon.
> > > 
> > > Doesn't locking this memory down also protect ntpd from OOM kills?
> > > If so that is a MUST preserve functionality, as IMHO killing ntpd
> > > on a box that has it configured is a total no win situation.
> > > 
> > 
> > Does it have that effect?  I don't know.  But I would argue that that's
> > a separate issue, and we should make that happen by adding
> > ntpd_oomprotect=YES to /etc/defaults/rc.conf
> Wiring process memory has no effect on OOM selection. More, because
> all potentially allocated pages are allocated for real after mlockall(),
> the size of the vmspace, as accounted by OOM, is the largest possible
> size from the whole lifetime.
>
> On the other hand, the code execution times are not predictable if the
> process's pages can be paged out. Under severe load next instruction
> might take several seconds or even minutes to start. It is quite unlike
> the scheduler delays. That introduces a jitter in the local time
> measurements and their usage as done in userspace. Wouldn't this affect
> the accuracy ?

IMO it would and would be unacceptable when used as a stratum N server or 
with some OLTP or database applications. Locking a few megabytes is a small 
cost.

>
> > 
> > Right now only syslogd has oomprotect set to YES by default.  Maybe
> > that's a good choice -- once we start declaring one daemon to be more
> > important than others, you'll discover there's a whole back lot full of
> > bikesheds that need painting.
> > 
> > So maybe we should just document ntpd_oomprotect=YES in some more-
> > prominent way.  If we were to add a comment block to ntp.conf
> > describing rlimit, that might be a good place to mention setting
> > ntpd_oomprotect in rc.conf.
> > 
> > -- Ian
> > 


-- 
Cheers,
Cy Schubert <Cy.Schubert@cschubert.com>
FreeBSD UNIX:  <cy@FreeBSD.org>   Web:  http://www.FreeBSD.org

	The need of the many outweighs the greed of the few.



From owner-freebsd-current@freebsd.org  Mon Sep  9 20:14:18 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 330FBDF17F
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 20:14:18 +0000 (UTC) (envelope-from neel@neelc.org)
Received: from nyc1.neelc.org (nyc1.neelc.org [IPv6:2a0d:5600:33:11::2])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Rzs51zZQz4ZM9
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 20:14:16 +0000 (UTC)
 (envelope-from neel@neelc.org)
Received: from mail.neelc.org (nyc1.neelc.org [IPv6:2a0d:5600:33:11::2])
 by nyc1.neelc.org (Postfix) with ESMTPSA id 32D7914D3E5
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 16:07:39 -0400 (EDT)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Mon, 09 Sep 2019 16:07:39 -0400
From: Neel Chauhan <neel@neelc.org>
To: freebsd-current@freebsd.org
Subject: TSC-low clock slow on WhiskeyLake i7-8565U (HP Spectre x360
 13-ap0043dx)
Message-ID: <72978644df330013de27c75e6285ab4d@neelc.org>
X-Sender: neel@neelc.org
User-Agent: Roundcube Webmail/1.3.9
X-Rspamd-Queue-Id: 46Rzs51zZQz4ZM9
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of neel@neelc.org designates
 2a0d:5600:33:11::2 as permitted sender) smtp.mailfrom=neel@neelc.org
X-Spamd-Result: default: False [-2.30 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0];
 FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1];
 NEURAL_HAM_LONG(-1.00)[-0.998,0]; DMARC_NA(0.00)[neelc.org];
 IP_SCORE(-0.11)[asn: 63473(-0.50), country: US(-0.05)];
 RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[];
 R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:63473, ipnet:2a0d:5600:33::/48, country:US];
 MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[];
 ONCE_RECEIVED(0.10)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 20:14:18 -0000

Hi,

I recently got a HP Spectre x360 13-ap0043dx as a gift and by default 
the clock runs slower on FreeBSD than Windows or Linux. Yes, I know the 
ThinkPad is the best laptop for FreeBSD, but getting an X1 Carbon would 
increase the price of the gift even more which couldn't be done.

The kern.timecounter.hardware by default is set to TSC-low and the clock 
is slow on the Spectre x360. Setting it to ACPI-slow resolves this 
issue.

The CPU is a Intel WhiskeyLake Core i7-8565U.

Is the slow TSC-low an issue with WhiskeyLake in general or specifically 
HP? Is it something else?

I am considering writing a patch but before I write one, do other people 
with WhiskeyLake laptops (or newer) have slow clocks (where one second 
on the system is actually more in real life), or not.

-Neel

===

https://www.neelc.org/

From owner-freebsd-current@freebsd.org  Mon Sep  9 21:12:24 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id D6B01E175F
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 21:12:24 +0000 (UTC) (envelope-from ian@freebsd.org)
Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org
 [54.149.155.156])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46S1883K7mz4fd9
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 21:12:24 +0000 (UTC)
 (envelope-from ian@freebsd.org)
ARC-Seal: i=1; a=rsa-sha256; t=1568063543; cv=none;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 b=oqbi0IAv/cKwJMQFruTgPztIVmNBTbTr9ZidypcZf2YRLTA8Lm4B+7F3Zq7x0fh7yp/VXnihPeFll
 40iyxUuymg+TPEZaIWXJzlHaDcWdIuwW8EKnekgvf7VdDEHlAUC/OfYbrf3QTNAQTmbwSk05443JGs
 Iw5J7XO9hxMjUuximoK8HWEDm+CYoNC9+8h1VZqz1xRyKZVFdGTiMqnYpl70PrO5U50LmJol1wGeAY
 4EubSgMZTdNpkYHxPwljT6kbOSm82bDppJCBQZY1nyph/lkxJey5COYWG/p9g0mUKHY0lbyyPSF4Cd
 xGxD7yfrV7ghdWNaIvQgUMAcyCHL07g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:dkim-signature:from;
 bh=Bv2ydlIUZ+kCz/pyFE3b2tNML0DRTgLRmiKYZUrQKvk=;
 b=Ggl4wP5AJLe9h6yE3WNlbv0DwvxlJ1NlfcVttSosn4LatykkN1GhcoZbtU1xv9anmXO5S4ljCHtO3
 qu013Qt+1HKJqr1ue53Gxx8xmj5ukr/amhpgdM4gZw5srH+R0WvFWpcuZmSv1FRPi/P5+/mIQvkpRJ
 tfXvKNfwQDWiCFjU3xsaiBLJubVqL+BUJS0hruPFXpxX/tFdsKT3gkjIuHsvI6ar/lo740uaM1cvwL
 lMj5pKUtVYAMlleVmPQaRvcY8QeYFNJ+npxnrghH3kl1ZpimBZPkUdIK+U8vUYj4i6/WZg1YCAkDkF
 4c9gs8pPTXznOXYOI6pb7//yNK81TOA==
ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org;
 spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60;
 dmarc=none header.from=freebsd.org;
 arc=none header.oldest-pass=0;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=dkim-high;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:from;
 bh=Bv2ydlIUZ+kCz/pyFE3b2tNML0DRTgLRmiKYZUrQKvk=;
 b=H17egwVEMpkmRlxWj+DYCZsPcoVOZXK636hAGZWnJkYTGEmuJamQCamzOLTeAhMeJdooAq3ZeDLoJ
 1RjbRbVQPktOqplLbecgIDcbM0L5kyaHjFg5NdN52llusnQq3/9w6MoTVUlcX6BXgc5yyuDKdku7ma
 4b5Y2HcmJgRbU5Gs6jedJaJ683qQjpaz6Ys1XNPzBSZ+xkpFk2quvcaZhfB5pE9kf3zGuwrDFI4rfJ
 hiKSpoVU9nLUuInIJ9VOD5G954rsWAidE+av9Bo3p7XdG4XhUMh1WARf+O+JbH4AwjSFdlg12QDnQV
 TKkala8PPUvJN/YzKYt9yypPWd8Y8LQ==
X-MHO-RoutePath: aGlwcGll
X-MHO-User: 82052336-d346-11e9-85ed-13b9aae3a1d2
X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information
X-Originating-IP: 67.177.211.60
X-Mail-Handler: DuoCircle Outbound SMTP
Received: from ilsoft.org (unknown [67.177.211.60])
 by outbound4.ore.mailhop.org (Halon) with ESMTPSA
 id 82052336-d346-11e9-85ed-13b9aae3a1d2;
 Mon, 09 Sep 2019 21:12:21 +0000 (UTC)
Received: from rev (rev [172.22.42.240])
 by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89LCJk0063759;
 Mon, 9 Sep 2019 15:12:19 -0600 (MDT) (envelope-from ian@freebsd.org)
Message-ID: <c12493eb2f1bcd98303a33c0e1b0e8756d13b245.camel@freebsd.org>
Subject: Re: ntpd segfaults on start
From: Ian Lepore <ian@freebsd.org>
To: Konstantin Belousov <kostikbel@gmail.com>
Cc: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>, Cy Schubert
 <Cy.Schubert@cschubert.com>, Harlan Stenn <stenn@nwtime.org>, Vladimir
 Zakharov <zakharov.vv@gmail.com>, freebsd-current@freebsd.org
Date: Mon, 09 Sep 2019 15:12:18 -0600
In-Reply-To: <20190909184446.GU2559@kib.kiev.ua>
References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net>
 <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org>
 <20190909184446.GU2559@kib.kiev.ua>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46S1883K7mz4fd9
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-2.00 / 15.00]; TAGGED_RCPT(0.00)[];
 local_wl_from(0.00)[freebsd.org];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 21:12:24 -0000

On Mon, 2019-09-09 at 21:44 +0300, Konstantin Belousov wrote:
> On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote:
> > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote:
> > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin
> > > > > Belousov writes:
> > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert
> > > > > > wrote:
> > > > > > > [...]
> > > 
> > > Doesn't locking this memory down also protect ntpd from OOM kills?
> > > If so that is a MUST preserve functionality, as IMHO killing ntpd
> > > on a box that has it configured is a total no win situation.
> > > 
> > 
> > Does it have that effect?  I don't know.  But I would argue that that's
> > a separate issue, and we should make that happen by adding
> > ntpd_oomprotect=YES to /etc/defaults/rc.conf
> 
> Wiring process memory has no effect on OOM selection. More, because
> all potentially allocated pages are allocated for real after mlockall(),
> the size of the vmspace, as accounted by OOM, is the largest possible
> size from the whole lifetime.
> 
> On the other hand, the code execution times are not predictable if the
> process's pages can be paged out. Under severe load next instruction
> might take several seconds or even minutes to start. It is quite unlike
> the scheduler delays. That introduces a jitter in the local time
> measurements and their usage as done in userspace. Wouldn't this affect
> the accuracy ?
> 

IMO, there is a large gap between "in theory, paging could cause
indeterminate delays in code execution" and "time will be inaccurate on
your system".  If there were a delay in a part of the code where it
matters that amounted to "seconds or even minutes", what you'd end up
with is a measurement that would be discarded by the median filter as
an outlier.  There would be some danger that if that kind of delay
happened for too many polling cycles in a row, you'd end up with no
usable measurements after a while and clock accuracy would suffer. 
Sub-second delays would be more worriesome because they might not be
rejected as outliers.

There are only a couple code paths in freebsd ntpd processing where a
paging (or scheduling) delay could cause measurement inaccuracy:

 - When stepping the clock, the code that runs between calling
clock_gettime() and calling clock_settime() to apply the step
adjustment to the clock.

 - When beginning an exchange with or replying to a peer, the code that
runs between obtaining system time for the outgoing Transmit Timestamp
and actually transmitting that packet.

Stepping the clock typically only happens once at startup.  The ntpd
code itself recognizes that this is a time-critical path (it has
comments to that effect) but unfortunately the code that runs is
scattered among several different .c files so it's hard to say what the
likelyhood is that code in the critical section will all be in the same
page (or be already-resident because other startup-time code faulted in
those pages).  IMO, the right fix for this would be a kernel interface
that let you apply a step-delta to the clock with a single syscall
(perhaps as an extension to the existing ntp_adjtime() using a new mode
flag).

On freebsd, the Receive timestamps are captured in the kernel and
delivered along with the packet to userland, and are retrieved by the
ntpd code from the SCM_BINTIME control message in the packet, so there
is no latency problem in the receive path.  There isn't a corresponding
kernel mechanism for setting the outgoing timestamps, so whether it's
originating a request to a peer or replying to a request from a peer,
the transmit timestamp could be wrong due to:

 - paging delays
 - scheduler delays
 - network stack, outgoing queues, and driver delays

So the primary vulnerability is on the transmit path between obtaining
system time and the packet leaving the system.  A quick glance at that
code makes me think that most of the data being touched has already
been referenced pretty recently during the process of assembling the
outgoing packet, so it's unlikely that storing the timestamp into the
outgoing packet or the other bit of work that happens after that
triggers a pagein unless the system is pathologically overloaded. 
Naturally, obtaining the timestamp and putting it into the packet is
one of the last things it does before sending, so the code path is
relatively short, but it's not clear to me whether it's likely or not
that the code involved all lives in the same page.  Still, it's one of
the heavily exercised paths within ntpd, which should increase the odds
of the pages being resident because of recent use.

So, I'm not disputing the point that a sufficiently overloaded system
can lead to an indeterminate delay between *any* two instructions
executed in userland.  What I've said above is more along the lines of
considering the usual situation, not the most pathlogical one.  In the
most pathological cases, either the delays introduced are fairly minor
and you get some minor jitter in system time (ameliorated by the median
filtering built in to ntpd), or the delays are major (a full second or
more) and get rejected as outliers, not affecting system time at all
unless the situation persists and prevents getting any good
measurements for many hours.

-- Ian



From owner-freebsd-current@freebsd.org  Mon Sep  9 21:17:00 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id CFD29E19A5
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 21:17:00 +0000 (UTC) (envelope-from ian@freebsd.org)
Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org
 [54.186.57.195])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46S1FS33sJz4fxD
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 21:17:00 +0000 (UTC)
 (envelope-from ian@freebsd.org)
ARC-Seal: i=1; a=rsa-sha256; t=1568063818; cv=none;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 b=AvOJBCLAJm/BOLWBjnzW9nIxnLi6q5W1nY5M3K4flQmsDdoESPci2bzqKcXrOZ8Tm/WkRMoQQf5KZ
 rETrMwP4x3kj5g1oO8RPoQGSDYQFvUpT06tWqrkX02BwdHcmLJCv6/YHHZxaWNJVzQMENNGKspFilp
 GqZjhoaNB4KmaTac1wTfYpp+xn6hhJfu7XgqVqixq4VmHr+1sczc/7Qu2i+ELWJ39V1oQPiNIC4AxI
 ml0kPhZU/vdIAy/HiCq0fM7TVW21XsA11FQeQKZwDk8uSAbgcjvjspyZ6qAFGZKknVtiqq8WceyIRL
 g/gYGCcCa9jy6AimlbNlIMzkQd528PA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:dkim-signature:from;
 bh=K8JlJv5D8SqwpQDVvyqkrrgFFVQ7wa3mT+zv7Vk5bJA=;
 b=qvp/zU3IOmsNoHmKWO2o8njhO0tyYRGMo+Sn7dSMjuwNQ/4x2giW++gGC7g2jjD47RYxVUifI5Gej
 K0jMvQk1lfyi5VNItepTA5pUWQMCSlbQVF2ianGeQ2voLgzvSSpc1rF/NzKQus4vpsRTkrM5p8mUq6
 FGOulReOhupzWekwXN1/XJm9ZW7ivfRaOkywHC4IK9VpV8kq155GocvRp8XA/rY95dBa1wrb/Qcr2z
 tQqzkrOPhaa9YMQqizLsr9hWd2qC3DlnxLAC2py3NCVDKPO8qMz/wcKIQFc7J0KDQwaz4pWV+CSgAJ
 +bxaDFqdwPCwdKEXPEOD783ArsuWCog==
ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org;
 spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60;
 dmarc=none header.from=freebsd.org;
 arc=none header.oldest-pass=0;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=dkim-high;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:from;
 bh=K8JlJv5D8SqwpQDVvyqkrrgFFVQ7wa3mT+zv7Vk5bJA=;
 b=V5Xxjj01rwxohA69FqToybtkzl5syv4IfWI7y5ph9eg53Tc0c5g7cQFUSaZkCWsj+04MZo5kQHNHQ
 pbv+IC9+0x6demgpZW+7C1Tuoa45bDmDpKjoXcKAXoCQTTiwiaX/iZUaRvJExAzUyHQCovHqaLYGyy
 X8rByfL0+CbaQeD2H4PPQ36Qzw5lH6qPYDhrYvLRMQPeV9VwYtnmZt7YphaD/LFvOC4IKHZYbaSPN/
 khCevnF9R4TYO9Ja23NX0bC5+yK3lIkmiPCXsCv0h9uT+Nmhuw4X52pJT9bFbIPlLM+JMvInLH/CpS
 8h1FMcF27Y4lzOBXuCDEUX3QeGG/duw==
X-MHO-RoutePath: aGlwcGll
X-MHO-User: 24b2fc5f-d347-11e9-b67d-cdd75d6ce7a8
X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information
X-Originating-IP: 67.177.211.60
X-Mail-Handler: DuoCircle Outbound SMTP
Received: from ilsoft.org (unknown [67.177.211.60])
 by outbound3.ore.mailhop.org (Halon) with ESMTPSA
 id 24b2fc5f-d347-11e9-b67d-cdd75d6ce7a8;
 Mon, 09 Sep 2019 21:16:57 +0000 (UTC)
Received: from rev (rev [172.22.42.240])
 by ilsoft.org (8.15.2/8.15.2) with ESMTP id x89LGqjV063776;
 Mon, 9 Sep 2019 15:16:52 -0600 (MDT) (envelope-from ian@freebsd.org)
Message-ID: <b4051fd6d6f50dabaaff7dfefb5d47148811126a.camel@freebsd.org>
Subject: Re: ntpd segfaults on start
From: Ian Lepore <ian@freebsd.org>
To: Cy Schubert <Cy.Schubert@cschubert.com>, Konstantin Belousov
 <kostikbel@gmail.com>
Cc: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>, Harlan Stenn
 <stenn@nwtime.org>,
 Vladimir Zakharov <zakharov.vv@gmail.com>, freebsd-current@freebsd.org
Date: Mon, 09 Sep 2019 15:16:52 -0600
In-Reply-To: <201909091928.x89JSZMm062482@slippy.cwsent.com>
References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net>
 <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org>
 <20190909184446.GU2559@kib.kiev.ua>
 <201909091928.x89JSZMm062482@slippy.cwsent.com>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46S1FS33sJz4fxD
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-2.00 / 15.00];
 local_wl_from(0.00)[freebsd.org];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[];
 ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 21:17:00 -0000

On Mon, 2019-09-09 at 12:28 -0700, Cy Schubert wrote:
> > 
> > On the other hand, the code execution times are not predictable if the
> > process's pages can be paged out. Under severe load next instruction
> > might take several seconds or even minutes to start. It is quite unlike
> > the scheduler delays. That introduces a jitter in the local time
> > measurements and their usage as done in userspace. Wouldn't this affect
> > the accuracy ?
> 
> IMO it would and would be unacceptable when used as a stratum N server or 
> with some OLTP or database applications. Locking a few megabytes is a small 
> cost.

What I proposed was changing the default to not lock all of ntpd into
memory, because I think that would work well for most users.  Admins
running stratum 1 or 2 servers are probably a bit more savvy about ntp
configuration than the average user, and would use the rlimit command
in ntp.conf.

-- Ian


From owner-freebsd-current@freebsd.org  Mon Sep  9 21:42:56 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id C585FE266F
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Mon,  9 Sep 2019 21:42:56 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com
 [IPv6:2607:f8b0:4864:20::842])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46S1qM60nkz4hWD
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 21:42:55 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: by mail-qt1-x842.google.com with SMTP id c9so18128648qth.9
 for <freebsd-current@freebsd.org>; Mon, 09 Sep 2019 14:42:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=bsdimp-com.20150623.gappssmtp.com; s=20150623;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=1dFVajpxXasov17Y9iIIJ8EAzEXfFSpxDyhlcCvDCQI=;
 b=wOoxKvJUJ9wvl+EEqu4H7k3ecdBVa4sM+ivLY01og/kDPHo0dia/414bOLScIk+XG8
 QVFlwz2MqEPOvuCpEQ34nLiHtH6AIdqnZBN8DzqnmfSydfwn6HZ/guXxLoIGOAEu15z/
 DFTpwtGn/1osPxtORk9veJeZgf1khKuZao0TlxC417RUi3JlW9If9KRJqNFRAPlbZxzL
 9rCoMKmPlKRgjUtAOZ+iW0efyfzkMDmZf+bFPLt8ZBjdRGzrzH/He9zdSF3EA8P1SS8C
 KXSmYQcAfvvco0N5iEUiZEu7MslY9QMBUIognOyonsfvp9gdF7A1I5fDNUOzFT43q+V/
 9Glw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=1dFVajpxXasov17Y9iIIJ8EAzEXfFSpxDyhlcCvDCQI=;
 b=nZBV1Z/CEdOWL3M1zurGQVABIyktYgJ/nGKdLonxGBvLppHgPT9OFd6xb3Qkc2Vi6A
 MdbcCZPvOvyYLDHwg55rwAqHMNH+RVPNWVLZvBLRl5+FUJC9dX+WAswbw/duwWyC7Lfo
 vw9cvIn2Q0UBF/yhqePqLAvSyGJjaQfY8v4GMsCf56mju03OKRdLYUXAD4w+WJN2h8tX
 s8LWWzQoOgkv7+Pl4R0s51tL6KetsC+BKWtEoFYvMH8EElw+IPSxhBUdGk3X4IZBNi+T
 ullyY6MaAfvSRH7ex9OwvVPnv/qVOUFNFu6S3QhfTtKaiJKJdluMVzApTglA9o98ewxE
 L0vw==
X-Gm-Message-State: APjAAAXqZwPZnaLsZqmeJekHBDjzhJs7F6J8GUwe8q4dAjDdMVsKpq1d
 Yh8VusF2pEAvq5eFXP7g7+Ka0k3ePmCmf9cVcmvLSw==
X-Google-Smtp-Source: APXvYqzYeW2qJ5ZcC2svv/0Q1jJX857T4V6a6ii43l206NAo1GVv1UP/2e+rr3FAV0rh+/Ib2ruvQbK46ste68kyzps=
X-Received: by 2002:ac8:760e:: with SMTP id t14mr25774010qtq.175.1568065374445; 
 Mon, 09 Sep 2019 14:42:54 -0700 (PDT)
MIME-Version: 1.0
References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net>
 <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org>
 <20190909184446.GU2559@kib.kiev.ua>
 <c12493eb2f1bcd98303a33c0e1b0e8756d13b245.camel@freebsd.org>
In-Reply-To: <c12493eb2f1bcd98303a33c0e1b0e8756d13b245.camel@freebsd.org>
From: Warner Losh <imp@bsdimp.com>
Date: Mon, 9 Sep 2019 15:42:43 -0600
Message-ID: <CANCZdfptNY_sYcKAXZYmBErRgFtZD_gh_ve=4YxPL5xHEL5StQ@mail.gmail.com>
Subject: Re: ntpd segfaults on start
To: Ian Lepore <ian@freebsd.org>
Cc: Konstantin Belousov <kostikbel@gmail.com>,
 "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>, 
 Cy Schubert <Cy.Schubert@cschubert.com>, Harlan Stenn <stenn@nwtime.org>, 
 Vladimir Zakharov <zakharov.vv@gmail.com>,
 FreeBSD Current <freebsd-current@freebsd.org>
X-Rspamd-Queue-Id: 46S1qM60nkz4hWD
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623
 header.b=wOoxKvJU; dmarc=none;
 spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when
 checking 2607:f8b0:4864:20::842) smtp.mailfrom=wlosh@bsdimp.com
X-Spamd-Result: default: False [-1.08 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623];
 RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[];
 RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 TAGGED_RCPT(0.00)[];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[bsdimp.com];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+];
 RCVD_IN_DNSWL_NONE(0.00)[2.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[7]; R_SPF_NA(0.00)[];
 FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com];
 MIME_TRACE(0.00)[0:+,1:+,2:~];
 IP_SCORE(-0.58)[ip: (2.17), ipnet: 2607:f8b0::/32(-2.74), asn: 15169(-2.26),
 country: US(-0.05)]; 
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com];
 SUSPICIOUS_RECIPS(1.50)[]; FREEMAIL_CC(0.00)[gmail.com]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Sep 2019 21:42:56 -0000

On Mon, Sep 9, 2019 at 3:12 PM Ian Lepore <ian@freebsd.org> wrote:

> On Mon, 2019-09-09 at 21:44 +0300, Konstantin Belousov wrote:
> > On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote:
> > > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote:
> > > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin
> > > > > > Belousov writes:
> > > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert
> > > > > > > wrote:
> > > > > > > > [...]
> > > >
> > > > Doesn't locking this memory down also protect ntpd from OOM kills?
> > > > If so that is a MUST preserve functionality, as IMHO killing ntpd
> > > > on a box that has it configured is a total no win situation.
> > > >
> > >
> > > Does it have that effect?  I don't know.  But I would argue that that's
> > > a separate issue, and we should make that happen by adding
> > > ntpd_oomprotect=YES to /etc/defaults/rc.conf
> >
> > Wiring process memory has no effect on OOM selection. More, because
> > all potentially allocated pages are allocated for real after mlockall(),
> > the size of the vmspace, as accounted by OOM, is the largest possible
> > size from the whole lifetime.
> >
> > On the other hand, the code execution times are not predictable if the
> > process's pages can be paged out. Under severe load next instruction
> > might take several seconds or even minutes to start. It is quite unlike
> > the scheduler delays. That introduces a jitter in the local time
> > measurements and their usage as done in userspace. Wouldn't this affect
> > the accuracy ?
> >
>
> IMO, there is a large gap between "in theory, paging could cause
> indeterminate delays in code execution" and "time will be inaccurate on
> your system".  If there were a delay in a part of the code where it
> matters that amounted to "seconds or even minutes", what you'd end up
> with is a measurement that would be discarded by the median filter as
> an outlier.  There would be some danger that if that kind of delay
> happened for too many polling cycles in a row, you'd end up with no
> usable measurements after a while and clock accuracy would suffer.
> Sub-second delays would be more worriesome because they might not be
> rejected as outliers.
>
> There are only a couple code paths in freebsd ntpd processing where a
> paging (or scheduling) delay could cause measurement inaccuracy:
>
>  - When stepping the clock, the code that runs between calling
> clock_gettime() and calling clock_settime() to apply the step
> adjustment to the clock.
>
>  - When beginning an exchange with or replying to a peer, the code that
> runs between obtaining system time for the outgoing Transmit Timestamp
> and actually transmitting that packet.
>
> Stepping the clock typically only happens once at startup.  The ntpd
> code itself recognizes that this is a time-critical path (it has
> comments to that effect) but unfortunately the code that runs is
> scattered among several different .c files so it's hard to say what the
> likelyhood is that code in the critical section will all be in the same
> page (or be already-resident because other startup-time code faulted in
> those pages).  IMO, the right fix for this would be a kernel interface
> that let you apply a step-delta to the clock with a single syscall
> (perhaps as an extension to the existing ntp_adjtime() using a new mode
> flag).
>
> On freebsd, the Receive timestamps are captured in the kernel and
> delivered along with the packet to userland, and are retrieved by the
> ntpd code from the SCM_BINTIME control message in the packet, so there
> is no latency problem in the receive path.  There isn't a corresponding
> kernel mechanism for setting the outgoing timestamps, so whether it's
> originating a request to a peer or replying to a request from a peer,
> the transmit timestamp could be wrong due to:
>
>  - paging delays
>  - scheduler delays
>  - network stack, outgoing queues, and driver delays
>
> So the primary vulnerability is on the transmit path between obtaining
> system time and the packet leaving the system.  A quick glance at that
> code makes me think that most of the data being touched has already
> been referenced pretty recently during the process of assembling the
> outgoing packet, so it's unlikely that storing the timestamp into the
> outgoing packet or the other bit of work that happens after that
> triggers a pagein unless the system is pathologically overloaded.
> Naturally, obtaining the timestamp and putting it into the packet is
> one of the last things it does before sending, so the code path is
> relatively short, but it's not clear to me whether it's likely or not
> that the code involved all lives in the same page.  Still, it's one of
> the heavily exercised paths within ntpd, which should increase the odds
> of the pages being resident because of recent use.
>
> So, I'm not disputing the point that a sufficiently overloaded system
> can lead to an indeterminate delay between *any* two instructions
> executed in userland.  What I've said above is more along the lines of
> considering the usual situation, not the most pathlogical one.  In the
> most pathological cases, either the delays introduced are fairly minor
> and you get some minor jitter in system time (ameliorated by the median
> filtering built in to ntpd), or the delays are major (a full second or
> more) and get rejected as outliers, not affecting system time at all
> unless the situation persists and prevents getting any good
> measurements for many hours.
>

I've read through all this and agree with it as well. Paging delays can
happen, but if they do who cares: the measurements will be rejected as
outliers for long delays, but might introduce some noise if the delay is on
the order of tens of milliseconds. Shorter may won't matter long term: they
will average out. Longer will definitely be rejected. And it will likely be
just for the first packet in the exchange since the code path will be
paged/swapped into the working set for that.  The loop is a combination of
PLL/FLL so that if we think there's a big phase step, we'll also think
there's a frequency error. Both are steered out the same way: by setting
the offset. But ntpd is wise enough to know that there will be noisy
measurements, so even if the measurements make it through the filters, we
only remove a portion of the error each polling interval anyway. Any over
or undershoot will be correct the next measurement interval. And if you
have so much memory pressure that ntpd is paged out every single
measurement interval, your system likely needs careful tuning anyway.

In days of yore (like the mid 90s), these defaults were setup when it took
tens or even hundreds of milliseconds to page in it mattered. Today those
numbers are submillisecond to single digit milliseconds, so in the typical
case the disruption is much less severe than it was when things were
initially locked into memory. I'm guessing that's why Linux has moved to -1
for default. In addition, ntpd's algorithms have improved somewhat since
then as well to cope with noise. These days, good ntpd performance is
submillisecond over the internet, so the noise has to be approximately on
that order to affect the filters in ntpd. On the LAN you're good to 10's of
microseconds, typically, so any noise > several 10's of microseconds would
be eliminated as an outlier anyway.  I strongly suspect careful
measurements will declare no difference in performance except, maybe, on
the most overloaded of servers (and then it would need to be extremely
overloaded to have a delay in scheduling often enough to matter).

For people that want to be sure, by all means lock it into memory. But as a
default, I'm extremely skeptical one could measure a difference, at all,
let alone measure a difference that matters.

Warner

From owner-freebsd-current@freebsd.org  Tue Sep 10 03:25:23 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id B35ABEDC1A
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 03:25:23 +0000 (UTC)
 (envelope-from rebecca@bsdio.com)
Received: from muon.bsdio.com (muon.bluestop.org [65.103.231.193])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46S9QV4Yp1z3K9J
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 03:25:22 +0000 (UTC)
 (envelope-from rebecca@bsdio.com)
Received: from muon.bsdio.com (localhost [127.0.0.1])
 by muon.bsdio.com (Postfix) with ESMTP id 5EDCF108956
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 21:25:27 -0600 (MDT)
Received: from muon.bsdio.com ([127.0.0.1])
 by muon.bsdio.com (muon.bsdio.com [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id 5nxINbmbwiN7 for <freebsd-current@freebsd.org>;
 Mon,  9 Sep 2019 21:25:26 -0600 (MDT)
Received: from photon.int.bluestop.org (unknown [10.0.10.120])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest
 SHA256) (No client certificate requested)
 by muon.bsdio.com (Postfix) with ESMTPSA
 for <freebsd-current@freebsd.org>; Mon,  9 Sep 2019 21:25:26 -0600 (MDT)
To: FreeBSD Current <freebsd-current@freebsd.org>
From: Rebecca Cran <rebecca@bsdio.com>
Subject: Building world with gcc9 not working
Autocrypt: addr=rebecca@bsdio.com; keydata=
 mQINBFrUMZ4BEADI1yUEGeZeXeTCPay1ZpTBdDEpGPAw1dq2VCSTc1VhsnrEBa1iZxAfaeSv
 Uu5Ti7jlhQ/3sQMl0bJMKGB/RtmIW7k8h2w476oZmG8gChk8su5ZEx/pV1gdqInyFmmJKTYc
 gabJz8pL+m82w07qPv+oalepZ4dbj+HF++RAK/iEju+q9UHlsjj8e3mMNsvtrOz1K6bnpveO
 jZ+ms/2H3Hs5a4k8y6buwe2RvwhJQaXa13cR3LhzL+nwj4B9PHZZEa2WpEyYpw/bI0V9YSQN
 QgC1CYRzDyakZge6BCM6wHOgZSUzRPufGilrNKUwIVbRoIBR9/85+0wR+PlFUOUOfOc6ox7T
 dWcIx6PuPhek48rh4uwmmwsPtPiH4Z3T5p+GmWQ9NLFZKA1YnEdaSkWtYZsDxwVZZeYG2plt
 MfhXP0Hj4rf9Y3eoUenCaGioxAbUOBCtXdTGNAhNjz1g5NGDBVyhjKkzwJQvt9UrYTseERit
 5dX2CMTy8hYLvSXd/Ivy+HylUS5IslfZxW5z9LgWX7Z97kILgkH3N0ewtLkygkG+Y+x7uaAV
 dFqp9ASOyzaiwKbJdeOI+WxRSh+AqeCR0S+bpkcLudLmbjrPmaFwjKycy1H85Z5R2J3YHyXY
 oT6OYjD8vLbUU2GWp6Onkcy1Pu8EMbRuzKil6HnpYg3BexbPFwARAQABtCNSZWJlY2NhIENy
 YW4gPHJlYmVjY2FAYmx1ZXN0b3Aub3JnPokCVwQTAQgAQQIbIwUJCWYBgAULCQgHAgYVCgkI
 CwIEFgIDAQIeAQIXgBYhBB+5fZtkTdO940Yr4g0CK1MRvhAgBQJa2B9zAhkBAAoJEA0CK1MR
 vhAgzJEQAJUqVmTRO9OqCSS2CVKjrqnEWJMvyo0K8B+WiXo0nSQg9+uyoVU7h2s/kkWVGy4u
 IWbGy2Qe8LiXzBJjHC3TadGvOvakfdMeKKXcgxgX6KlhA9hA2LW6tg22aHUk7Flr/8diHpgf
 qIwrXhqJXZmK72GR1QfhgoHsOsTJ9GWPswo1kUMc0cJowq0qP1RDdua6BwvDHHPJwu9OmC/i
 oQlMNm9gkBDq8H2B+m125ANwCnqBizXaiTTLQdewTMbCSuxbsni2icDqwBfFXzEgcJGaYYfB
 cQeFsfCmtXQK3JUd4Myx128Dxk9P3X64I93SB7QzB0nmWlyvmCFBNoCp0PCLA4qbwbw2sMRX
 Wx4BqYa8nI/jg+Nqo+Ut2BfltNZIlsHxK+XhxejfLqAjRCZeLnu1otvFnFuGLaAVYx9x1Y1q
 J8VizZxq6ujio62Qpultp6KNhlkJ+OKoGwA0k4NHh26SxvlsNxlfg/2v9b1LqWRzNujnwbcF
 8g4902XjyBLxV+9YpXZEa8H6zzEHxpeDPWT3QfvrT8JuoHa1IyYnUKvG674UKW5zEGEwkQc9
 cuQwR1RHd1ZrKtH1duXzaLr/caMp8ZDfGDDxFpenJTRxNRlg4+K7HSdhpac7sBVMUA8uVdE+
 iuTThOmdf0c4DorL3BIh6Yv3FV4/NSqT1Wn3CG2fgG1guQINBFrUMZ4BEADkc4mvMcMcDF1t
 dNxNQuIBE1F243oZamG3LACCKfc1Yur3CPzHwIk5LXCUmbq23iE5bowxMWw3mlVT0p5xM0Wn
 UidIBwCKu4kRyy/fY4NyWWBuwy9srpTdmUcKRBRNB8zEZE8xIlidD1ijjgqLBfeM7n9ylawA
 xHLxwU96sdpdHFzb7Z0yKY2e/bzDaHiG0fUvcCmkgLf+uwKKZid1j8zR5PzKpgPqfy/PF01e
 KyGV3MNu8Y90xMoiEMWfCI2IB1m+hTuzZoboFvGV54SiMuvfWK/VMQjhsL6K2ddOqwVuy2nI
 MI4G3xDQW/v8KVyn43OSIAyW1eaklhzu0Ir2sO60PXRkvbTUrouvmSvpJfIQS49rU0M/X6FS
 DgXQLKrZ3my94+g8ptz9KoVml6s4OAwYVz+sb49nuSxipFKkU5FwhKOLmzbsBxCtytcUJoLm
 juJPJPDQue6YJiIXyc86GVY2pH3DjemKdbB4dSgqAJIp+lCzKSJzz7bgueh2Ox8vzx1tSxKj
 7V8Nal+UTKKbkxPmMh+e20YZ4esAVifO3bS6IJP/aDnfagghB71vA7+aWGXPbjPlc2UHpCBi
 RSsl+IgoQXvdvZBsKRyfBx8neODa2C6JIE5vcaCjilSeKF8SzsFXvimnndhQNhAPU/DwQwSX
 dCl4gTsFVi5d8Oxq1sce+wARAQABiQI8BBgBCAAmFiEEH7l9m2RN073jRiviDQIrUxG+ECAF
 AlrUMZ4CGwwFCQlmAYAACgkQDQIrUxG+ECAWnRAAsmZX+KgNxW3v7R/76Tz4Wjmh4AGeE+Ji
 3p5QsdTYny1B6vYBL9vCzPJ/AK8pgKMDRaweUP5eZQpfrdWC8Q7SNGgi4Q+97KEs+i2xZLQ+
 WJb8a+WEEIc716u0y4ITiHfOgM5jWcFO4MXQATbJgv0drLLesa+LQCvZgPBqupt307EsCubQ
 s+Sxt+RVjf6rOUolp1GJXEQYwGsKklVd6yqLC8M1BSG53/WE5tSv5GzBZ8fp6EtmjT7leuid
 FtEvKYHQz4DqG9ELpHUF0X0UUCBK/MgXe3kCVLKE060UrJ4M6uPSx57rmVFA2MvwQR8M7GsW
 C5UsSM4PYwPWBhwxE7vcx0691YKAHT/5q8LxRVBdUyzPSprMhSQFttsBt+ygm6wRi3Pi3TuC
 EARNubPkQefyeC34yr40SAUCkOl3eWxSXPF4NfXFQb4AAzZSE5hv3qbDuwo3lrL0LqpIpEQP
 Az+JZ1QZ6mMFQ5/JD9Gukj54kZc0X8w3sQt0a8vyE/qrJg8vKgv2rCHrPc5MeDkEUEFiiJiC
 EDdkJtMyoRlU3S4NrnbyLOLEcHE8fGe3hStPX8hY62id2ecdQ5WZ7vLZW5SFeLarbUciuHIk
 VL6MHnUjbV7XlY50N7ebeFCIdlCWhdum2FJs/Ni+SSxbZC564vrokwlBBGSo6WTPQTa8IWx1 DtU=
Message-ID: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com>
Date: Mon, 9 Sep 2019 21:25:20 -0600
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101
 Thunderbird/68.1.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Rspamd-Queue-Id: 46S9QV4Yp1z3K9J
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=none (mx1.freebsd.org: domain of rebecca@bsdio.com has no SPF policy when
 checking 65.103.231.193) smtp.mailfrom=rebecca@bsdio.com
X-Spamd-Result: default: False [-2.95 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[bsdio.com]; AUTH_NA(1.00)[];
 RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4];
 RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_NA(0.00)[];
 FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:209, ipnet:65.103.224.0/19, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 IP_SCORE(-1.85)[ip: (-9.14), asn: 209(-0.05), country: US(-0.05)]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 03:25:23 -0000

Is building world with gcc still supported? I'm getting an error running
the following:

make XCC=/usr/local/bin/gcc9 XCXX=/usr/local/bin/g++9 buildworld

/usr/local/bin/gcc9 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin  -O2 -pipe   -DNO__SCCSID
-DNO__RCSID -I/usr/src/lib/libc/include -I/usr/src/include
-I/usr/src/lib/libc/amd64 -DNLS  -D__DBINTERFACE_PRIVATE
-I/usr/src/contrib/gdtoa -I/usr/src/contrib/libc-vis -DINET6
-I/usr/obj/usr/src/amd64.amd64/lib/libc -I/usr/src/lib/libc/resolv
-D_ACL_PRIVATE -DPOSIX_MISTAKE -I/usr/src/lib/libmd
-I/usr/src/contrib/jemalloc/include -DMALLOC_PRODUCTION
-I/usr/src/contrib/tzcode/stdtime -I/usr/src/lib/libc/stdtime
-I/usr/src/lib/libc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN
-I/usr/src/lib/libc/rpc -DWANT_HYPERV -DYP -DNS_CACHING
-DSYMBOL_VERSIONING -g -MD  -MF.depend.jemalloc_malloc_io.o
-MTjemalloc_malloc_io.o -std=gnu99 -Wno-format-zero-length
-fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k
-Wno-uninitialized -Wno-pointer-sign -Wno-error=address
-Wno-error=array-bounds -Wno-error=attributes -Wno-error=bool-compare
-Wno-error=cast-align -Wno-error=clobbered -Wno-error=enum-compare
-Wno-error=extra -Wno-error=inline -Wno-error=logical-not-parentheses
-Wno-error=strict-aliasing -Wno-error=uninitialized
-Wno-error=unused-but-set-variable -Wno-error=unused-function
-Wno-error=unused-value -Wno-error=misleading-indentation
-Wno-error=nonnull-compare -Wno-error=shift-negative-value
-Wno-error=tautological-compare -Wno-error=unused-const-variable
-Wno-error=bool-operation -Wno-error=deprecated
-Wno-error=expansion-to-defined -Wno-error=format-overflow
-Wno-error=format-truncation -Wno-error=implicit-fallthrough
-Wno-error=int-in-bool-context -Wno-error=memset-elt-size
-Wno-error=noexcept-type -Wno-error=nonnull -Wno-error=pointer-compare
-Wno-error=stringop-overflow -Wno-error=aggressive-loop-optimizations
-Wno-error=cast-function-type -Wno-error=catch-value
-Wno-error=multistatement-macros -Wno-error=restrict
-Wno-error=sizeof-pointer-memaccess -Wno-error=stringop-truncation      
-I/usr/src/lib/libutil -I/usr/src/lib/msun/amd64 -I/usr/src/lib/msun/x86
-I/usr/src/lib/msun/src -c jemalloc_malloc_io.c -o jemalloc_malloc_io.o
jemalloc_malloc_io.c: In function '__je_malloc_vsnprintf':
jemalloc_malloc_io.c:383:2: error: case label value exceeds maximum
value for type [-Werror]
  383 |  case '?' | 0x80:      \
      |  ^~~~
jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'
  595 |     GET_ARG_NUMERIC(val, 'p');
      |     ^~~~~~~~~~~~~~~
jemalloc_malloc_io.c:401:2: error: case label value exceeds maximum
value for type [-Werror]
  401 |  case 'j' | 0x80:      \
      |  ^~~~
jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'
  595 |     GET_ARG_NUMERIC(val, 'p');
      |     ^~~~~~~~~~~~~~~
jemalloc_malloc_io.c:389:2: error: case label value exceeds maximum
value for type [-Werror]
  389 |  case 'l' | 0x80:      \
      |  ^~~~
jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'
  595 |     GET_ARG_NUMERIC(val, 'p');
      |     ^~~~~~~~~~~~~~~
jemalloc_malloc_io.c:395:2: error: case label value exceeds maximum
value for type [-Werror]
  395 |  case 'q' | 0x80:      \
      |  ^~~~
jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'
  595 |     GET_ARG_NUMERIC(val, 'p');
      |     ^~~~~~~~~~~~~~~
jemalloc_malloc_io.c:410:2: error: case label value exceeds maximum
value for type [-Werror]
  410 |  case 'z' | 0x80:      \
      |  ^~~~
jemalloc_malloc_io.c:595:5: note: in expansion of macro 'GET_ARG_NUMERIC'
  595 |     GET_ARG_NUMERIC(val, 'p');
      |     ^~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
*** Error code 1


-- 
Rebecca Cran


From owner-freebsd-current@freebsd.org  Tue Sep 10 05:05:03 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id AE037F0EA3
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 05:05:03 +0000 (UTC)
 (envelope-from cse.cem@gmail.com)
Received: from mail-io1-f45.google.com (mail-io1-f45.google.com
 [209.85.166.45])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SCdV3rRwz3Pgl
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 05:05:02 +0000 (UTC)
 (envelope-from cse.cem@gmail.com)
Received: by mail-io1-f45.google.com with SMTP id r4so34583326iop.4
 for <freebsd-current@freebsd.org>; Mon, 09 Sep 2019 22:05:02 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:reply-to
 :from:date:message-id:subject:to:cc:content-transfer-encoding;
 bh=Lx2Wsm+tG+mRRAukRGgExonOq0DXQJ8qRJiwBuSAQoQ=;
 b=BF7rS1mAfRaqffs+4Q4lgIJaLGnzI/+y0/AaSXask6cqWojg07rBM9w7XalvDDyijn
 14YG1qs11G3ziDP97d7Tmh3SvedvZ7jFxnUiFyDytK0gLIItuuOlpztkHYpCsZWMmH0b
 RPTjE+DtfRMbATPNzp/2OIBsT+/+68vjW2m/ZRJDRTmZVxjKo+mwau/rFFXdxYVSqqHg
 Z0zLI/IOmc9uu2b8qfpK8VVsZFiUvhuH9GvaFhiiF69hpVVE15BMsMi1e59STGOuEbv9
 pZNwCB7FWkOIXPk8LgLXRsL0qUS8tzxZ16njFHYZM3McRTisOS6sd5VexsWO6SHYEDa+
 1ZAw==
X-Gm-Message-State: APjAAAU92bBgITJ6zA8WwF3DZYQqS53Ku9YFc9OjKUwaXVwwol8EiBfT
 dbTVxJFpwgMf5ZLnRrhLnxfp05I/
X-Google-Smtp-Source: APXvYqztkh/yWf4sK1YW3KvofK2VqOJaZupq7xLF82lDIKeXR3lecMTZKmM3rbr7y07RXMfIJuPYVA==
X-Received: by 2002:a6b:dc0e:: with SMTP id s14mr4650766ioc.184.1568091900778; 
 Mon, 09 Sep 2019 22:05:00 -0700 (PDT)
Received: from mail-io1-f41.google.com (mail-io1-f41.google.com.
 [209.85.166.41])
 by smtp.gmail.com with ESMTPSA id g66sm6300219ioa.53.2019.09.09.22.05.00
 for <freebsd-current@freebsd.org>
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Mon, 09 Sep 2019 22:05:00 -0700 (PDT)
Received: by mail-io1-f41.google.com with SMTP id k13so19353216ioj.1
 for <freebsd-current@freebsd.org>; Mon, 09 Sep 2019 22:05:00 -0700 (PDT)
X-Received: by 2002:a5d:9856:: with SMTP id p22mr7342621ios.231.1568091899742; 
 Mon, 09 Sep 2019 22:04:59 -0700 (PDT)
MIME-Version: 1.0
References: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com>
In-Reply-To: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com>
Reply-To: cem@freebsd.org
From: Conrad Meyer <cem@freebsd.org>
Date: Mon, 9 Sep 2019 22:04:48 -0700
X-Gmail-Original-Message-ID: <CAG6CVpU2-Ekz0GiLRF1Kf=SWxUDrQEa-c6rYaJgjRqX9z5FOsg@mail.gmail.com>
Message-ID: <CAG6CVpU2-Ekz0GiLRF1Kf=SWxUDrQEa-c6rYaJgjRqX9z5FOsg@mail.gmail.com>
Subject: Re: Building world with gcc9 not working
To: Rebecca Cran <rebecca@bsdio.com>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Rspamd-Queue-Id: 46SCdV3rRwz3Pgl
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of csecem@gmail.com designates 209.85.166.45
 as permitted sender) smtp.mailfrom=csecem@gmail.com
X-Spamd-Result: default: False [-4.23 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
 HAS_REPLYTO(0.00)[cem@freebsd.org];
 R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17];
 REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[4];
 TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2];
 FORGED_SENDER(0.30)[cem@freebsd.org,csecem@gmail.com];
 MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
 TAGGED_FROM(0.00)[]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 FROM_NEQ_ENVFROM(0.00)[cem@freebsd.org,csecem@gmail.com];
 FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[45.166.85.209.list.dnswl.org : 127.0.5.0];
 IP_SCORE(-2.23)[ip: (-5.50), ipnet: 209.85.128.0/17(-3.32), asn: 15169(-2.26),
 country: US(-0.05)]; 
 RWL_MAILSPIKE_POSSIBLE(0.00)[45.166.85.209.rep.mailspike.net : 127.0.0.17];
 RCVD_TLS_ALL(0.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 05:05:03 -0000

On Mon, Sep 9, 2019 at 8:25 PM Rebecca Cran <rebecca@bsdio.com> wrote:
>
> Is building world with gcc still supported? I'm getting an error running
> the following:
>
> make XCC=3D/usr/local/bin/gcc9 XCXX=3D/usr/local/bin/g++9 buildworld

Hi Rebecca,

Aspirationally, yes.  I did a successful CROSS_TOOLCHAIN=3Damd64-gcc
buildworld earlier this week.  (It doesn't play well with binary pkg's
built with Clang, so I ended up replacing it with a Clang-built world
instead, but it compiled.)

Unfortunately, amd64-xtoolchain-gcc is stuck on GCC 6.4.0, while
you're running the much more recent GCC9.  I think GCC6.4.0 is more or
less expected to build world today, but I don't think many people are
building with GCC9.  I would love for amd64-xtoolchain to move to a
newer version, but I don't know what is blocking that =E2=80=94 it seems li=
ke
it should be straightforward.

Best regards,
Conrad

From owner-freebsd-current@freebsd.org  Tue Sep 10 05:08:56 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id CC260F1033
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 05:08:56 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com
 [IPv6:2607:f8b0:4864:20::836])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SCjz57DRz3Pr9
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 05:08:55 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: by mail-qt1-x836.google.com with SMTP id g4so19256002qtq.7
 for <freebsd-current@freebsd.org>; Mon, 09 Sep 2019 22:08:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=bsdimp-com.20150623.gappssmtp.com; s=20150623;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=2O2WJjPCf9pHLKztQElG+QooeNNCmWs13v7CLFPbCPk=;
 b=DfS3UfG+uaA2YZfAPcoLXM5tk0xR2B/grkcSbxxPaeRbLgkG6WCMs/sVdq35vw69lA
 ia/4x5vBDdlx5rUInI1VQ4WEiydgMNfNX0ws+1lgm2lb/Mt8QxtQYcpf6nLrcFx9YO1E
 L4wWVXls1bwSIWzFOVvBcSDzEAaZYfUgot6nJhSbTxnmUyW7hZ07mm0EznzLgcrcdLko
 Joe4D6gvTOWY7DIK1ywQBDiJv9xTclrCxzEnZNgQLHuAeEltCZX5bIkPWkADfgV/uVEM
 8SdENefGHXNQozuMzNougL7SjLChBJANXdrjkKnBLVD/6Eu5fLT6/AjjnzKktas0F6zv
 BdkQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=2O2WJjPCf9pHLKztQElG+QooeNNCmWs13v7CLFPbCPk=;
 b=IbjYoMeHLmI49+MvMWLY0iu+jVbvrYTstYYiwObZS0s5qE9t1XNXQ12ksJBliPR3Q5
 +CRmT66VK0stAiV0cINZmz88M4s5g37unAn0F37EaZ4BnC5CqnCmMSDg1umIo8DJVK1K
 uj5TESx0SYYo5wL9gCpoHGVRPrlVRR25Ryb8OI9bHSOrvBHbRSD1wqZhRpgDWCMRGZDW
 ch9+aic+juBINCJFaDkLl1Y5t6T1g6Z+AvLRgBrCa5hUjDAody1eK71L5JErKXW/lytz
 LH8FtfRjE59tfDJ77PdfZ+DrrnRoyOeIQTKwDpver//SiDRr707DkrhNIts+djK9LbKl
 E1Jg==
X-Gm-Message-State: APjAAAXypQY6+E2wu+vlXfGMulRIXfkOTtfb9g+3YiuoULgEp72nk62Q
 NHHOCQphFqHZeXbIfRN8It+j89vLNs6G3v0aVTXt9J4o
X-Google-Smtp-Source: APXvYqy4mWjj7+0UwwLTp0cVcSBSi8QQEazT40RARJgPhgRgTxPOndAZ+3sM5XhP8/oGjFYAQXu8EMU3OvSJ16avFDY=
X-Received: by 2002:ac8:71cb:: with SMTP id i11mr25868833qtp.32.1568092134544; 
 Mon, 09 Sep 2019 22:08:54 -0700 (PDT)
MIME-Version: 1.0
References: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com>
 <CAG6CVpU2-Ekz0GiLRF1Kf=SWxUDrQEa-c6rYaJgjRqX9z5FOsg@mail.gmail.com>
In-Reply-To: <CAG6CVpU2-Ekz0GiLRF1Kf=SWxUDrQEa-c6rYaJgjRqX9z5FOsg@mail.gmail.com>
From: Warner Losh <imp@bsdimp.com>
Date: Mon, 9 Sep 2019 23:08:43 -0600
Message-ID: <CANCZdfr0mP+b_=fEizvAV8+zUAQ2uCmroEGXe_K-2g6Hx5hTOA@mail.gmail.com>
Subject: Re: Building world with gcc9 not working
To: "Conrad E. Meyer" <cem@freebsd.org>
Cc: Rebecca Cran <rebecca@bsdio.com>,
 FreeBSD Current <freebsd-current@freebsd.org>
X-Rspamd-Queue-Id: 46SCjz57DRz3Pr9
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623
 header.b=DfS3UfG+; dmarc=none;
 spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when
 checking 2607:f8b0:4864:20::836) smtp.mailfrom=wlosh@bsdimp.com
X-Spamd-Result: default: False [-4.86 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623];
 FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3];
 NEURAL_HAM_LONG(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+];
 RCVD_IN_DNSWL_NONE(0.00)[6.3.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; R_SPF_NA(0.00)[];
 FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com];
 MIME_TRACE(0.00)[0:+,1:+,2:~];
 IP_SCORE(-2.86)[ip: (-9.26), ipnet: 2607:f8b0::/32(-2.74), asn: 15169(-2.26),
 country: US(-0.05)]; 
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com];
 RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 05:08:56 -0000

On Mon, Sep 9, 2019 at 11:05 PM Conrad Meyer <cem@freebsd.org> wrote:

> On Mon, Sep 9, 2019 at 8:25 PM Rebecca Cran <rebecca@bsdio.com> wrote:
> >
> > Is building world with gcc still supported? I'm getting an error runnin=
g
> > the following:
> >
> > make XCC=3D/usr/local/bin/gcc9 XCXX=3D/usr/local/bin/g++9 buildworld
>
> Hi Rebecca,
>
> Aspirationally, yes.  I did a successful CROSS_TOOLCHAIN=3Damd64-gcc
> buildworld earlier this week.  (It doesn't play well with binary pkg's
> built with Clang, so I ended up replacing it with a Clang-built world
> instead, but it compiled.)
>
> Unfortunately, amd64-xtoolchain-gcc is stuck on GCC 6.4.0, while
> you're running the much more recent GCC9.  I think GCC6.4.0 is more or
> less expected to build world today, but I don't think many people are
> building with GCC9.  I would love for amd64-xtoolchain to move to a
> newer version, but I don't know what is blocking that =E2=80=94 it seems =
like
> it should be straightforward.
>

I did a gcc8 build about a year or so ago, though I had to turn off Werror
to complete the build...

Warner

From owner-freebsd-current@freebsd.org  Tue Sep 10 05:44:18 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id A06F9F1DC4
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 05:44:18 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SDVn2r2Gz3wmq
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 05:44:17 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from tom.home (kib@localhost [127.0.0.1])
 by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x8A5i8hB035919
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Tue, 10 Sep 2019 08:44:11 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x8A5i8hB035919
Received: (from kostik@localhost)
 by tom.home (8.15.2/8.15.2/Submit) id x8A5i7WA035918;
 Tue, 10 Sep 2019 08:44:07 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
 using -f
Date: Tue, 10 Sep 2019 08:44:07 +0300
From: Konstantin Belousov <kostikbel@gmail.com>
To: Neel Chauhan <neel@neelc.org>
Cc: freebsd-current@freebsd.org
Subject: Re: TSC-low clock slow on WhiskeyLake i7-8565U (HP Spectre x360
 13-ap0043dx)
Message-ID: <20190910054407.GW2559@kib.kiev.ua>
References: <72978644df330013de27c75e6285ab4d@neelc.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <72978644df330013de27c75e6285ab4d@neelc.org>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM,
 NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home
X-Rspamd-Queue-Id: 46SDVn2r2Gz3wmq
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org; dkim=none;
 dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com
 (policy=none); 
 spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor
 denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com
X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[];
 FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[];
 FREEMAIL_FROM(0.00)[gmail.com];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c];
 IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCPT_COUNT_TWO(0.00)[2];
 IP_SCORE(0.00)[ip: (-2.58), ipnet: 2001:470::/32(-4.46), asn: 6939(-3.17),
 country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[];
 R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US];
 RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com];
 DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 05:44:18 -0000

On Mon, Sep 09, 2019 at 04:07:39PM -0400, Neel Chauhan wrote:
> Hi,
> 
> I recently got a HP Spectre x360 13-ap0043dx as a gift and by default 
> the clock runs slower on FreeBSD than Windows or Linux. Yes, I know the 
> ThinkPad is the best laptop for FreeBSD, but getting an X1 Carbon would 
> increase the price of the gift even more which couldn't be done.
> 
> The kern.timecounter.hardware by default is set to TSC-low and the clock 
> is slow on the Spectre x360. Setting it to ACPI-slow resolves this 
> issue.
> 
> The CPU is a Intel WhiskeyLake Core i7-8565U.
> 
> Is the slow TSC-low an issue with WhiskeyLake in general or specifically 
> HP? Is it something else?
> 
> I am considering writing a patch but before I write one, do other people 
> with WhiskeyLake laptops (or newer) have slow clocks (where one second 
> on the system is actually more in real life), or not.

Start with providing full listing dmesg for verbose boot.

Out of blue, try to set
machdep.disable_tsc_calibration=1
in loader.conf and see if this improves things.

From owner-freebsd-current@freebsd.org  Tue Sep 10 06:10:05 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6F17F2563
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 06:10:05 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SF4Y0rPJz3xvv
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 06:10:05 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from tom.home (kib@localhost [127.0.0.1])
 by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x8A69oiY041839
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Tue, 10 Sep 2019 09:09:53 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x8A69oiY041839
Received: (from kostik@localhost)
 by tom.home (8.15.2/8.15.2/Submit) id x8A69nS2041838;
 Tue, 10 Sep 2019 09:09:49 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
 using -f
Date: Tue, 10 Sep 2019 09:09:49 +0300
From: Konstantin Belousov <kostikbel@gmail.com>
To: Warner Losh <imp@bsdimp.com>
Cc: Ian Lepore <ian@freebsd.org>,
 "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>,
 Cy Schubert <Cy.Schubert@cschubert.com>, Harlan Stenn <stenn@nwtime.org>,
 Vladimir Zakharov <zakharov.vv@gmail.com>,
 FreeBSD Current <freebsd-current@freebsd.org>
Subject: Re: ntpd segfaults on start
Message-ID: <20190910060949.GY2559@kib.kiev.ua>
References: <201909091630.x89GUjGX044288@gndrsh.dnsmgr.net>
 <ef8b67577e6512313261693cbbf40e24f007b6c4.camel@freebsd.org>
 <20190909184446.GU2559@kib.kiev.ua>
 <c12493eb2f1bcd98303a33c0e1b0e8756d13b245.camel@freebsd.org>
 <CANCZdfptNY_sYcKAXZYmBErRgFtZD_gh_ve=4YxPL5xHEL5StQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CANCZdfptNY_sYcKAXZYmBErRgFtZD_gh_ve=4YxPL5xHEL5StQ@mail.gmail.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM,
 NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home
X-Rspamd-Queue-Id: 46SF4Y0rPJz3xvv
X-Spamd-Bar: /
Authentication-Results: mx1.freebsd.org; dkim=none;
 dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com
 (policy=none); 
 spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor
 denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com
X-Spamd-Result: default: False [-0.50 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none];
 RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[];
 FREEMAIL_FROM(0.00)[gmail.com];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 TAGGED_RCPT(0.00)[]; HAS_XAW(0.00)[];
 R_SPF_SOFTFAIL(0.00)[~all:c]; IP_SCORE_FREEMAIL(0.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 RCPT_COUNT_SEVEN(0.00)[7];
 IP_SCORE(0.00)[ip: (-2.54), ipnet: 2001:470::/32(-4.46), asn: 6939(-3.16),
 country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[];
 R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US];
 RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[gmail.com];
 SUSPICIOUS_RECIPS(1.50)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 06:10:06 -0000

On Mon, Sep 09, 2019 at 03:42:43PM -0600, Warner Losh wrote:
> On Mon, Sep 9, 2019 at 3:12 PM Ian Lepore <ian@freebsd.org> wrote:
> 
> > On Mon, 2019-09-09 at 21:44 +0300, Konstantin Belousov wrote:
> > > On Mon, Sep 09, 2019 at 12:13:24PM -0600, Ian Lepore wrote:
> > > > On Mon, 2019-09-09 at 09:30 -0700, Rodney W. Grimes wrote:
> > > > > > On Sat, 2019-09-07 at 09:28 -0700, Cy Schubert wrote:
> > > > > > > In message <20190907161749.GJ2559@kib.kiev.ua>, Konstantin
> > > > > > > Belousov writes:
> > > > > > > > On Sat, Sep 07, 2019 at 08:45:21AM -0700, Cy Schubert
> > > > > > > > wrote:
> > > > > > > > > [...]
> > > > >
> > > > > Doesn't locking this memory down also protect ntpd from OOM kills?
> > > > > If so that is a MUST preserve functionality, as IMHO killing ntpd
> > > > > on a box that has it configured is a total no win situation.
> > > > >
> > > >
> > > > Does it have that effect?  I don't know.  But I would argue that that's
> > > > a separate issue, and we should make that happen by adding
> > > > ntpd_oomprotect=YES to /etc/defaults/rc.conf
> > >
> > > Wiring process memory has no effect on OOM selection. More, because
> > > all potentially allocated pages are allocated for real after mlockall(),
> > > the size of the vmspace, as accounted by OOM, is the largest possible
> > > size from the whole lifetime.
> > >
> > > On the other hand, the code execution times are not predictable if the
> > > process's pages can be paged out. Under severe load next instruction
> > > might take several seconds or even minutes to start. It is quite unlike
> > > the scheduler delays. That introduces a jitter in the local time
> > > measurements and their usage as done in userspace. Wouldn't this affect
> > > the accuracy ?
> > >
> >
> > IMO, there is a large gap between "in theory, paging could cause
> > indeterminate delays in code execution" and "time will be inaccurate on
> > your system".  If there were a delay in a part of the code where it
> > matters that amounted to "seconds or even minutes", what you'd end up
> > with is a measurement that would be discarded by the median filter as
> > an outlier.  There would be some danger that if that kind of delay
> > happened for too many polling cycles in a row, you'd end up with no
> > usable measurements after a while and clock accuracy would suffer.
> > Sub-second delays would be more worriesome because they might not be
> > rejected as outliers.
> >
> > There are only a couple code paths in freebsd ntpd processing where a
> > paging (or scheduling) delay could cause measurement inaccuracy:
> >
> >  - When stepping the clock, the code that runs between calling
> > clock_gettime() and calling clock_settime() to apply the step
> > adjustment to the clock.
> >
> >  - When beginning an exchange with or replying to a peer, the code that
> > runs between obtaining system time for the outgoing Transmit Timestamp
> > and actually transmitting that packet.
> >
> > Stepping the clock typically only happens once at startup.  The ntpd
> > code itself recognizes that this is a time-critical path (it has
> > comments to that effect) but unfortunately the code that runs is
> > scattered among several different .c files so it's hard to say what the
> > likelyhood is that code in the critical section will all be in the same
> > page (or be already-resident because other startup-time code faulted in
> > those pages).  IMO, the right fix for this would be a kernel interface
> > that let you apply a step-delta to the clock with a single syscall
> > (perhaps as an extension to the existing ntp_adjtime() using a new mode
> > flag).
> >
> > On freebsd, the Receive timestamps are captured in the kernel and
> > delivered along with the packet to userland, and are retrieved by the
> > ntpd code from the SCM_BINTIME control message in the packet, so there
> > is no latency problem in the receive path.  There isn't a corresponding
> > kernel mechanism for setting the outgoing timestamps, so whether it's
> > originating a request to a peer or replying to a request from a peer,
> > the transmit timestamp could be wrong due to:
> >
> >  - paging delays
> >  - scheduler delays
> >  - network stack, outgoing queues, and driver delays
> >
> > So the primary vulnerability is on the transmit path between obtaining
> > system time and the packet leaving the system.  A quick glance at that
> > code makes me think that most of the data being touched has already
> > been referenced pretty recently during the process of assembling the
> > outgoing packet, so it's unlikely that storing the timestamp into the
> > outgoing packet or the other bit of work that happens after that
> > triggers a pagein unless the system is pathologically overloaded.
> > Naturally, obtaining the timestamp and putting it into the packet is
> > one of the last things it does before sending, so the code path is
> > relatively short, but it's not clear to me whether it's likely or not
> > that the code involved all lives in the same page.  Still, it's one of
> > the heavily exercised paths within ntpd, which should increase the odds
> > of the pages being resident because of recent use.
> >
> > So, I'm not disputing the point that a sufficiently overloaded system
> > can lead to an indeterminate delay between *any* two instructions
> > executed in userland.  What I've said above is more along the lines of
> > considering the usual situation, not the most pathlogical one.  In the
> > most pathological cases, either the delays introduced are fairly minor
> > and you get some minor jitter in system time (ameliorated by the median
> > filtering built in to ntpd), or the delays are major (a full second or
> > more) and get rejected as outliers, not affecting system time at all
> > unless the situation persists and prevents getting any good
> > measurements for many hours.
> >
> 
> I've read through all this and agree with it as well. Paging delays can
> happen, but if they do who cares: the measurements will be rejected as
> outliers for long delays, but might introduce some noise if the delay is on
> the order of tens of milliseconds. Shorter may won't matter long term: they
> will average out. Longer will definitely be rejected. And it will likely be
> just for the first packet in the exchange since the code path will be
> paged/swapped into the working set for that.  The loop is a combination of
> PLL/FLL so that if we think there's a big phase step, we'll also think
> there's a frequency error. Both are steered out the same way: by setting
> the offset. But ntpd is wise enough to know that there will be noisy
> measurements, so even if the measurements make it through the filters, we
> only remove a portion of the error each polling interval anyway. Any over
> or undershoot will be correct the next measurement interval. And if you
> have so much memory pressure that ntpd is paged out every single
> measurement interval, your system likely needs careful tuning anyway.
> 
> In days of yore (like the mid 90s), these defaults were setup when it took
> tens or even hundreds of milliseconds to page in it mattered. Today those
> numbers are submillisecond to single digit milliseconds, so in the typical
> case the disruption is much less severe than it was when things were
> initially locked into memory. I'm guessing that's why Linux has moved to -1
> for default. In addition, ntpd's algorithms have improved somewhat since
> then as well to cope with noise. These days, good ntpd performance is
> submillisecond over the internet, so the noise has to be approximately on
> that order to affect the filters in ntpd. On the LAN you're good to 10's of
> microseconds, typically, so any noise > several 10's of microseconds would
> be eliminated as an outlier anyway.  I strongly suspect careful
> measurements will declare no difference in performance except, maybe, on
> the most overloaded of servers (and then it would need to be extremely
> overloaded to have a delay in scheduling often enough to matter).
> 
> For people that want to be sure, by all means lock it into memory. But as a
> default, I'm extremely skeptical one could measure a difference, at all,
> let alone measure a difference that matters.

>From the overall system performance, I am all for stopping wiring ntpd.
One of the unfortunate consequences of doing that is the wiring of rtld,
libc, and libthr.

Small note, besides large delays caused by real pageouts (hard faults),
there are small jitter-like delays when pagedaemon emulates access and
dirty bits on machines which lack them by unmapping still resident
pages. The cost of reinstalling the pages is just the cost of locking
and calculating PTEs, but locking makes it dependent on other system
activities. I think that ARM is the largest suspect there.

From owner-freebsd-current@freebsd.org  Tue Sep 10 12:37:59 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2111DD5923
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 12:37:59 +0000 (UTC)
 (envelope-from yuripv@yuripv.net)
Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com
 [66.111.4.230])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SPh61wjlz4PBP
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 12:37:57 +0000 (UTC)
 (envelope-from yuripv@yuripv.net)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailnew.nyi.internal (Postfix) with ESMTP id B40612A7D
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 08:37:55 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute5.internal (MEProxy); Tue, 10 Sep 2019 08:37:55 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h=to
 :from:subject:message-id:date:mime-version:content-type
 :content-transfer-encoding; s=fm1; bh=+v53l4HjxJOtbWb2PL/isgF4la
 fR8IG/t73vecOIgsE=; b=IqLWHbVaZLa2oufoMj5VmDod4iKJOCgcM/9mpHCdjp
 mp98UOZ00ngfDVHCuru9WsUGfeuJzmkbR1u7hpnlTpnnAxkCLzGoWTQAFCk2wAEH
 rkUm4HV4kVuOc2Hpg+VmOPzxPxzninsvIeH/bKp2s4730BcWDStGLYm+y9szRW6c
 cm4OZJampYnuKyNLn6sU3IZwjgqFltlHfCkhcMHhhFzJ3ZxDm44iLJhgjs0RNia2
 1G8jU2CHnSJdcRQPjbaCDbV0mQiMFLoMxNIZoetUlDV01DccgI9XylcdtYjs0aI4
 xv0ZgKlLyaUyZ/NJqqRFS0p/+lNyzcs3wiEMbQQ8lpow==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=content-transfer-encoding:content-type
 :date:from:message-id:mime-version:subject:to:x-me-proxy
 :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=+v53l4
 HjxJOtbWb2PL/isgF4lafR8IG/t73vecOIgsE=; b=kDkdc63KXVihXkn7fgp+BQ
 eXN+VkH7GYyNcLxNvJTbHrGS2KNLR/97Rn919+WJQv3vjI7pFu+qG0Mn4lkf+UvJ
 RKjkHUG/TkJ0Aw2/GZacuBGYQrx39c8ym9no8RmpoXXwzl/UQbbmylllcyWmJ9kp
 OBt88x6vRICPg3/58oa35dBk9OhAgKj4N5bfK9dejv/SC4J7hkTwBwGNz1iDKGSq
 5fUpydnTgq4Gis1fSEDCy7SW7jVs7RAzM/IeFi1y199g5gU9MS0jL5Z6CyhyFTkv
 /u+WRM2R/br6mc0qE87gR5pjjgLNQ6x/4l9Q+M5DVg+jMEMcu/mGNUStO6zI5+2g
 ==
X-ME-Sender: <xms:I5l3XQl6MHaCJpBxf35tY7mqpGuq7F2leqYS_v-Mtw_SZPpbcG_8eg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrtddtgdefudcutefuodetggdotefrodftvf
 curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
 uegrihhlohhuthemuceftddtnecunecujfgurhepvffhuffkffgfgggtgfesthejredttd
 efjeenucfhrhhomhepjghurhhiucfrrghnkhhovhcuoeihuhhrihhpvheshihurhhiphhv
 rdhnvghtqeenucffohhmrghinhepfhhrvggvsghsugdrohhrghdpphgrshhtvggsohgrrh
 gurdgtohenucfkphepuddtledrjedtrddugeegrdduhedtnecurfgrrhgrmhepmhgrihhl
 fhhrohhmpeihuhhrihhpvheshihurhhiphhvrdhnvghtnecuvehluhhsthgvrhfuihiivg
 eptd
X-ME-Proxy: <xmx:I5l3XfZKssyCr_nvfn1G4jcF0mVzdfgfsxD9jNFErAlZsyrQOhSzBQ>
 <xmx:I5l3XS-ChqgS6T5320Xj_negRetc1FQxtoieqSnWhKp20V2AX9GT_g>
 <xmx:I5l3XfEncKCVrWomfrl2aR0rW_xHMQ_jx6-3-G8xgJilw5ZOTGbl7w>
 <xmx:I5l3XdFrqetls3QBXJMhzD87C6tuq93hT7cZHJiJAF4anWeNRYPfBw>
Received: from [192.168.1.11] (unknown [109.70.144.150])
 by mail.messagingengine.com (Postfix) with ESMTPA id 03277D6005D
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 08:37:54 -0400 (EDT)
To: freebsd-current <freebsd-current@freebsd.org>
From: Yuri Pankov <yuripv@yuripv.net>
Subject: panic: rcv_start < rcv_end
Message-ID: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net>
Date: Tue, 10 Sep 2019 15:37:54 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101
 Thunderbird/68.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46SPh61wjlz4PBP
X-Spamd-Bar: ------
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=yuripv.net header.s=fm1 header.b=IqLWHbVa;
 dkim=pass header.d=messagingengine.com header.s=fm3 header.b=kDkdc63K;
 dmarc=none;
 spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.230
 as permitted sender) smtp.mailfrom=yuripv@yuripv.net
X-Spamd-Result: default: False [-6.09 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[];
 R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
 R_SPF_ALLOW(-0.20)[+ip4:66.111.4.230];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[yuripv.net]; RCPT_COUNT_ONE(0.00)[1];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4];
 RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+];
 IP_SCORE(-3.49)[ip: (-9.87), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68),
 country: US(-0.05)]; 
 RCVD_IN_DNSWL_LOW(-0.10)[230.4.111.66.list.dnswl.org : 127.0.5.1];
 FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US];
 MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 12:37:59 -0000

Just seen this almost immediately after booting the system installed 
from amd64-20190906-r351901 snapshot, trying to do initial pkg 
bootstrap.  Sadly, I didn't have the swap/dump device configured at the 
time, so no dump was saved.

But it looks like I'm not alone, seeing the 
https://forums.freebsd.org/threads/kernel-panic-on-bhyve-virtualization.72222/ 
topic.  Note that I'm running on bare metal, so bhyve isn't involved. 
My panic screenshot is at https://pasteboard.co/IwLaXXb.jpg.

In (the most likely) case it's not helpful enough, I'm now running with 
dump device configured, and will update if/when the panic reproduces.

From owner-freebsd-current@freebsd.org  Tue Sep 10 14:20:45 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 008D9D7ACE
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 14:20:45 +0000 (UTC)
 (envelope-from tuexen@freebsd.org)
Received: from drew.franken.de (mail-n.franken.de [193.175.24.27])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "*.franken.de",
 Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SRyh6BpKz4V6j
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 14:20:44 +0000 (UTC)
 (envelope-from tuexen@freebsd.org)
Received: from [10.211.20.102] (unknown [194.95.11.51])
 (Authenticated sender: macmic)
 by mail-n.franken.de (Postfix) with ESMTPSA id 67969721E2829;
 Tue, 10 Sep 2019 16:20:42 +0200 (CEST)
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: panic: rcv_start < rcv_end
From: Michael Tuexen <tuexen@freebsd.org>
In-Reply-To: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net>
Date: Tue, 10 Sep 2019 16:20:41 +0200
Cc: freebsd-current <freebsd-current@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <D71ACAB5-8DB1-460A-BEFE-C398F9A12FB3@freebsd.org>
References: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net>
To: Yuri Pankov <yuripv@yuripv.net>
X-Mailer: Apple Mail (2.3445.104.11)
X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00
 autolearn=disabled version=3.4.1
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de
X-Rspamd-Queue-Id: 46SRyh6BpKz4V6j
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-2.00 / 15.00];
 local_wl_from(0.00)[freebsd.org];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0];
 NEURAL_HAM_LONG(-1.00)[-0.999,0];
 ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 14:20:45 -0000

> On 10. Sep 2019, at 14:37, Yuri Pankov <yuripv@yuripv.net> wrote:
>=20
> Just seen this almost immediately after booting the system installed =
from amd64-20190906-r351901 snapshot, trying to do initial pkg =
bootstrap.  Sadly, I didn't have the swap/dump device configured at the =
time, so no dump was saved.
>=20
> But it looks like I'm not alone, seeing the =
https://forums.freebsd.org/threads/kernel-panic-on-bhyve-virtualization.72=
222/ topic.  Note that I'm running on bare metal, so bhyve isn't =
involved. My panic screenshot is at https://pasteboard.co/IwLaXXb.jpg.
>=20
> In (the most likely) case it's not helpful enough, I'm now running =
with dump device configured, and will update if/when the panic =
reproduces.
This panic should be fixed by:
https://svnweb.freebsd.org/changeset/base/352072

Please drop me a note if not.

Best regards
Michael
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to =
"freebsd-current-unsubscribe@freebsd.org"


From owner-freebsd-current@freebsd.org  Tue Sep 10 14:23:17 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 741A6D7E19;
 Tue, 10 Sep 2019 14:23:17 +0000 (UTC) (envelope-from ler@FreeBSD.org)
Received: from thebighonker.lerctr.org (ns-b.lerctr.org
 [IPv6:2001:470:1f0f:3ad::53:2])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "*.lerctr.org", Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SS1d2VNzz4VW1;
 Tue, 10 Sep 2019 14:23:17 +0000 (UTC) (envelope-from ler@FreeBSD.org)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; 
 s=ler2019;
 h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:
 Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To:
 Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
 Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
 List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=3CDicv77rxfbInzeaIxweOfTNoRWHN6eVvNGqUf46UQ=; b=hHjn3Ty2/AmvfoteQ1+aaNic6j
 7gfOCy57TDGGNNbeQRLiEUZPoDkvFha58vnY3Pg6uHDP13bTPfcN/m5c8gRb1ndO2bEPAMwXG2Vp2
 ibSRqs1D0NUManJZD0Vy0aV34ANOhSoUJ7VcMtclKNTnHF21E6qKcvkMYXJR8SIjdWI3DutzAi0SY
 FMYWgALUlL2ki0FYHK1JquE7+OCJ/759N27W/34zTCltnsaO2y8/JQDFItJZqT8nD5HhkzTArROQD
 0Ghd3jBZmjTjgTRXXnykUfjXzna+9WPsECqaIQWtDzqAhlpMZHqxt13u22K3sx7IhAR8p4mF+JFre
 XmSbY9YA==;
Received: from thebighonker.lerctr.org
 ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:27726 helo=webmail.lerctr.org)
 by thebighonker.lerctr.org with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256)
 (Exim 4.92.2 (FreeBSD)) (envelope-from <ler@FreeBSD.org>)
 id 1i7h35-0006RE-Ib; Tue, 10 Sep 2019 09:23:15 -0500
Received: from 2600:1700:210:b180:f434:7f3b:2aaf:25b9 by webmail.lerctr.org
 with HTTP (HTTP/1.1 POST); Tue, 10 Sep 2019 09:23:15 -0500
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Tue, 10 Sep 2019 09:23:15 -0500
From: Larry Rosenman <ler@FreeBSD.org>
To: Michael Tuexen <tuexen@freebsd.org>
Cc: Yuri Pankov <yuripv@yuripv.net>, freebsd-current
 <freebsd-current@freebsd.org>, owner-freebsd-current@freebsd.org
Subject: Re: panic: rcv_start < rcv_end
In-Reply-To: <D71ACAB5-8DB1-460A-BEFE-C398F9A12FB3@freebsd.org>
References: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net>
 <D71ACAB5-8DB1-460A-BEFE-C398F9A12FB3@freebsd.org>
Message-ID: <88808aaf430b3184cd5e6921fc8c10ba@FreeBSD.org>
X-Sender: ler@FreeBSD.org
User-Agent: Roundcube Webmail/1.3.9
X-Rspamd-Queue-Id: 46SS1d2VNzz4VW1
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-6.00 / 15.00];
 NEURAL_HAM_MEDIUM(-1.00)[-0.997,0];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 14:23:17 -0000

On 09/10/2019 9:20 am, Michael Tuexen wrote:
>> On 10. Sep 2019, at 14:37, Yuri Pankov <yuripv@yuripv.net> wrote:
>> 
>> Just seen this almost immediately after booting the system installed 
>> from amd64-20190906-r351901 snapshot, trying to do initial pkg 
>> bootstrap.  Sadly, I didn't have the swap/dump device configured at 
>> the time, so no dump was saved.
>> 
>> But it looks like I'm not alone, seeing the 
>> https://forums.freebsd.org/threads/kernel-panic-on-bhyve-virtualization.72222/ 
>> topic.  Note that I'm running on bare metal, so bhyve isn't involved. 
>> My panic screenshot is at https://pasteboard.co/IwLaXXb.jpg.
>> 
>> In (the most likely) case it's not helpful enough, I'm now running 
>> with dump device configured, and will update if/when the panic 
>> reproduces.
> This panic should be fixed by:
> https://svnweb.freebsd.org/changeset/base/352072
> 
> Please drop me a note if not.
> 
> Best regards
> Michael
>> _______________________________________________
>> freebsd-current@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to 
>> "freebsd-current-unsubscribe@freebsd.org"
> 
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to 
> "freebsd-current-unsubscribe@freebsd.org"


is this the same panic:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240471

I *DO* have a core.


-- 
Larry Rosenman                     http://people.freebsd.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler@FreeBSD.org
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

From owner-freebsd-current@freebsd.org  Tue Sep 10 14:32:45 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5D1B4D8208
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 14:32:45 +0000 (UTC)
 (envelope-from yuripv@yuripv.net)
Received: from new2-smtp.messagingengine.com (new2-smtp.messagingengine.com
 [66.111.4.224])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SSDX3H0yz4W57;
 Tue, 10 Sep 2019 14:32:44 +0000 (UTC)
 (envelope-from yuripv@yuripv.net)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailnew.nyi.internal (Postfix) with ESMTP id 7CB06CC4;
 Tue, 10 Sep 2019 10:32:43 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
 by compute5.internal (MEProxy); Tue, 10 Sep 2019 10:32:43 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h=
 subject:to:cc:references:from:message-id:date:mime-version
 :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=g
 nfYU3spXn0VXLLDwMHdoTGSPcEp2C0bbyUKqZ2i+3U=; b=HfH5/GPVzZqMepGeC
 a6qHl4w+QnXZ5hKF65FVXrpxYaMirDVz6q+h2gaY90gXbFyXhQVZezwJUUk3Wa9I
 h198m7JN7Fj7p9tC35QdiSCpTLFfM7QH9nkIrh7GjFeGSRtSsNDicdSRAYGHFUL+
 iJEfxjWQ1UkbrAO4JQ1LdTjZvyXCgpxYsKxO/UXOAyTxMaj9DqUOx/bP53TXPSnT
 +9kRvuFTL9DHAT9Ju7pX9RTFZ8ZMLgtyuICCRCcIwiZnLx8rAKfC1kvrYyliUM8z
 DUfDacgtUfj9TXxWTTEii+mZpH9RQQ5WiIzY23GKQ9ZzERnL5eWZdcmjcTiZnmrI
 1/WlQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:content-transfer-encoding:content-type
 :date:from:in-reply-to:message-id:mime-version:references
 :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
 :x-sasl-enc; s=fm3; bh=gnfYU3spXn0VXLLDwMHdoTGSPcEp2C0bbyUKqZ2i+
 3U=; b=v/O9wDpiZO+wlW3CDagu8AjxSl3WKTekJstCtX0SY/kAkmMnYux/xmrVG
 fW8NWBubrmkyKlmPoWtthNiEKgvV3TUUKBxV7iFo/EaF8Zj4uhXnKlsWt2Qe3iJq
 YXo5LfLgF+C7kANWlrlBrYMM3D8dO+l17ecLwT6SEHHBgFiqPndLg15gHIpXPa/w
 LqbReIeQ/qPJ7cjib27gBaI6vNB6UnIEImZEuWmV+Emlba+Wim26wm7pbHCdSMUR
 S8fElEs/8W8z6B3zIn4zLgwwe77momfWpGZyH7LIbEViuoPcE5QwCkFzFgVLht/c
 YTYPrIPeBtl9LWE848ZIMbAO6rpTQ==
X-ME-Sender: <xms:C7R3Xbkp6R2izA6vgvhSynUOoeLSokbZy3YUH1O2DTRB6Q0jGHeHkA>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrtddtgdehgecutefuodetggdotefrodftvf
 curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
 uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenog
 fuphgrmhfkphdqohhuthculdehtddtmdenucfjughrpefuvfhfhffkffgfgggjtgfgseht
 jeertddtfeejnecuhfhrohhmpegjuhhrihcurfgrnhhkohhvuceohihurhhiphhvseihuh
 hrihhpvhdrnhgvtheqnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgpdhprghsthgv
 sghorghrugdrtghonecukfhppedvudejrddugeeirdekvddrudeggeenucfrrghrrghmpe
 hmrghilhhfrhhomhephihurhhiphhvseihuhhrihhpvhdrnhgvthenucevlhhushhtvghr
 ufhiiigvpedt
X-ME-Proxy: <xmx:C7R3XXxTSyBeHNGvlgyatz6v4pNFhochc549yRxgEX83bWwc43_nEw>
 <xmx:C7R3XdM7nUGVH2cqAFbkshUy8un2Tc7vBQupyARjcLLUv2C9B-nuow>
 <xmx:C7R3XTpMp0sEx9fIy8P6e_udXOWvLzTyiVMM0CoCIgR7WSXlIgPi_w>
 <xmx:C7R3XY8bTtvzWIdXUh_Uzvlyx11-f-XlNS24VfTiuT6VDkBT7z3CELoKZVA>
Received: from [192.168.1.11] (unknown [217.146.82.144])
 by mail.messagingengine.com (Postfix) with ESMTPA id 8223F80059;
 Tue, 10 Sep 2019 10:32:42 -0400 (EDT)
Subject: Re: panic: rcv_start < rcv_end
To: Michael Tuexen <tuexen@freebsd.org>
Cc: freebsd-current <freebsd-current@freebsd.org>
References: <1601b17b-764f-ebaf-5b00-6ff26c82680e@yuripv.net>
 <D71ACAB5-8DB1-460A-BEFE-C398F9A12FB3@freebsd.org>
From: Yuri Pankov <yuripv@yuripv.net>
Message-ID: <b5b4c606-6d34-7ff0-4c47-bde073d155f4@yuripv.net>
Date: Tue, 10 Sep 2019 17:32:41 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101
 Thunderbird/68.0
MIME-Version: 1.0
In-Reply-To: <D71ACAB5-8DB1-460A-BEFE-C398F9A12FB3@freebsd.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46SSDX3H0yz4W57
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=yuripv.net header.s=fm1 header.b=HfH5/GPV;
 dkim=pass header.d=messagingengine.com header.s=fm3 header.b=v/O9wDpi;
 dmarc=none;
 spf=pass (mx1.freebsd.org: domain of yuripv@yuripv.net designates 66.111.4.224
 as permitted sender) smtp.mailfrom=yuripv@yuripv.net
X-Spamd-Result: default: False [-5.63 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[];
 R_DKIM_ALLOW(-0.20)[yuripv.net:s=fm1,messagingengine.com:s=fm3];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
 R_SPF_ALLOW(-0.20)[+ip4:66.111.4.224];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
 RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[yuripv.net];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4];
 IP_SCORE(-3.03)[ip: (-7.56), ipnet: 66.111.4.0/24(-4.84), asn: 11403(-2.68),
 country: US(-0.05)]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[yuripv.net:+,messagingengine.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 RCVD_IN_DNSWL_LOW(-0.10)[224.4.111.66.list.dnswl.org : 127.0.5.1];
 FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US];
 MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 14:32:45 -0000

Michael Tuexen wrote:
>> On 10. Sep 2019, at 14:37, Yuri Pankov <yuripv@yuripv.net> wrote:
>>
>> Just seen this almost immediately after booting the system installed from amd64-20190906-r351901 snapshot, trying to do initial pkg bootstrap.  Sadly, I didn't have the swap/dump device configured at the time, so no dump was saved.
>>
>> But it looks like I'm not alone, seeing the https://forums.freebsd.org/threads/kernel-panic-on-bhyve-virtualization.72222/ topic.  Note that I'm running on bare metal, so bhyve isn't involved. My panic screenshot is at https://pasteboard.co/IwLaXXb.jpg.
>>
>> In (the most likely) case it's not helpful enough, I'm now running with dump device configured, and will update if/when the panic reproduces.
> This panic should be fixed by:
> https://svnweb.freebsd.org/changeset/base/352072
> 
> Please drop me a note if not.

Good to know, thanks!  I only seen it once after installing the snapshot 
that didn't include the fix, and didn't notice that the fix is already 
committed.

From owner-freebsd-current@freebsd.org  Tue Sep 10 16:33:44 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 708B8DBA66
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 16:33:44 +0000 (UTC) (envelope-from hps@selasky.org)
Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SVw74CcLz4dj9
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 16:33:43 +0000 (UTC)
 (envelope-from hps@selasky.org)
Received: from hps2016.home.selasky.org (unknown [62.141.129.235])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits))
 (No client certificate requested)
 by mail.turbocat.net (Postfix) with ESMTPSA id 52B492602ED;
 Tue, 10 Sep 2019 18:33:40 +0200 (CEST)
To: FreeBSD Current <freebsd-current@freebsd.org>, Warner Losh <imp@bsdimp.com>
From: Hans Petter Selasky <hps@selasky.org>
Subject: Source tree has many empty directories?
Message-ID: <b3fb681e-e543-2da1-e628-55d856dfef5d@selasky.org>
Date: Tue, 10 Sep 2019 18:32:54 +0200
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101
 Thunderbird/68.1.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46SVw74CcLz4dj9
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of hps@selasky.org designates
 2a01:4f8:c17:6c4b::2 as permitted sender) smtp.mailfrom=hps@selasky.org
X-Spamd-Result: default: False [-3.87 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 FROM_HAS_DN(0.00)[];
 R_SPF_ALLOW(-0.20)[+a:mail.turbocat.net:c];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 SUBJECT_ENDS_QUESTION(1.00)[]; DMARC_NA(0.00)[selasky.org];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 RCPT_COUNT_TWO(0.00)[2];
 IP_SCORE(-2.57)[ip: (-9.07), ipnet: 2a01:4f8::/29(-1.97), asn: 24940(-1.78),
 country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[];
 R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE];
 MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 16:33:44 -0000

Hi Developers,

My -head source tree might be dirty over the years, but there appears to 
be some empty directories. Can these just be removed?

--HPS

find . -type d -empty
./sys/fs/nandfs
./sys/mips/gxemul
./sys/gnu/dts/include/dt-bindings/genpd
./sys/modules/drm/r128
./sys/modules/drm/sis
./sys/modules/drm/via
./sys/modules/drm/drm
./sys/modules/drm/mach64
./sys/modules/drm/mga
./sys/modules/drm/tdfx
./sys/modules/drm/savage
./sys/modules/if_tun
./sys/modules/nandfs
./sys/modules/nand
./sys/modules/nandsim
./sys/modules/drm2/drm2
./sys/modules/drm2/radeonkmsfw/ARUBA_me
./sys/modules/drm2/radeonkmsfw/VERDE_ce
./sys/modules/drm2/radeonkmsfw/TURKS_pfp
./sys/modules/drm2/radeonkmsfw/HAINAN_mc
./sys/modules/drm2/radeonkmsfw/CAYMAN_pfp
./sys/modules/drm2/radeonkmsfw/HAINAN_me
./sys/modules/drm2/radeonkmsfw/BARTS_pfp
./sys/modules/drm2/radeonkmsfw/CAICOS_mc
./sys/modules/drm2/radeonkmsfw/CAICOS_me
./sys/modules/drm2/radeonkmsfw/CEDAR_pfp
./sys/modules/drm2/radeonkmsfw/RV710_pfp
./sys/modules/drm2/radeonkmsfw/RV630_pfp
./sys/modules/drm2/radeonkmsfw/R600_rlc
./sys/modules/drm2/radeonkmsfw/TAHITI_ce
./sys/modules/drm2/radeonkmsfw/RV670_pfp
./sys/modules/drm2/radeonkmsfw/BARTS_mc
./sys/modules/drm2/radeonkmsfw/ARUBA_rlc
./sys/modules/drm2/radeonkmsfw/RV635_pfp
./sys/modules/drm2/radeonkmsfw/BARTS_me
./sys/modules/drm2/radeonkmsfw/CYPRESS_pfp
./sys/modules/drm2/radeonkmsfw/PALM_pfp
./sys/modules/drm2/radeonkmsfw/HAINAN_rlc
./sys/modules/drm2/radeonkmsfw/RV710_me
./sys/modules/drm2/radeonkmsfw/OLAND_pfp
./sys/modules/drm2/radeonkmsfw/RV730_me
./sys/modules/drm2/radeonkmsfw/OLAND_ce
./sys/modules/drm2/radeonkmsfw/R200_cp
./sys/modules/drm2/radeonkmsfw/RV770_me
./sys/modules/drm2/radeonkmsfw/REDWOOD_pfp
./sys/modules/drm2/radeonkmsfw/SUMO2_pfp
./sys/modules/drm2/radeonkmsfw/JUNIPER_rlc
./sys/modules/drm2/radeonkmsfw/PITCAIRN_pfp
./sys/modules/drm2/radeonkmsfw/PITCAIRN_ce
./sys/modules/drm2/radeonkmsfw/SUMO_rlc
./sys/modules/drm2/radeonkmsfw/REDWOOD_me
./sys/modules/drm2/radeonkmsfw/TAHITI_pfp
./sys/modules/drm2/radeonkmsfw/CEDAR_me
./sys/modules/drm2/radeonkmsfw/SUMO_uvd
./sys/modules/drm2/radeonkmsfw/VERDE_rlc
./sys/modules/drm2/radeonkmsfw/HAINAN_ce
./sys/modules/drm2/radeonkmsfw/CAICOS_pfp
./sys/modules/drm2/radeonkmsfw/R300_cp
./sys/modules/drm2/radeonkmsfw/BTC_rlc
./sys/modules/drm2/radeonkmsfw/CAYMAN_rlc
./sys/modules/drm2/radeonkmsfw/CEDAR_rlc
./sys/modules/drm2/radeonkmsfw/RV610_pfp
./sys/modules/drm2/radeonkmsfw/VERDE_mc
./sys/modules/drm2/radeonkmsfw/VERDE_me
./sys/modules/drm2/radeonkmsfw/RV730_pfp
./sys/modules/drm2/radeonkmsfw/CYPRESS_rlc
./sys/modules/drm2/radeonkmsfw/R700_rlc
./sys/modules/drm2/radeonkmsfw/RS780_pfp
./sys/modules/drm2/radeonkmsfw/RV770_pfp
./sys/modules/drm2/radeonkmsfw/R600_pfp
./sys/modules/drm2/radeonkmsfw/RV710_uvd
./sys/modules/drm2/radeonkmsfw/JUNIPER_me
./sys/modules/drm2/radeonkmsfw/OLAND_rlc
./sys/modules/drm2/radeonkmsfw/ARUBA_pfp
./sys/modules/drm2/radeonkmsfw/TAHITI_mc
./sys/modules/drm2/radeonkmsfw/TAHITI_me
./sys/modules/drm2/radeonkmsfw/HAINAN_pfp
./sys/modules/drm2/radeonkmsfw/REDWOOD_rlc
./sys/modules/drm2/radeonkmsfw/RS780_me
./sys/modules/drm2/radeonkmsfw/CYPRESS_uvd
./sys/modules/drm2/radeonkmsfw/RV635_me
./sys/modules/drm2/radeonkmsfw/R600_me
./sys/modules/drm2/radeonkmsfw/R420_cp
./sys/modules/drm2/radeonkmsfw/PITCAIRN_rlc
./sys/modules/drm2/radeonkmsfw/PALM_me
./sys/modules/drm2/radeonkmsfw/OLAND_mc
./sys/modules/drm2/radeonkmsfw/OLAND_me
./sys/modules/drm2/radeonkmsfw/JUNIPER_pfp
./sys/modules/drm2/radeonkmsfw/TAHITI_rlc
./sys/modules/drm2/radeonkmsfw/RV620_pfp
./sys/modules/drm2/radeonkmsfw/SUMO2_me
./sys/modules/drm2/radeonkmsfw/CAYMAN_mc
./sys/modules/drm2/radeonkmsfw/TURKS_mc
./sys/modules/drm2/radeonkmsfw/PITCAIRN_mc
./sys/modules/drm2/radeonkmsfw/SUMO_pfp
./sys/modules/drm2/radeonkmsfw/CAYMAN_me
./sys/modules/drm2/radeonkmsfw/TURKS_me
./sys/modules/drm2/radeonkmsfw/PITCAIRN_me
./sys/modules/drm2/radeonkmsfw/RS600_cp
./sys/modules/drm2/radeonkmsfw/RV610_me
./sys/modules/drm2/radeonkmsfw/RV620_me
./sys/modules/drm2/radeonkmsfw/TAHITI_uvd
./sys/modules/drm2/radeonkmsfw/RV630_me
./sys/modules/drm2/radeonkmsfw/R100_cp
./sys/modules/drm2/radeonkmsfw/SUMO_me
./sys/modules/drm2/radeonkmsfw/RS690_cp
./sys/modules/drm2/radeonkmsfw/RV670_me
./sys/modules/drm2/radeonkmsfw/CYPRESS_me
./sys/modules/drm2/radeonkmsfw/R520_cp
./sys/modules/drm2/radeonkmsfw/VERDE_pfp
./sys/modules/drm2/i915kms
./sys/modules/drm2/radeonkms
./sys/modules/if_tap
./sys/dev/nand
./crypto/heimdal/lib/sqlite
./usr.bin/send-pr
./sbin/nandfs
./sbin/newfs_nandfs
./tools/tools/nanobsd/gateworks/Files/root
./tools/tools/nanobsd/gateworks/cfg/ssh
./tools/tools/nanobsd/rescue/Pkg
./contrib/traceroute/lbl
./contrib/ipfilter/net
./contrib/ipfilter/ipsd/Celler
./contrib/netbsd-tests/dev/usb/libhid
./contrib/netbsd-tests/dev/usb/t_hid
./contrib/netbsd-tests/crypto/libcrypto/x509v3
./contrib/netbsd-tests/crypto/libcrypto/rsa
./contrib/netbsd-tests/crypto/libcrypto/rc2
./contrib/netbsd-tests/crypto/libcrypto/bf
./contrib/netbsd-tests/crypto/libcrypto/rc4
./contrib/netbsd-tests/crypto/libcrypto/rc5
./contrib/netbsd-tests/crypto/libcrypto/dh
./contrib/netbsd-tests/crypto/libcrypto/lhash
./contrib/netbsd-tests/crypto/libcrypto/bn/exp
./contrib/netbsd-tests/crypto/libcrypto/bn/bn
./contrib/netbsd-tests/crypto/libcrypto/bn/div
./contrib/netbsd-tests/crypto/libcrypto/idea
./contrib/netbsd-tests/crypto/libcrypto/sha
./contrib/netbsd-tests/crypto/libcrypto/ecdsa
./contrib/netbsd-tests/crypto/libcrypto/ripemd
./contrib/netbsd-tests/crypto/libcrypto/md2
./contrib/netbsd-tests/crypto/libcrypto/md4
./contrib/netbsd-tests/crypto/libcrypto/rand
./contrib/netbsd-tests/crypto/libcrypto/md5
./contrib/netbsd-tests/crypto/libcrypto/mdc2
./contrib/netbsd-tests/crypto/libcrypto/ec
./contrib/netbsd-tests/crypto/libcrypto/cast
./contrib/netbsd-tests/crypto/libcrypto/evp
./contrib/netbsd-tests/crypto/libcrypto/threads
./contrib/netbsd-tests/crypto/libcrypto/sha1
./contrib/netbsd-tests/crypto/libcrypto/ecdh
./contrib/netbsd-tests/crypto/libcrypto/srp
./contrib/netbsd-tests/crypto/libcrypto/engine
./contrib/netbsd-tests/crypto/libcrypto/dsa
./contrib/netbsd-tests/crypto/libcrypto/des
./contrib/netbsd-tests/crypto/libcrypto/hmac
./contrib/netbsd-tests/lib/libtre
./contrib/netbsd-tests/lib/libposix/posix2
./contrib/netbsd-tests/lib/libposix/bsd
./contrib/netbsd-tests/lib/libposix/posix1
./contrib/apr/include/private
./contrib/wpa/patches
./contrib/wpa/src/hlr_auc_gw
./contrib/wpa/wpa_supplicant/tests
./contrib/compiler-rt/lib/builtins/armv6m
./contrib/compiler-rt/lib/sancov
./contrib/llvm/tools/lldb/source/Plugins/OperatingSystem/Go
./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Go
./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Java
./contrib/llvm/tools/lldb/source/Plugins/ExpressionParser/Go
./contrib/llvm/tools/lldb/source/Plugins/Language/Go
./contrib/llvm/tools/lldb/source/Plugins/Language/Java
./contrib/llvm/tools/lldb/source/Plugins/Language/OCaml
./contrib/llvm/tools/llvm-mca/include/HardwareUnits
./contrib/llvm/tools/llvm-mca/include/Stages
./contrib/llvm/tools/llvm-mca/lib/HardwareUnits
./contrib/llvm/tools/llvm-mca/lib/Stages
./contrib/llvm/include/llvm/MC/MCAnalysis
./contrib/llvm/include/llvm/BinaryFormat/WasmRelocs
./contrib/llvm/include/llvm/TextAPI/MachO
./contrib/llvm/lib/ExecutionEngine/JIT
./contrib/llvm/lib/MC/MCAnalysis
./contrib/llvm/lib/Target/Nios2/MCTargetDesc
./contrib/llvm/lib/Target/Nios2/TargetInfo
./contrib/llvm/lib/Target/Nios2/InstPrinter
./contrib/llvm/lib/TextAPI/MachO
./contrib/libxo/m4
./usr.sbin/nandsim
./usr.sbin/nandtool
./usr.sbin/bsdconfig/fdisk
./lib/libnandfs
./cddl/contrib/opensolaris/common/avl
./stand/sparc64/zfsloader

From owner-freebsd-current@freebsd.org  Tue Sep 10 16:59:02 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id C998EDCB71
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 16:59:02 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com
 [IPv6:2607:f8b0:4864:20::842])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SWTL1HkPz3D7k
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 16:59:01 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: by mail-qt1-x842.google.com with SMTP id r5so21617022qtd.0
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 09:59:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=bsdimp-com.20150623.gappssmtp.com; s=20150623;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=vvArdjmWeDEfzaC1tGkfhOuL7e+sDPnvEEVm1RDnopU=;
 b=SWz2+AmXLxnpBzCHdtuggNuZacsUSSvDLKbUpu4I3SXv6UkDLplMb5H7II7G5N/qJ3
 KmkqySdXvy8geSjoAsfdE1LGTY/rbgYeTi5eFS8VM/VO0FTzlJnQ8TLeZdUL31tAOkQG
 UXDnEf1ZxrGSkexLZvkx0ouVRSJVjo5jFLCPy14WLrmHTnA/uYe6WjhQpRXr55ku2IwM
 8TAZGwUCr1ZFIKtyGm8AEL32fDJVN4WLbJ7ZRSrdvC2GyHAveMpjaW2FYn1id74+a6tc
 A850IqwJeGCqA9Pr8/Az6Q4z5UFiC9yLSk6HI+4S+mMSvUpMxELOpZS9HCJnoJsEezAA
 NWzw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=vvArdjmWeDEfzaC1tGkfhOuL7e+sDPnvEEVm1RDnopU=;
 b=uVXMNksdL3EOZbJmaFghs158cgNatn2G5E9/ekVUjFe6mv6b8Ird6JRNq5DcoWvKQ8
 ytKljkSlsX0TWgStSi5kZ55CXXpPL1DtR22N7iewxvbwzhMNP40bW/Hqquvlrmbmr2W/
 rFSYrK5nJMeNfr9OVaCf7A08tXuVRVA+3EIyXi+4OC45pvs1ofwaET33H0ckZaKIo5aS
 +SrfWDCW/3KxsIWaKplsdTCr6Vy8P4UcG2d2BHIJngRp9tqifUWune+FP7mGGRBLdHPI
 26UvyH9TobvWdFVH65b0Hx4snUJEGbhji56/EBOTAQuMrWnGdrfdUMLMG3lwSQFgXdNQ
 EXrw==
X-Gm-Message-State: APjAAAUhcmSMvmZ9XD7pF0XmVAL1T7/yrrRSEw+4plWmTCU+yiKVjZ25
 Fsx/Ov6bWooSUPwP8NNKh7uWctx+Db+NsTakf9EWH7n/hEQ=
X-Google-Smtp-Source: APXvYqwfjxYv0pivL8oHXdNLOub3dlRqh4yMGRoHn7ml1wot7O3yybdWytV1owjpwz6ZbHWGC4k4CQKktI5mC4ZLX7U=
X-Received: by 2002:ac8:71cb:: with SMTP id i11mr28754320qtp.32.1568134740878; 
 Tue, 10 Sep 2019 09:59:00 -0700 (PDT)
MIME-Version: 1.0
References: <b3fb681e-e543-2da1-e628-55d856dfef5d@selasky.org>
In-Reply-To: <b3fb681e-e543-2da1-e628-55d856dfef5d@selasky.org>
From: Warner Losh <imp@bsdimp.com>
Date: Tue, 10 Sep 2019 10:58:50 -0600
Message-ID: <CANCZdfqkw-NYcG=B8+HyD4Amr7wkWPoX0ewMhM2Gywq9XqjDtQ@mail.gmail.com>
Subject: Re: Source tree has many empty directories?
To: Hans Petter Selasky <hps@selasky.org>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
X-Rspamd-Queue-Id: 46SWTL1HkPz3D7k
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623
 header.b=SWz2+AmX; dmarc=none;
 spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when
 checking 2607:f8b0:4864:20::842) smtp.mailfrom=wlosh@bsdimp.com
X-Spamd-Result: default: False [-1.58 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623];
 FROM_HAS_DN(0.00)[];
 IP_SCORE(-0.58)[ip: (2.16), ipnet: 2607:f8b0::/32(-2.73), asn: 15169(-2.26),
 country: US(-0.05)]; 
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[bsdimp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 RCVD_IN_DNSWL_NONE(0.00)[2.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; R_SPF_NA(0.00)[];
 FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com];
 SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com];
 RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 16:59:02 -0000

On Tue, Sep 10, 2019 at 10:33 AM Hans Petter Selasky <hps@selasky.org>
wrote:

> Hi Developers,
>
> My -head source tree might be dirty over the years, but there appears to
> be some empty directories. Can these just be removed?
>

I've removed the ones I know are safe to remove, trying to mirror the
commits they were originally made empty. I can do the rest if nobody else
objects if people would like...

Warner


> --HPS
>
> find . -type d -empty
> ./sys/fs/nandfs
> ./sys/mips/gxemul
> ./sys/gnu/dts/include/dt-bindings/genpd
> ./sys/modules/drm/r128
> ./sys/modules/drm/sis
> ./sys/modules/drm/via
> ./sys/modules/drm/drm
> ./sys/modules/drm/mach64
> ./sys/modules/drm/mga
> ./sys/modules/drm/tdfx
> ./sys/modules/drm/savage
> ./sys/modules/if_tun
> ./sys/modules/nandfs
> ./sys/modules/nand
> ./sys/modules/nandsim
> ./sys/modules/drm2/drm2
> ./sys/modules/drm2/radeonkmsfw/ARUBA_me
> ./sys/modules/drm2/radeonkmsfw/VERDE_ce
> ./sys/modules/drm2/radeonkmsfw/TURKS_pfp
> ./sys/modules/drm2/radeonkmsfw/HAINAN_mc
> ./sys/modules/drm2/radeonkmsfw/CAYMAN_pfp
> ./sys/modules/drm2/radeonkmsfw/HAINAN_me
> ./sys/modules/drm2/radeonkmsfw/BARTS_pfp
> ./sys/modules/drm2/radeonkmsfw/CAICOS_mc
> ./sys/modules/drm2/radeonkmsfw/CAICOS_me
> ./sys/modules/drm2/radeonkmsfw/CEDAR_pfp
> ./sys/modules/drm2/radeonkmsfw/RV710_pfp
> ./sys/modules/drm2/radeonkmsfw/RV630_pfp
> ./sys/modules/drm2/radeonkmsfw/R600_rlc
> ./sys/modules/drm2/radeonkmsfw/TAHITI_ce
> ./sys/modules/drm2/radeonkmsfw/RV670_pfp
> ./sys/modules/drm2/radeonkmsfw/BARTS_mc
> ./sys/modules/drm2/radeonkmsfw/ARUBA_rlc
> ./sys/modules/drm2/radeonkmsfw/RV635_pfp
> ./sys/modules/drm2/radeonkmsfw/BARTS_me
> ./sys/modules/drm2/radeonkmsfw/CYPRESS_pfp
> ./sys/modules/drm2/radeonkmsfw/PALM_pfp
> ./sys/modules/drm2/radeonkmsfw/HAINAN_rlc
> ./sys/modules/drm2/radeonkmsfw/RV710_me
> ./sys/modules/drm2/radeonkmsfw/OLAND_pfp
> ./sys/modules/drm2/radeonkmsfw/RV730_me
> ./sys/modules/drm2/radeonkmsfw/OLAND_ce
> ./sys/modules/drm2/radeonkmsfw/R200_cp
> ./sys/modules/drm2/radeonkmsfw/RV770_me
> ./sys/modules/drm2/radeonkmsfw/REDWOOD_pfp
> ./sys/modules/drm2/radeonkmsfw/SUMO2_pfp
> ./sys/modules/drm2/radeonkmsfw/JUNIPER_rlc
> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_pfp
> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_ce
> ./sys/modules/drm2/radeonkmsfw/SUMO_rlc
> ./sys/modules/drm2/radeonkmsfw/REDWOOD_me
> ./sys/modules/drm2/radeonkmsfw/TAHITI_pfp
> ./sys/modules/drm2/radeonkmsfw/CEDAR_me
> ./sys/modules/drm2/radeonkmsfw/SUMO_uvd
> ./sys/modules/drm2/radeonkmsfw/VERDE_rlc
> ./sys/modules/drm2/radeonkmsfw/HAINAN_ce
> ./sys/modules/drm2/radeonkmsfw/CAICOS_pfp
> ./sys/modules/drm2/radeonkmsfw/R300_cp
> ./sys/modules/drm2/radeonkmsfw/BTC_rlc
> ./sys/modules/drm2/radeonkmsfw/CAYMAN_rlc
> ./sys/modules/drm2/radeonkmsfw/CEDAR_rlc
> ./sys/modules/drm2/radeonkmsfw/RV610_pfp
> ./sys/modules/drm2/radeonkmsfw/VERDE_mc
> ./sys/modules/drm2/radeonkmsfw/VERDE_me
> ./sys/modules/drm2/radeonkmsfw/RV730_pfp
> ./sys/modules/drm2/radeonkmsfw/CYPRESS_rlc
> ./sys/modules/drm2/radeonkmsfw/R700_rlc
> ./sys/modules/drm2/radeonkmsfw/RS780_pfp
> ./sys/modules/drm2/radeonkmsfw/RV770_pfp
> ./sys/modules/drm2/radeonkmsfw/R600_pfp
> ./sys/modules/drm2/radeonkmsfw/RV710_uvd
> ./sys/modules/drm2/radeonkmsfw/JUNIPER_me
> ./sys/modules/drm2/radeonkmsfw/OLAND_rlc
> ./sys/modules/drm2/radeonkmsfw/ARUBA_pfp
> ./sys/modules/drm2/radeonkmsfw/TAHITI_mc
> ./sys/modules/drm2/radeonkmsfw/TAHITI_me
> ./sys/modules/drm2/radeonkmsfw/HAINAN_pfp
> ./sys/modules/drm2/radeonkmsfw/REDWOOD_rlc
> ./sys/modules/drm2/radeonkmsfw/RS780_me
> ./sys/modules/drm2/radeonkmsfw/CYPRESS_uvd
> ./sys/modules/drm2/radeonkmsfw/RV635_me
> ./sys/modules/drm2/radeonkmsfw/R600_me
> ./sys/modules/drm2/radeonkmsfw/R420_cp
> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_rlc
> ./sys/modules/drm2/radeonkmsfw/PALM_me
> ./sys/modules/drm2/radeonkmsfw/OLAND_mc
> ./sys/modules/drm2/radeonkmsfw/OLAND_me
> ./sys/modules/drm2/radeonkmsfw/JUNIPER_pfp
> ./sys/modules/drm2/radeonkmsfw/TAHITI_rlc
> ./sys/modules/drm2/radeonkmsfw/RV620_pfp
> ./sys/modules/drm2/radeonkmsfw/SUMO2_me
> ./sys/modules/drm2/radeonkmsfw/CAYMAN_mc
> ./sys/modules/drm2/radeonkmsfw/TURKS_mc
> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_mc
> ./sys/modules/drm2/radeonkmsfw/SUMO_pfp
> ./sys/modules/drm2/radeonkmsfw/CAYMAN_me
> ./sys/modules/drm2/radeonkmsfw/TURKS_me
> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_me
> ./sys/modules/drm2/radeonkmsfw/RS600_cp
> ./sys/modules/drm2/radeonkmsfw/RV610_me
> ./sys/modules/drm2/radeonkmsfw/RV620_me
> ./sys/modules/drm2/radeonkmsfw/TAHITI_uvd
> ./sys/modules/drm2/radeonkmsfw/RV630_me
> ./sys/modules/drm2/radeonkmsfw/R100_cp
> ./sys/modules/drm2/radeonkmsfw/SUMO_me
> ./sys/modules/drm2/radeonkmsfw/RS690_cp
> ./sys/modules/drm2/radeonkmsfw/RV670_me
> ./sys/modules/drm2/radeonkmsfw/CYPRESS_me
> ./sys/modules/drm2/radeonkmsfw/R520_cp
> ./sys/modules/drm2/radeonkmsfw/VERDE_pfp
> ./sys/modules/drm2/i915kms
> ./sys/modules/drm2/radeonkms
> ./sys/modules/if_tap
> ./sys/dev/nand
> ./crypto/heimdal/lib/sqlite
> ./usr.bin/send-pr
> ./sbin/nandfs
> ./sbin/newfs_nandfs
> ./tools/tools/nanobsd/gateworks/Files/root
> ./tools/tools/nanobsd/gateworks/cfg/ssh
> ./tools/tools/nanobsd/rescue/Pkg
> ./contrib/traceroute/lbl
> ./contrib/ipfilter/net
> ./contrib/ipfilter/ipsd/Celler
> ./contrib/netbsd-tests/dev/usb/libhid
> ./contrib/netbsd-tests/dev/usb/t_hid
> ./contrib/netbsd-tests/crypto/libcrypto/x509v3
> ./contrib/netbsd-tests/crypto/libcrypto/rsa
> ./contrib/netbsd-tests/crypto/libcrypto/rc2
> ./contrib/netbsd-tests/crypto/libcrypto/bf
> ./contrib/netbsd-tests/crypto/libcrypto/rc4
> ./contrib/netbsd-tests/crypto/libcrypto/rc5
> ./contrib/netbsd-tests/crypto/libcrypto/dh
> ./contrib/netbsd-tests/crypto/libcrypto/lhash
> ./contrib/netbsd-tests/crypto/libcrypto/bn/exp
> ./contrib/netbsd-tests/crypto/libcrypto/bn/bn
> ./contrib/netbsd-tests/crypto/libcrypto/bn/div
> ./contrib/netbsd-tests/crypto/libcrypto/idea
> ./contrib/netbsd-tests/crypto/libcrypto/sha
> ./contrib/netbsd-tests/crypto/libcrypto/ecdsa
> ./contrib/netbsd-tests/crypto/libcrypto/ripemd
> ./contrib/netbsd-tests/crypto/libcrypto/md2
> ./contrib/netbsd-tests/crypto/libcrypto/md4
> ./contrib/netbsd-tests/crypto/libcrypto/rand
> ./contrib/netbsd-tests/crypto/libcrypto/md5
> ./contrib/netbsd-tests/crypto/libcrypto/mdc2
> ./contrib/netbsd-tests/crypto/libcrypto/ec
> ./contrib/netbsd-tests/crypto/libcrypto/cast
> ./contrib/netbsd-tests/crypto/libcrypto/evp
> ./contrib/netbsd-tests/crypto/libcrypto/threads
> ./contrib/netbsd-tests/crypto/libcrypto/sha1
> ./contrib/netbsd-tests/crypto/libcrypto/ecdh
> ./contrib/netbsd-tests/crypto/libcrypto/srp
> ./contrib/netbsd-tests/crypto/libcrypto/engine
> ./contrib/netbsd-tests/crypto/libcrypto/dsa
> ./contrib/netbsd-tests/crypto/libcrypto/des
> ./contrib/netbsd-tests/crypto/libcrypto/hmac
> ./contrib/netbsd-tests/lib/libtre
> ./contrib/netbsd-tests/lib/libposix/posix2
> ./contrib/netbsd-tests/lib/libposix/bsd
> ./contrib/netbsd-tests/lib/libposix/posix1
> ./contrib/apr/include/private
> ./contrib/wpa/patches
> ./contrib/wpa/src/hlr_auc_gw
> ./contrib/wpa/wpa_supplicant/tests
> ./contrib/compiler-rt/lib/builtins/armv6m
> ./contrib/compiler-rt/lib/sancov
> ./contrib/llvm/tools/lldb/source/Plugins/OperatingSystem/Go
> ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Go
> ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Java
> ./contrib/llvm/tools/lldb/source/Plugins/ExpressionParser/Go
> ./contrib/llvm/tools/lldb/source/Plugins/Language/Go
> ./contrib/llvm/tools/lldb/source/Plugins/Language/Java
> ./contrib/llvm/tools/lldb/source/Plugins/Language/OCaml
> ./contrib/llvm/tools/llvm-mca/include/HardwareUnits
> ./contrib/llvm/tools/llvm-mca/include/Stages
> ./contrib/llvm/tools/llvm-mca/lib/HardwareUnits
> ./contrib/llvm/tools/llvm-mca/lib/Stages
> ./contrib/llvm/include/llvm/MC/MCAnalysis
> ./contrib/llvm/include/llvm/BinaryFormat/WasmRelocs
> ./contrib/llvm/include/llvm/TextAPI/MachO
> ./contrib/llvm/lib/ExecutionEngine/JIT
> ./contrib/llvm/lib/MC/MCAnalysis
> ./contrib/llvm/lib/Target/Nios2/MCTargetDesc
> ./contrib/llvm/lib/Target/Nios2/TargetInfo
> ./contrib/llvm/lib/Target/Nios2/InstPrinter
> ./contrib/llvm/lib/TextAPI/MachO
> ./contrib/libxo/m4
> ./usr.sbin/nandsim
> ./usr.sbin/nandtool
> ./usr.sbin/bsdconfig/fdisk
> ./lib/libnandfs
> ./cddl/contrib/opensolaris/common/avl
> ./stand/sparc64/zfsloader
>

From owner-freebsd-current@freebsd.org  Tue Sep 10 17:01:45 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id DF992DD194
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 17:01:45 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com
 [IPv6:2607:f8b0:4864:20::742])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SWXT0Z5Tz3DcX
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 17:01:44 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: by mail-qk1-x742.google.com with SMTP id f13so17740438qkm.9
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 10:01:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=bsdimp-com.20150623.gappssmtp.com; s=20150623;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=MzC7XonyX1sn+W0a/fTD2Ew+4sXvlQNu6/uqhcDr2Qg=;
 b=Lod1d2ms35aam0lML0b6nF3apZ46XFyYXoTqcVKXxK+8Tna9Qxzv58ZXkchl1+6VaA
 +MqOnZjLN2pwtNsrxRpBJF3yHZtmedzPJ76aKKyU8uEsaFeDRfy1w8Y8iKnDGLDDdx03
 x07PNOa0Z5Dhc3TsSTxzfkOIH/jjgrzDxIcntofigq1bt8xwjonQiZIhe+HptZXjpSrg
 lNBMLcn+v6NDWeBdhW3xR8+ssw4bKKYtAsmTbBsUn+N0wv7Egs4hEaJfoYCiezKX4O1n
 aTRx6fA07UJ89sUlx4Yry0f21AGil+3YS06XMt8H4GLKRvVz1HWyHdORflfoP8mB+zKF
 klaQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=MzC7XonyX1sn+W0a/fTD2Ew+4sXvlQNu6/uqhcDr2Qg=;
 b=sybMZl5yLIjiPpxeWREx2vGNbg3jKA6LAXmyMuMQQowpelLNGmiNfOqC2hvWrXNyAJ
 /CgU7qoKN/mQZshGQinUknN4CMITvBLejWfzOdveppUfwz6Sj9Lux6mYztn8pc4CAIYg
 nYVDI5Xwnqe5ERVyf7+bcElu+edyhUvZBXJWWYRs82ARVsSgnCapkpv/FbJuwf1zXQRg
 QZ6WbURK9pE+VQpXCuTh95O5e4NLC7rA8J3csxoqcHli6NNLunRyK4dW6tVvLveZHKjW
 mPg16q1r2cLZEHf0POWGEs90RTenLSRQubyW6Nb4uCATGwrB4fyccgzjKqlVEqOa27fp
 MkXg==
X-Gm-Message-State: APjAAAVdUCHtocK+y2L3D1ki5LwJg7xMmAFLkpaHt+GdvfprFe/qKf+k
 6AIFqsKcBKZYDYElgvhCNXxpxZykm4yu74s+DKouCb7bZS4TXg==
X-Google-Smtp-Source: APXvYqyRT0pb3FpeZCiUwok8rHpglqqgypYK7mM05g4Shd7i9XFgrhrE9AwH9iESoVlAz3WsCfbXlkKCXGKPQQnjT+c=
X-Received: by 2002:a05:620a:1671:: with SMTP id
 d17mr20273337qko.495.1568134903831; 
 Tue, 10 Sep 2019 10:01:43 -0700 (PDT)
MIME-Version: 1.0
References: <b3fb681e-e543-2da1-e628-55d856dfef5d@selasky.org>
 <CANCZdfqkw-NYcG=B8+HyD4Amr7wkWPoX0ewMhM2Gywq9XqjDtQ@mail.gmail.com>
In-Reply-To: <CANCZdfqkw-NYcG=B8+HyD4Amr7wkWPoX0ewMhM2Gywq9XqjDtQ@mail.gmail.com>
From: Warner Losh <imp@bsdimp.com>
Date: Tue, 10 Sep 2019 11:01:32 -0600
Message-ID: <CANCZdfp7ctPRhGC+-_TqNp-BcegOnrw=DE2zDKkAGHDLc5Oq+w@mail.gmail.com>
Subject: Re: Source tree has many empty directories?
To: Hans Petter Selasky <hps@selasky.org>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
X-Rspamd-Queue-Id: 46SWXT0Z5Tz3DcX
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623
 header.b=Lod1d2ms; dmarc=none;
 spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when
 checking 2607:f8b0:4864:20::742) smtp.mailfrom=wlosh@bsdimp.com
X-Spamd-Result: default: False [-1.39 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0];
 R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623];
 FROM_HAS_DN(0.00)[];
 IP_SCORE(-0.39)[ip: (3.10), ipnet: 2607:f8b0::/32(-2.73), asn: 15169(-2.26),
 country: US(-0.05)]; 
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[bsdimp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 RCVD_IN_DNSWL_NONE(0.00)[2.4.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; R_SPF_NA(0.00)[];
 FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com];
 SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com];
 RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 17:01:45 -0000

On Tue, Sep 10, 2019 at 10:58 AM Warner Losh <imp@bsdimp.com> wrote:

>
> On Tue, Sep 10, 2019 at 10:33 AM Hans Petter Selasky <hps@selasky.org>
> wrote:
>
>> Hi Developers,
>>
>> My -head source tree might be dirty over the years, but there appears to
>> be some empty directories. Can these just be removed?
>>
>
> I've removed the ones I know are safe to remove, trying to mirror the
> commits they were originally made empty. I can do the rest if nobody else
> objects if people would like...
>

The rest being:
/contrib/llvm/include/llvm/BinaryFormat/WasmRelocs
./contrib/llvm/include/llvm/MC/MCAnalysis
./contrib/llvm/include/llvm/TextAPI/MachO
./contrib/llvm/lib/Target/Nios2/MCTargetDesc
./contrib/llvm/lib/Target/Nios2/TargetInfo
./contrib/llvm/lib/Target/Nios2/InstPrinter
./contrib/llvm/lib/ExecutionEngine/JIT
./contrib/llvm/lib/MC/MCAnalysis
./contrib/llvm/lib/TextAPI/MachO
./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Go
./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Java
./contrib/llvm/tools/lldb/source/Plugins/OperatingSystem/Go
./contrib/llvm/tools/lldb/source/Plugins/ExpressionParser/Go
./contrib/llvm/tools/lldb/source/Plugins/Language/Go
./contrib/llvm/tools/lldb/source/Plugins/Language/Java
./contrib/llvm/tools/lldb/source/Plugins/Language/OCaml
./contrib/llvm/tools/llvm-mca/include/HardwareUnits
./contrib/llvm/tools/llvm-mca/include/Stages
./contrib/llvm/tools/llvm-mca/lib/HardwareUnits
./contrib/llvm/tools/llvm-mca/lib/Stages
./contrib/wpa/patches
./contrib/wpa/src/hlr_auc_gw
./contrib/wpa/wpa_supplicant/tests
./contrib/compiler-rt/lib/builtins/armv6m
./contrib/compiler-rt/lib/sancov
./contrib/libxo/m4
./contrib/ipfilter/net
./contrib/ipfilter/ipsd/Celler

However, please do *NOT* remove the sys/*/compile directories.

Warner


> Warner
>
>
>> --HPS
>>
>> find . -type d -empty
>> ./sys/fs/nandfs
>> ./sys/mips/gxemul
>> ./sys/gnu/dts/include/dt-bindings/genpd
>> ./sys/modules/drm/r128
>> ./sys/modules/drm/sis
>> ./sys/modules/drm/via
>> ./sys/modules/drm/drm
>> ./sys/modules/drm/mach64
>> ./sys/modules/drm/mga
>> ./sys/modules/drm/tdfx
>> ./sys/modules/drm/savage
>> ./sys/modules/if_tun
>> ./sys/modules/nandfs
>> ./sys/modules/nand
>> ./sys/modules/nandsim
>> ./sys/modules/drm2/drm2
>> ./sys/modules/drm2/radeonkmsfw/ARUBA_me
>> ./sys/modules/drm2/radeonkmsfw/VERDE_ce
>> ./sys/modules/drm2/radeonkmsfw/TURKS_pfp
>> ./sys/modules/drm2/radeonkmsfw/HAINAN_mc
>> ./sys/modules/drm2/radeonkmsfw/CAYMAN_pfp
>> ./sys/modules/drm2/radeonkmsfw/HAINAN_me
>> ./sys/modules/drm2/radeonkmsfw/BARTS_pfp
>> ./sys/modules/drm2/radeonkmsfw/CAICOS_mc
>> ./sys/modules/drm2/radeonkmsfw/CAICOS_me
>> ./sys/modules/drm2/radeonkmsfw/CEDAR_pfp
>> ./sys/modules/drm2/radeonkmsfw/RV710_pfp
>> ./sys/modules/drm2/radeonkmsfw/RV630_pfp
>> ./sys/modules/drm2/radeonkmsfw/R600_rlc
>> ./sys/modules/drm2/radeonkmsfw/TAHITI_ce
>> ./sys/modules/drm2/radeonkmsfw/RV670_pfp
>> ./sys/modules/drm2/radeonkmsfw/BARTS_mc
>> ./sys/modules/drm2/radeonkmsfw/ARUBA_rlc
>> ./sys/modules/drm2/radeonkmsfw/RV635_pfp
>> ./sys/modules/drm2/radeonkmsfw/BARTS_me
>> ./sys/modules/drm2/radeonkmsfw/CYPRESS_pfp
>> ./sys/modules/drm2/radeonkmsfw/PALM_pfp
>> ./sys/modules/drm2/radeonkmsfw/HAINAN_rlc
>> ./sys/modules/drm2/radeonkmsfw/RV710_me
>> ./sys/modules/drm2/radeonkmsfw/OLAND_pfp
>> ./sys/modules/drm2/radeonkmsfw/RV730_me
>> ./sys/modules/drm2/radeonkmsfw/OLAND_ce
>> ./sys/modules/drm2/radeonkmsfw/R200_cp
>> ./sys/modules/drm2/radeonkmsfw/RV770_me
>> ./sys/modules/drm2/radeonkmsfw/REDWOOD_pfp
>> ./sys/modules/drm2/radeonkmsfw/SUMO2_pfp
>> ./sys/modules/drm2/radeonkmsfw/JUNIPER_rlc
>> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_pfp
>> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_ce
>> ./sys/modules/drm2/radeonkmsfw/SUMO_rlc
>> ./sys/modules/drm2/radeonkmsfw/REDWOOD_me
>> ./sys/modules/drm2/radeonkmsfw/TAHITI_pfp
>> ./sys/modules/drm2/radeonkmsfw/CEDAR_me
>> ./sys/modules/drm2/radeonkmsfw/SUMO_uvd
>> ./sys/modules/drm2/radeonkmsfw/VERDE_rlc
>> ./sys/modules/drm2/radeonkmsfw/HAINAN_ce
>> ./sys/modules/drm2/radeonkmsfw/CAICOS_pfp
>> ./sys/modules/drm2/radeonkmsfw/R300_cp
>> ./sys/modules/drm2/radeonkmsfw/BTC_rlc
>> ./sys/modules/drm2/radeonkmsfw/CAYMAN_rlc
>> ./sys/modules/drm2/radeonkmsfw/CEDAR_rlc
>> ./sys/modules/drm2/radeonkmsfw/RV610_pfp
>> ./sys/modules/drm2/radeonkmsfw/VERDE_mc
>> ./sys/modules/drm2/radeonkmsfw/VERDE_me
>> ./sys/modules/drm2/radeonkmsfw/RV730_pfp
>> ./sys/modules/drm2/radeonkmsfw/CYPRESS_rlc
>> ./sys/modules/drm2/radeonkmsfw/R700_rlc
>> ./sys/modules/drm2/radeonkmsfw/RS780_pfp
>> ./sys/modules/drm2/radeonkmsfw/RV770_pfp
>> ./sys/modules/drm2/radeonkmsfw/R600_pfp
>> ./sys/modules/drm2/radeonkmsfw/RV710_uvd
>> ./sys/modules/drm2/radeonkmsfw/JUNIPER_me
>> ./sys/modules/drm2/radeonkmsfw/OLAND_rlc
>> ./sys/modules/drm2/radeonkmsfw/ARUBA_pfp
>> ./sys/modules/drm2/radeonkmsfw/TAHITI_mc
>> ./sys/modules/drm2/radeonkmsfw/TAHITI_me
>> ./sys/modules/drm2/radeonkmsfw/HAINAN_pfp
>> ./sys/modules/drm2/radeonkmsfw/REDWOOD_rlc
>> ./sys/modules/drm2/radeonkmsfw/RS780_me
>> ./sys/modules/drm2/radeonkmsfw/CYPRESS_uvd
>> ./sys/modules/drm2/radeonkmsfw/RV635_me
>> ./sys/modules/drm2/radeonkmsfw/R600_me
>> ./sys/modules/drm2/radeonkmsfw/R420_cp
>> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_rlc
>> ./sys/modules/drm2/radeonkmsfw/PALM_me
>> ./sys/modules/drm2/radeonkmsfw/OLAND_mc
>> ./sys/modules/drm2/radeonkmsfw/OLAND_me
>> ./sys/modules/drm2/radeonkmsfw/JUNIPER_pfp
>> ./sys/modules/drm2/radeonkmsfw/TAHITI_rlc
>> ./sys/modules/drm2/radeonkmsfw/RV620_pfp
>> ./sys/modules/drm2/radeonkmsfw/SUMO2_me
>> ./sys/modules/drm2/radeonkmsfw/CAYMAN_mc
>> ./sys/modules/drm2/radeonkmsfw/TURKS_mc
>> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_mc
>> ./sys/modules/drm2/radeonkmsfw/SUMO_pfp
>> ./sys/modules/drm2/radeonkmsfw/CAYMAN_me
>> ./sys/modules/drm2/radeonkmsfw/TURKS_me
>> ./sys/modules/drm2/radeonkmsfw/PITCAIRN_me
>> ./sys/modules/drm2/radeonkmsfw/RS600_cp
>> ./sys/modules/drm2/radeonkmsfw/RV610_me
>> ./sys/modules/drm2/radeonkmsfw/RV620_me
>> ./sys/modules/drm2/radeonkmsfw/TAHITI_uvd
>> ./sys/modules/drm2/radeonkmsfw/RV630_me
>> ./sys/modules/drm2/radeonkmsfw/R100_cp
>> ./sys/modules/drm2/radeonkmsfw/SUMO_me
>> ./sys/modules/drm2/radeonkmsfw/RS690_cp
>> ./sys/modules/drm2/radeonkmsfw/RV670_me
>> ./sys/modules/drm2/radeonkmsfw/CYPRESS_me
>> ./sys/modules/drm2/radeonkmsfw/R520_cp
>> ./sys/modules/drm2/radeonkmsfw/VERDE_pfp
>> ./sys/modules/drm2/i915kms
>> ./sys/modules/drm2/radeonkms
>> ./sys/modules/if_tap
>> ./sys/dev/nand
>> ./crypto/heimdal/lib/sqlite
>> ./usr.bin/send-pr
>> ./sbin/nandfs
>> ./sbin/newfs_nandfs
>> ./tools/tools/nanobsd/gateworks/Files/root
>> ./tools/tools/nanobsd/gateworks/cfg/ssh
>> ./tools/tools/nanobsd/rescue/Pkg
>> ./contrib/traceroute/lbl
>> ./contrib/ipfilter/net
>> ./contrib/ipfilter/ipsd/Celler
>> ./contrib/netbsd-tests/dev/usb/libhid
>> ./contrib/netbsd-tests/dev/usb/t_hid
>> ./contrib/netbsd-tests/crypto/libcrypto/x509v3
>> ./contrib/netbsd-tests/crypto/libcrypto/rsa
>> ./contrib/netbsd-tests/crypto/libcrypto/rc2
>> ./contrib/netbsd-tests/crypto/libcrypto/bf
>> ./contrib/netbsd-tests/crypto/libcrypto/rc4
>> ./contrib/netbsd-tests/crypto/libcrypto/rc5
>> ./contrib/netbsd-tests/crypto/libcrypto/dh
>> ./contrib/netbsd-tests/crypto/libcrypto/lhash
>> ./contrib/netbsd-tests/crypto/libcrypto/bn/exp
>> ./contrib/netbsd-tests/crypto/libcrypto/bn/bn
>> ./contrib/netbsd-tests/crypto/libcrypto/bn/div
>> ./contrib/netbsd-tests/crypto/libcrypto/idea
>> ./contrib/netbsd-tests/crypto/libcrypto/sha
>> ./contrib/netbsd-tests/crypto/libcrypto/ecdsa
>> ./contrib/netbsd-tests/crypto/libcrypto/ripemd
>> ./contrib/netbsd-tests/crypto/libcrypto/md2
>> ./contrib/netbsd-tests/crypto/libcrypto/md4
>> ./contrib/netbsd-tests/crypto/libcrypto/rand
>> ./contrib/netbsd-tests/crypto/libcrypto/md5
>> ./contrib/netbsd-tests/crypto/libcrypto/mdc2
>> ./contrib/netbsd-tests/crypto/libcrypto/ec
>> ./contrib/netbsd-tests/crypto/libcrypto/cast
>> ./contrib/netbsd-tests/crypto/libcrypto/evp
>> ./contrib/netbsd-tests/crypto/libcrypto/threads
>> ./contrib/netbsd-tests/crypto/libcrypto/sha1
>> ./contrib/netbsd-tests/crypto/libcrypto/ecdh
>> ./contrib/netbsd-tests/crypto/libcrypto/srp
>> ./contrib/netbsd-tests/crypto/libcrypto/engine
>> ./contrib/netbsd-tests/crypto/libcrypto/dsa
>> ./contrib/netbsd-tests/crypto/libcrypto/des
>> ./contrib/netbsd-tests/crypto/libcrypto/hmac
>> ./contrib/netbsd-tests/lib/libtre
>> ./contrib/netbsd-tests/lib/libposix/posix2
>> ./contrib/netbsd-tests/lib/libposix/bsd
>> ./contrib/netbsd-tests/lib/libposix/posix1
>> ./contrib/apr/include/private
>> ./contrib/wpa/patches
>> ./contrib/wpa/src/hlr_auc_gw
>> ./contrib/wpa/wpa_supplicant/tests
>> ./contrib/compiler-rt/lib/builtins/armv6m
>> ./contrib/compiler-rt/lib/sancov
>> ./contrib/llvm/tools/lldb/source/Plugins/OperatingSystem/Go
>> ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Go
>> ./contrib/llvm/tools/lldb/source/Plugins/LanguageRuntime/Java
>> ./contrib/llvm/tools/lldb/source/Plugins/ExpressionParser/Go
>> ./contrib/llvm/tools/lldb/source/Plugins/Language/Go
>> ./contrib/llvm/tools/lldb/source/Plugins/Language/Java
>> ./contrib/llvm/tools/lldb/source/Plugins/Language/OCaml
>> ./contrib/llvm/tools/llvm-mca/include/HardwareUnits
>> ./contrib/llvm/tools/llvm-mca/include/Stages
>> ./contrib/llvm/tools/llvm-mca/lib/HardwareUnits
>> ./contrib/llvm/tools/llvm-mca/lib/Stages
>> ./contrib/llvm/include/llvm/MC/MCAnalysis
>> ./contrib/llvm/include/llvm/BinaryFormat/WasmRelocs
>> ./contrib/llvm/include/llvm/TextAPI/MachO
>> ./contrib/llvm/lib/ExecutionEngine/JIT
>> ./contrib/llvm/lib/MC/MCAnalysis
>> ./contrib/llvm/lib/Target/Nios2/MCTargetDesc
>> ./contrib/llvm/lib/Target/Nios2/TargetInfo
>> ./contrib/llvm/lib/Target/Nios2/InstPrinter
>> ./contrib/llvm/lib/TextAPI/MachO
>> ./contrib/libxo/m4
>> ./usr.sbin/nandsim
>> ./usr.sbin/nandtool
>> ./usr.sbin/bsdconfig/fdisk
>> ./lib/libnandfs
>> ./cddl/contrib/opensolaris/common/avl
>> ./stand/sparc64/zfsloader
>>
>

From owner-freebsd-current@freebsd.org  Tue Sep 10 18:14:11 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7DF89E0074
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 18:14:11 +0000 (UTC) (envelope-from ian@freebsd.org)
Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org
 [54.149.155.156])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SY832G7Qz3Ln9
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 18:14:11 +0000 (UTC)
 (envelope-from ian@freebsd.org)
ARC-Seal: i=1; a=rsa-sha256; t=1568139250; cv=none;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 b=FePgCyzd8qNil9tzva7MfF6JpB2dYkuycu0QhHXHdCKsaR9cE143TvVyzwibOc8GdXJqjA+0Y4GgX
 qTEs9Ib/Q7cC2Ob07mttJTmgzHi7BdHHMR1iKbNHD7TL8EskxHbvDgpAfD3TxciKhtXwHCQCiorJlZ
 mB2weAXi9QZ3kNh/nfRVJMb7iEzlsfk4vOG+0EZbmek2Uc+Z8oSa+9FQXzFdqdXezZDpkG3dQcCUkI
 ppw+L7Ie4uEqL1BlbNeatQMko6BD8PlwAlgXzuiAOtAqZmDumWMPYwcNLPFozJf2wpR71vzveFc94v
 yMP7Qe2oRL5pVOs0RgmhAzubzeXt4YQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:dkim-signature:from;
 bh=g2BMxC2/jtidTqEEAP8W6x/rtNAvYl6lDBABqtKcbLE=;
 b=eUjZcn2o0sV+ZzufpDUkghq3gCrqngsufiRsIkevAOBxiyrWTSWub5xMokYV43UyMuo3nIlwG8ci6
 LpJzF6CtzfC1FGx0nPfB8tLizqauh2aZfMium1RhRxk+ZlataruUf/ZaWkBVIyqR0GFc0xMU6OEmTG
 bemLWeh2RAkqgep8gkqilRwDtx9AUnhP3YQTZse5h4zbOkvxT9nwdnB/SmptIYIUFaNDiKZkAxhHCF
 uQ35fAyHQnVu4bhK06E+UxwAZm7T/y9hWlLClotPnxL05hGSC/OJKBk/s67eNaP7WMevUdMswp+I6k
 AONkhY1ZYvi6XFylzUL7YP/nLL2Z3hA==
ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org;
 spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60;
 dmarc=none header.from=freebsd.org;
 arc=none header.oldest-pass=0;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=dkim-high;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:from;
 bh=g2BMxC2/jtidTqEEAP8W6x/rtNAvYl6lDBABqtKcbLE=;
 b=bTGiiRi7cOu8rmAjPU+NEym85FhvcKkPUVWA5Flu8RMTfLbdELTC/0BwiRW8VCnBELkqFSOL1wg2k
 c+VQAuBNkgND0SLZZRRHUW75eKGHvxaoNuVLMUiTuSBNWoMwsJDFfIssINqwAmYWy6OVTo85TZ62tw
 90/SY1VGIszHfLZS9mWtIH5YYpmVTYKBgMgfCcw4SyIbCbmVXDi9O7Iz3fYtttl9Ly7CG2iHOc4wSZ
 bgkgi6Yybg8do0rFe87uCiDcwvbICF0DwoF87zH2cewFqeOGIX+UT15npbR3qULImFU9o9d3J5Uzip
 MESEzF9D9r/P8ooJJOEWso5RpRmV0dA==
X-MHO-RoutePath: aGlwcGll
X-MHO-User: c6a1ac29-d3f6-11e9-85ed-13b9aae3a1d2
X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information
X-Originating-IP: 67.177.211.60
X-Mail-Handler: DuoCircle Outbound SMTP
Received: from ilsoft.org (unknown [67.177.211.60])
 by outbound4.ore.mailhop.org (Halon) with ESMTPSA
 id c6a1ac29-d3f6-11e9-85ed-13b9aae3a1d2;
 Tue, 10 Sep 2019 18:14:07 +0000 (UTC)
Received: from rev (rev [172.22.42.240])
 by ilsoft.org (8.15.2/8.15.2) with ESMTP id x8AIE5BH067623;
 Tue, 10 Sep 2019 12:14:05 -0600 (MDT) (envelope-from ian@freebsd.org)
Message-ID: <ed46d0d5e781089766e7cabcce595551fdb3c501.camel@freebsd.org>
Subject: Re: Source tree has many empty directories?
From: Ian Lepore <ian@freebsd.org>
To: Warner Losh <imp@bsdimp.com>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Date: Tue, 10 Sep 2019 12:14:05 -0600
In-Reply-To: <CANCZdfp7ctPRhGC+-_TqNp-BcegOnrw=DE2zDKkAGHDLc5Oq+w@mail.gmail.com>
References: <b3fb681e-e543-2da1-e628-55d856dfef5d@selasky.org>
 <CANCZdfqkw-NYcG=B8+HyD4Amr7wkWPoX0ewMhM2Gywq9XqjDtQ@mail.gmail.com>
 <CANCZdfp7ctPRhGC+-_TqNp-BcegOnrw=DE2zDKkAGHDLc5Oq+w@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46SY832G7Qz3Ln9
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-2.00 / 15.00];
 local_wl_from(0.00)[freebsd.org];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0];
 NEURAL_HAM_LONG(-1.00)[-1.000,0];
 ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 18:14:11 -0000

On Tue, 2019-09-10 at 11:01 -0600, Warner Losh wrote:
> However, please do *NOT* remove the sys/*/compile directories.
> 
> Warner

Uhhh... that's interesting.  I just nuked one of those on my system
yesterday, because it had been hanging around since 2013 and I had no
idea what was -- I just assumed the build machinery created it because
I had accidentally done a make in a wrong directory once.

So what are those directories about?  I'm not used to seeing mystery
directories appear inside a source tree.

-- Ian


From owner-freebsd-current@freebsd.org  Tue Sep 10 19:41:08 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6407E225D
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 19:41:08 +0000 (UTC) (envelope-from dim@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (4096 bits) client-digest SHA256)
 (Client CN "smtp.freebsd.org",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Sb4N5qMjz3QYG;
 Tue, 10 Sep 2019 19:41:08 +0000 (UTC) (envelope-from dim@FreeBSD.org)
Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:470:7a58:1::1])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
 bits)) (Client CN "tensor.andric.com",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 (Authenticated sender: dim)
 by smtp.freebsd.org (Postfix) with ESMTPSA id 92F5416BD2;
 Tue, 10 Sep 2019 19:41:08 +0000 (UTC) (envelope-from dim@FreeBSD.org)
Received: from [IPv6:2001:470:7a58::74d0:76b2:f541:2f0c] (unknown
 [IPv6:2001:470:7a58:0:74d0:76b2:f541:2f0c])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by tensor.andric.com (Postfix) with ESMTPSA id 5625147C77;
 Tue, 10 Sep 2019 21:41:07 +0200 (CEST)
From: Dimitry Andric <dim@FreeBSD.org>
Message-Id: <289ADD78-FD39-435B-80D5-91901FE065DD@FreeBSD.org>
Content-Type: multipart/signed;
 boundary="Apple-Mail=_BB7FCB97-51CC-4187-9732-B98C4374371D";
 protocol="application/pgp-signature"; micalg=pgp-sha1
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: Source tree has many empty directories?
Date: Tue, 10 Sep 2019 21:41:03 +0200
In-Reply-To: <ed46d0d5e781089766e7cabcce595551fdb3c501.camel@freebsd.org>
Cc: Warner Losh <imp@bsdimp.com>, FreeBSD Current <freebsd-current@freebsd.org>
To: Ian Lepore <ian@FreeBSD.org>
References: <b3fb681e-e543-2da1-e628-55d856dfef5d@selasky.org>
 <CANCZdfqkw-NYcG=B8+HyD4Amr7wkWPoX0ewMhM2Gywq9XqjDtQ@mail.gmail.com>
 <CANCZdfp7ctPRhGC+-_TqNp-BcegOnrw=DE2zDKkAGHDLc5Oq+w@mail.gmail.com>
 <ed46d0d5e781089766e7cabcce595551fdb3c501.camel@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 19:41:09 -0000


--Apple-Mail=_BB7FCB97-51CC-4187-9732-B98C4374371D
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On 10 Sep 2019, at 20:14, Ian Lepore <ian@FreeBSD.org> wrote:
>=20
> On Tue, 2019-09-10 at 11:01 -0600, Warner Losh wrote:
>> However, please do *NOT* remove the sys/*/compile directories.
>>=20
>> Warner
>=20
> Uhhh... that's interesting.  I just nuked one of those on my system
> yesterday, because it had been hanging around since 2013 and I had no
> idea what was -- I just assumed the build machinery created it because
> I had accidentally done a make in a wrong directory once.
>=20
> So what are those directories about?  I'm not used to seeing mystery
> directories appear inside a source tree.

There's not much mystery to be found.  Subversion does not warn you when =
you
remove the last files from a directory, and it also does not =
automatically
remove such empty directories, like Git.  Hence, those directories tend =
to
stick around, because every simply forgets about them.

With regards to those empty directories under contrib/llvm, those were =
actually
imported from upstream.  But since the llvm project will switch to Git =
soon,
this problem will automagically disappear. :-)

-Dimitry


--Apple-Mail=_BB7FCB97-51CC-4187-9732-B98C4374371D
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.2

iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCXXf8TwAKCRCwXqMKLiCW
o7cGAKDOzpwbyGGwYSL74TSUjC1JrYpdOwCgsDHOh2aYTppN0uAQ2qqBGWwKWPI=
=6CWy
-----END PGP SIGNATURE-----

--Apple-Mail=_BB7FCB97-51CC-4187-9732-B98C4374371D--

From owner-freebsd-current@freebsd.org  Tue Sep 10 19:43:02 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id D23BBE23AD
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 19:43:02 +0000 (UTC) (envelope-from ian@freebsd.org)
Received: from outbound2m.ore.mailhop.org (outbound2m.ore.mailhop.org
 [54.149.155.156])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Sb6Z3HsKz3Qv0
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 19:43:02 +0000 (UTC)
 (envelope-from ian@freebsd.org)
ARC-Seal: i=1; a=rsa-sha256; t=1568144581; cv=none;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 b=I8A6Aic5xitGiZHELEXfsyCqEhUUU70CQW3G4LcBCgdNBy4V2MUSlnZYakZq7XVBqhkj1AYQnrXwy
 kkK+r0ijdPSbm7b39ba38CkMNpjC/Pg9G3NnOUfa+BwVvgAOI83qUNFwGOvVpxVNdITbIqdJIHDWKE
 WHKefgUA+yNIM3fkc85ZEAgqyA/AXeMd/XN9jTxD1Lb7Ow2YVBYEYnc+1hXBaiI1Blets7jRdBxCtn
 +AE/GszY/r2xUI7cNbwSRPsTfalflej4+Q88zlLv0NgmF84wHwSEKxenSimR05oAz1nGKuCu6071MG
 Xwq6eeACxlVPWO+CDdjpJ0CCnzcnH2w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:dkim-signature:from;
 bh=nkYiFlUuSdrBn728NaspgElxgeWnZGSp9RjrSsDKuAc=;
 b=u6rUUgpVs7SuCBtaARxz2aP+B+DDwPeLt7ErXYjoyp5Ztmp3OI2By2e83zSlCMhvVnFrCu18XN7ea
 dbhsv7j/KcYJSXQFJZmlBEZkubbFd100ifbOprKY+KFhceDmuXF1Idgod7LQ1lbU/cqrtzzP/TaKja
 f7HLPAH0htx+l7vjLHBj+F2ntRgfFqLy78BMgFj4sC4zdIp1jeplvjk3CPowwR48v37Sdi1S2fwDuk
 7bfEk+XhPu42B+JaMMX/IPfn/9lhD/NvGtg8ucMxzeokoG/CxlcB4v7HbwIBIBhhr+4G89rIvPfwUn
 Gq1SPk1QWiGuPqhrfBD2ioTZgqDGy4Q==
ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org;
 spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60;
 dmarc=none header.from=freebsd.org;
 arc=none header.oldest-pass=0;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=dkim-high;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:cc:to:from:subject:message-id:from;
 bh=nkYiFlUuSdrBn728NaspgElxgeWnZGSp9RjrSsDKuAc=;
 b=mtRagBa2FXW5Nv7DiiagYg6CPdgNZ3sPRRbTAOjyN6x6X4KMajGjrqU59crG0UM2zD1TrklRao2ax
 DvKHx4I0qmlpDEplKKuX8eXb2GvPYD820pzu0Sem0ZDyyoPNTl9EBit9aj0W/0dCC+sDw2qdW6XfC7
 kOBgaj0o/VCOqPj71CaYJEvq+/Yd4WuKOf4vbrFwtwfdRKe7q4BioJMNMASR5EH8F6jBpn+buv+yoU
 dXUXGC8gBruy1AcrCRSmKMxZzcwG/6gCkNoItQEkpJYucDu2h7x80uUFiMdl43P+5kMBa5XcmvpCrG
 BcxbZdpjmiN16hRfzsH2sG02qC59Rug==
X-MHO-RoutePath: aGlwcGll
X-MHO-User: 313be3de-d403-11e9-85ed-13b9aae3a1d2
X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information
X-Originating-IP: 67.177.211.60
X-Mail-Handler: DuoCircle Outbound SMTP
Received: from ilsoft.org (unknown [67.177.211.60])
 by outbound4.ore.mailhop.org (Halon) with ESMTPSA
 id 313be3de-d403-11e9-85ed-13b9aae3a1d2;
 Tue, 10 Sep 2019 19:43:00 +0000 (UTC)
Received: from rev (rev [172.22.42.240])
 by ilsoft.org (8.15.2/8.15.2) with ESMTP id x8AJgwCa067863;
 Tue, 10 Sep 2019 13:42:58 -0600 (MDT) (envelope-from ian@freebsd.org)
Message-ID: <6140ae514e02b029e7451065da08c49c749f029e.camel@freebsd.org>
Subject: Re: Source tree has many empty directories?
From: Ian Lepore <ian@freebsd.org>
To: Dimitry Andric <dim@FreeBSD.org>
Cc: Warner Losh <imp@bsdimp.com>, FreeBSD Current <freebsd-current@freebsd.org>
Date: Tue, 10 Sep 2019 13:42:58 -0600
In-Reply-To: <289ADD78-FD39-435B-80D5-91901FE065DD@FreeBSD.org>
References: <b3fb681e-e543-2da1-e628-55d856dfef5d@selasky.org>
 <CANCZdfqkw-NYcG=B8+HyD4Amr7wkWPoX0ewMhM2Gywq9XqjDtQ@mail.gmail.com>
 <CANCZdfp7ctPRhGC+-_TqNp-BcegOnrw=DE2zDKkAGHDLc5Oq+w@mail.gmail.com>
 <ed46d0d5e781089766e7cabcce595551fdb3c501.camel@freebsd.org>
 <289ADD78-FD39-435B-80D5-91901FE065DD@FreeBSD.org>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46Sb6Z3HsKz3Qv0
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-2.00 / 15.00];
 local_wl_from(0.00)[freebsd.org];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0];
 NEURAL_HAM_LONG(-1.00)[-1.000,0];
 ASN(0.00)[asn:16509, ipnet:54.148.0.0/15, country:US]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 19:43:02 -0000

On Tue, 2019-09-10 at 21:41 +0200, Dimitry Andric wrote:
> On 10 Sep 2019, at 20:14, Ian Lepore <ian@FreeBSD.org> wrote:
> > 
> > On Tue, 2019-09-10 at 11:01 -0600, Warner Losh wrote:
> > > However, please do *NOT* remove the sys/*/compile directories.
> > > 
> > > Warner
> > 
> > Uhhh... that's interesting.  I just nuked one of those on my system
> > yesterday, because it had been hanging around since 2013 and I had no
> > idea what was -- I just assumed the build machinery created it because
> > I had accidentally done a make in a wrong directory once.
> > 
> > So what are those directories about?  I'm not used to seeing mystery
> > directories appear inside a source tree.
> 
> There's not much mystery to be found.  Subversion does not warn you when you
> remove the last files from a directory, and it also does not automatically
> remove such empty directories, like Git.  Hence, those directories tend to
> stick around, because every simply forgets about them.
> 
> With regards to those empty directories under contrib/llvm, those were actually
> imported from upstream.  But since the llvm project will switch to Git soon,
> this problem will automagically disappear. :-)
> 
> -Dimitry
> 

I was referring specifically to the sys/*/compile directories Warner
mentioned.

-- Ian


From owner-freebsd-current@freebsd.org  Tue Sep 10 19:48:30 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 85A9BE2620
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 19:48:30 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com
 [IPv6:2607:f8b0:4864:20::741])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SbDs5DlCz3R8W
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 19:48:29 +0000 (UTC)
 (envelope-from wlosh@bsdimp.com)
Received: by mail-qk1-x741.google.com with SMTP id x5so18314000qkh.5
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 12:48:29 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=bsdimp-com.20150623.gappssmtp.com; s=20150623;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=oYinMlE0C7pCJgPKN/9oA5BzF+lHzM/G/B/7N9eucAw=;
 b=Pu6VliFNoOD64Q9hbgGGEwCdp8G9SW8my80hKX8uqDhCk9UtjbCPqrN55ilgAaKRyu
 WyrQKjkpymXei+e3sLwnqRmEB8EP8oHOqwR6azKpgBfuzzDZ20n+2d6wSq6mCrDHJKcJ
 VVu6PHtwYVwTVS3TByvuK50ptIoEa0q1k6oJ22jx6Lb1cnYS77vtdUQs2Ucq2FOZ343n
 O7LLctWyAe/9ZyGeF+1kgRjWrupTCY/AnbYAatdLG2SqcWpz0FVX3URMZIwUvrmHsUBx
 SXhq/+9oxsScMdfDrOs/T3OFzNYvGZwgs1ZBHS17coW4xH6S3LxeGoHgzYjylCCRYdF7
 cwBw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=oYinMlE0C7pCJgPKN/9oA5BzF+lHzM/G/B/7N9eucAw=;
 b=iq+keyhYU6j0SeUIs9rtUI6ifqgl3QgcW8lHJ3XTR9uUWBIw6MJ8yrtoh/heVWH+vh
 c4BgEwGQ3oDwLx3THFhJtodQUqFUKxsMIzzp24MKq1LN9AeTjnC8bJGHHws9pmP3pfk0
 SxVfAj9FALhPImUWOr3KrtHA0nwb61h4hUWeouFT31dN+fV67xYCPE20Q1Fmc5jsRO2J
 nf3OIzQB2t9suODm13br8VDM6bhBjbdlAR9IRvUe00ikoT3w6WO/qxKztcI7DPx7mTRd
 bVE9sfvD2CslTB4Y1JHbTNwIEm39djXTNNwCCvNyimkEM7IhpG6yQoTj7TrTwaJEztv7
 8e1Q==
X-Gm-Message-State: APjAAAUZAeYxY1XsKFvpETQHUQ1v66lmNPSkr+HtaxpxtIfDulhUgfga
 +x/JSz1yjN2DWcd0j32PcdVqGgTklVktEXNO95s3zw==
X-Google-Smtp-Source: APXvYqzyFE/3r8ZgjhjaigItbRssGzdmoiDS+w8cr54E5RKSzzAu43TJqh/Ndgw5Y6IDC1daPyWyRmzMRidoqJJejJo=
X-Received: by 2002:a05:620a:16ab:: with SMTP id
 s11mr32239864qkj.215.1568144908285; 
 Tue, 10 Sep 2019 12:48:28 -0700 (PDT)
MIME-Version: 1.0
References: <b3fb681e-e543-2da1-e628-55d856dfef5d@selasky.org>
 <CANCZdfqkw-NYcG=B8+HyD4Amr7wkWPoX0ewMhM2Gywq9XqjDtQ@mail.gmail.com>
 <CANCZdfp7ctPRhGC+-_TqNp-BcegOnrw=DE2zDKkAGHDLc5Oq+w@mail.gmail.com>
 <ed46d0d5e781089766e7cabcce595551fdb3c501.camel@freebsd.org>
 <289ADD78-FD39-435B-80D5-91901FE065DD@FreeBSD.org>
 <6140ae514e02b029e7451065da08c49c749f029e.camel@freebsd.org>
In-Reply-To: <6140ae514e02b029e7451065da08c49c749f029e.camel@freebsd.org>
From: Warner Losh <imp@bsdimp.com>
Date: Tue, 10 Sep 2019 13:48:16 -0600
Message-ID: <CANCZdfrFQeNfLF2-sE9op7JTQ_8azv=A4j2Hc2Fib9dd1hCsVg@mail.gmail.com>
Subject: Re: Source tree has many empty directories?
To: Ian Lepore <ian@freebsd.org>
Cc: Dimitry Andric <dim@freebsd.org>,
 FreeBSD Current <freebsd-current@freebsd.org>
X-Rspamd-Queue-Id: 46SbDs5DlCz3R8W
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623
 header.b=Pu6VliFN; dmarc=none;
 spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when
 checking 2607:f8b0:4864:20::741) smtp.mailfrom=wlosh@bsdimp.com
X-Spamd-Result: default: False [-1.52 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623];
 FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3];
 IP_SCORE(-0.52)[ip: (2.42), ipnet: 2607:f8b0::/32(-2.73), asn: 15169(-2.26),
 country: US(-0.05)]; 
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[bsdimp.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+];
 RCVD_IN_DNSWL_NONE(0.00)[1.4.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; R_SPF_NA(0.00)[];
 FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com];
 SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com];
 RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 19:48:30 -0000

On Tue, Sep 10, 2019, 1:43 PM Ian Lepore <ian@freebsd.org> wrote:

> On Tue, 2019-09-10 at 21:41 +0200, Dimitry Andric wrote:
> > On 10 Sep 2019, at 20:14, Ian Lepore <ian@FreeBSD.org> wrote:
> > >
> > > On Tue, 2019-09-10 at 11:01 -0600, Warner Losh wrote:
> > > > However, please do *NOT* remove the sys/*/compile directories.
> > > >
> > > > Warner
> > >
> > > Uhhh... that's interesting.  I just nuked one of those on my system
> > > yesterday, because it had been hanging around since 2013 and I had no
> > > idea what was -- I just assumed the build machinery created it because
> > > I had accidentally done a make in a wrong directory once.
> > >
> > > So what are those directories about?  I'm not used to seeing mystery
> > > directories appear inside a source tree.
> >
> > There's not much mystery to be found.  Subversion does not warn you when
> you
> > remove the last files from a directory, and it also does not
> automatically
> > remove such empty directories, like Git.  Hence, those directories tend
> to
> > stick around, because every simply forgets about them.
> >
> > With regards to those empty directories under contrib/llvm, those were
> actually
> > imported from upstream.  But since the llvm project will switch to Git
> soon,
> > this problem will automagically disappear. :-)
> >
> > -Dimitry
> >
>
> I was referring specifically to the sys/*/compile directories Warner
> mentioned.
>

They are for old school builds. Config(8) doesn't create them, so we have
them in the tree. Because svn does this, we deleted the keep me files after
the cut over.

Warner

>

From owner-freebsd-current@freebsd.org  Tue Sep 10 22:22:14 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id A5E2AE66B7
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Tue, 10 Sep 2019 22:22:14 +0000 (UTC)
 (envelope-from rebecca@bsdio.com)
Received: from muon.bsdio.com (muon.bluestop.org [65.103.231.193])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SffF5pK5z46XH;
 Tue, 10 Sep 2019 22:22:13 +0000 (UTC)
 (envelope-from rebecca@bsdio.com)
Received: from muon.bsdio.com (localhost [127.0.0.1])
 by muon.bsdio.com (Postfix) with ESMTP id 8DF44109D09;
 Tue, 10 Sep 2019 16:22:18 -0600 (MDT)
Received: from muon.bsdio.com ([127.0.0.1])
 by muon.bsdio.com (muon.bsdio.com [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id VtFfv6kJCTyT; Tue, 10 Sep 2019 16:22:18 -0600 (MDT)
Received: from photon.int.bluestop.org (unknown [10.0.10.120])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest
 SHA256) (No client certificate requested)
 by muon.bsdio.com (Postfix) with ESMTPSA;
 Tue, 10 Sep 2019 16:22:18 -0600 (MDT)
Subject: Re: Building world with gcc9 not working
To: Warner Losh <imp@bsdimp.com>, "Conrad E. Meyer" <cem@freebsd.org>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
References: <4c4d4ccf-4662-572a-d6ff-7ce9a0c36f38@bsdio.com>
 <CAG6CVpU2-Ekz0GiLRF1Kf=SWxUDrQEa-c6rYaJgjRqX9z5FOsg@mail.gmail.com>
 <CANCZdfr0mP+b_=fEizvAV8+zUAQ2uCmroEGXe_K-2g6Hx5hTOA@mail.gmail.com>
From: Rebecca Cran <rebecca@bsdio.com>
Autocrypt: addr=rebecca@bsdio.com; keydata=
 mQINBFrUMZ4BEADI1yUEGeZeXeTCPay1ZpTBdDEpGPAw1dq2VCSTc1VhsnrEBa1iZxAfaeSv
 Uu5Ti7jlhQ/3sQMl0bJMKGB/RtmIW7k8h2w476oZmG8gChk8su5ZEx/pV1gdqInyFmmJKTYc
 gabJz8pL+m82w07qPv+oalepZ4dbj+HF++RAK/iEju+q9UHlsjj8e3mMNsvtrOz1K6bnpveO
 jZ+ms/2H3Hs5a4k8y6buwe2RvwhJQaXa13cR3LhzL+nwj4B9PHZZEa2WpEyYpw/bI0V9YSQN
 QgC1CYRzDyakZge6BCM6wHOgZSUzRPufGilrNKUwIVbRoIBR9/85+0wR+PlFUOUOfOc6ox7T
 dWcIx6PuPhek48rh4uwmmwsPtPiH4Z3T5p+GmWQ9NLFZKA1YnEdaSkWtYZsDxwVZZeYG2plt
 MfhXP0Hj4rf9Y3eoUenCaGioxAbUOBCtXdTGNAhNjz1g5NGDBVyhjKkzwJQvt9UrYTseERit
 5dX2CMTy8hYLvSXd/Ivy+HylUS5IslfZxW5z9LgWX7Z97kILgkH3N0ewtLkygkG+Y+x7uaAV
 dFqp9ASOyzaiwKbJdeOI+WxRSh+AqeCR0S+bpkcLudLmbjrPmaFwjKycy1H85Z5R2J3YHyXY
 oT6OYjD8vLbUU2GWp6Onkcy1Pu8EMbRuzKil6HnpYg3BexbPFwARAQABtCNSZWJlY2NhIENy
 YW4gPHJlYmVjY2FAYmx1ZXN0b3Aub3JnPokCVwQTAQgAQQIbIwUJCWYBgAULCQgHAgYVCgkI
 CwIEFgIDAQIeAQIXgBYhBB+5fZtkTdO940Yr4g0CK1MRvhAgBQJa2B9zAhkBAAoJEA0CK1MR
 vhAgzJEQAJUqVmTRO9OqCSS2CVKjrqnEWJMvyo0K8B+WiXo0nSQg9+uyoVU7h2s/kkWVGy4u
 IWbGy2Qe8LiXzBJjHC3TadGvOvakfdMeKKXcgxgX6KlhA9hA2LW6tg22aHUk7Flr/8diHpgf
 qIwrXhqJXZmK72GR1QfhgoHsOsTJ9GWPswo1kUMc0cJowq0qP1RDdua6BwvDHHPJwu9OmC/i
 oQlMNm9gkBDq8H2B+m125ANwCnqBizXaiTTLQdewTMbCSuxbsni2icDqwBfFXzEgcJGaYYfB
 cQeFsfCmtXQK3JUd4Myx128Dxk9P3X64I93SB7QzB0nmWlyvmCFBNoCp0PCLA4qbwbw2sMRX
 Wx4BqYa8nI/jg+Nqo+Ut2BfltNZIlsHxK+XhxejfLqAjRCZeLnu1otvFnFuGLaAVYx9x1Y1q
 J8VizZxq6ujio62Qpultp6KNhlkJ+OKoGwA0k4NHh26SxvlsNxlfg/2v9b1LqWRzNujnwbcF
 8g4902XjyBLxV+9YpXZEa8H6zzEHxpeDPWT3QfvrT8JuoHa1IyYnUKvG674UKW5zEGEwkQc9
 cuQwR1RHd1ZrKtH1duXzaLr/caMp8ZDfGDDxFpenJTRxNRlg4+K7HSdhpac7sBVMUA8uVdE+
 iuTThOmdf0c4DorL3BIh6Yv3FV4/NSqT1Wn3CG2fgG1guQINBFrUMZ4BEADkc4mvMcMcDF1t
 dNxNQuIBE1F243oZamG3LACCKfc1Yur3CPzHwIk5LXCUmbq23iE5bowxMWw3mlVT0p5xM0Wn
 UidIBwCKu4kRyy/fY4NyWWBuwy9srpTdmUcKRBRNB8zEZE8xIlidD1ijjgqLBfeM7n9ylawA
 xHLxwU96sdpdHFzb7Z0yKY2e/bzDaHiG0fUvcCmkgLf+uwKKZid1j8zR5PzKpgPqfy/PF01e
 KyGV3MNu8Y90xMoiEMWfCI2IB1m+hTuzZoboFvGV54SiMuvfWK/VMQjhsL6K2ddOqwVuy2nI
 MI4G3xDQW/v8KVyn43OSIAyW1eaklhzu0Ir2sO60PXRkvbTUrouvmSvpJfIQS49rU0M/X6FS
 DgXQLKrZ3my94+g8ptz9KoVml6s4OAwYVz+sb49nuSxipFKkU5FwhKOLmzbsBxCtytcUJoLm
 juJPJPDQue6YJiIXyc86GVY2pH3DjemKdbB4dSgqAJIp+lCzKSJzz7bgueh2Ox8vzx1tSxKj
 7V8Nal+UTKKbkxPmMh+e20YZ4esAVifO3bS6IJP/aDnfagghB71vA7+aWGXPbjPlc2UHpCBi
 RSsl+IgoQXvdvZBsKRyfBx8neODa2C6JIE5vcaCjilSeKF8SzsFXvimnndhQNhAPU/DwQwSX
 dCl4gTsFVi5d8Oxq1sce+wARAQABiQI8BBgBCAAmFiEEH7l9m2RN073jRiviDQIrUxG+ECAF
 AlrUMZ4CGwwFCQlmAYAACgkQDQIrUxG+ECAWnRAAsmZX+KgNxW3v7R/76Tz4Wjmh4AGeE+Ji
 3p5QsdTYny1B6vYBL9vCzPJ/AK8pgKMDRaweUP5eZQpfrdWC8Q7SNGgi4Q+97KEs+i2xZLQ+
 WJb8a+WEEIc716u0y4ITiHfOgM5jWcFO4MXQATbJgv0drLLesa+LQCvZgPBqupt307EsCubQ
 s+Sxt+RVjf6rOUolp1GJXEQYwGsKklVd6yqLC8M1BSG53/WE5tSv5GzBZ8fp6EtmjT7leuid
 FtEvKYHQz4DqG9ELpHUF0X0UUCBK/MgXe3kCVLKE060UrJ4M6uPSx57rmVFA2MvwQR8M7GsW
 C5UsSM4PYwPWBhwxE7vcx0691YKAHT/5q8LxRVBdUyzPSprMhSQFttsBt+ygm6wRi3Pi3TuC
 EARNubPkQefyeC34yr40SAUCkOl3eWxSXPF4NfXFQb4AAzZSE5hv3qbDuwo3lrL0LqpIpEQP
 Az+JZ1QZ6mMFQ5/JD9Gukj54kZc0X8w3sQt0a8vyE/qrJg8vKgv2rCHrPc5MeDkEUEFiiJiC
 EDdkJtMyoRlU3S4NrnbyLOLEcHE8fGe3hStPX8hY62id2ecdQ5WZ7vLZW5SFeLarbUciuHIk
 VL6MHnUjbV7XlY50N7ebeFCIdlCWhdum2FJs/Ni+SSxbZC564vrokwlBBGSo6WTPQTa8IWx1 DtU=
Message-ID: <c4c84775-24b9-1f94-7053-b8d9925f9e4a@bsdio.com>
Date: Tue, 10 Sep 2019 16:22:11 -0600
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101
 Thunderbird/68.1.0
MIME-Version: 1.0
In-Reply-To: <CANCZdfr0mP+b_=fEizvAV8+zUAQ2uCmroEGXe_K-2g6Hx5hTOA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Rspamd-Queue-Id: 46SffF5pK5z46XH
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=none (mx1.freebsd.org: domain of rebecca@bsdio.com has no SPF policy when
 checking 65.103.231.193) smtp.mailfrom=rebecca@bsdio.com
X-Spamd-Result: default: False [-2.93 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991,0];
 FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[bsdio.com];
 AUTH_NA(1.00)[]; RCVD_COUNT_THREE(0.00)[4];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:209, ipnet:65.103.224.0/19, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 IP_SCORE(-1.84)[ip: (-9.09), asn: 209(-0.05), country: US(-0.05)]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Sep 2019 22:22:14 -0000

On 2019-09-09 23:08, Warner Losh wrote:
>
>> Aspirationally, yes.  I did a successful CROSS_TOOLCHAIN=amd64-gcc
>> buildworld earlier this week.  (It doesn't play well with binary pkg's
>> built with Clang, so I ended up replacing it with a Clang-built world
>> instead, but it compiled.)
>>
>> Unfortunately, amd64-xtoolchain-gcc is stuck on GCC 6.4.0, while
>> you're running the much more recent GCC9.  I think GCC6.4.0 is more or
>> less expected to build world today, but I don't think many people are
>> building with GCC9.  I would love for amd64-xtoolchain to move to a
>> newer version, but I don't know what is blocking that — it seems like
>> it should be straightforward.
>>
> I did a gcc8 build about a year or so ago, though I had to turn off Werror
> to complete the build...

Thanks. I kept running into build errors with gcc 8 and 9 (even with
WERROR= in src.conf), but building with gcc 6 from the amd64-gcc port
worked.


-- 
Rebecca Cran


From owner-freebsd-current@freebsd.org  Wed Sep 11 05:58:14 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id B155FEF70B
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Wed, 11 Sep 2019 05:58:14 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic307-10.consmr.mail.ne1.yahoo.com
 (sonic307-10.consmr.mail.ne1.yahoo.com [66.163.190.33])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SrmP40Sbz4Sq9
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 05:58:13 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: _au2af4VM1lqpJ0A4OskBEOOs.7EvmIwIN4YohKajFTSAq2hnAIjO1P2GFt08oi
 rEfT8EI.WwPwxT1kut1qhkh.gNi_OPuBEgwU_Qf517JGzasHOfZoUg3Dt_WOD2WKMN_cIwZ2HIga
 e7Qijx6ULs5n1IWT.v.woZYMLBrKccsXOxmNQt7uP7_SvgDHzS9bSQiYX37I2rjNA8npQO2PGv66
 gHSWvXiiuLrrcPXcP_HxbgPVYasO072RxP0LHQs5EUjSQShVpHvhlGeAjmwN5m13Ancrmgmxef5q
 ZQ7Fre4YgJ5rfkesLnxGbdsX3hlYW_zkckyfKqh64Hk_1k.OD0qs5WutVEFAqZPhrmk6kConUMa0
 l2jdHjCPfwdzqa_o8bmX3LbnZ1kDyps6ILOlE5gJfES4Vh93BcB_qAGCatQX0h.SwEN6UET1j846
 ZKcY1GPpc1BRrtZIR3CSyYDLSbQPyXlj4xtqGlLCuZwckvagnGWL4l_avA0umuw4vgNDzKIV7Ab7
 KBl95eYZYWnDJhuC6kAlDwHMTlUX6DKFZegZ3NZPOm0A3PIYxBJSom1dOdXBD_8jd_aGenZDhcUe
 DOK3MpaBpNOorhOS9JDzWE1KnQS234Ordl3G.kDhrtMQXAQoNFjxolqF7w.4jGdCpdhBVdLU0Ct9
 y74oOrt6Z.xpIW.i9XaHxVIjI8oTNUDirVJLM451vhVzGubdg1bNCt3zyvbudhC5FyzdqecRF8T7
 qZLZHJ_WADVUpALDPBpaOJQM1Pui5gy7C3F43P84R9TFJfGNfe138mQvBCDpmRyH8jSH2sWmAbA3
 THLOlfcM4uErz1ZMiNpYY.qPlULtDwsqwf89EUJ2mZKjYdnc1CO59pftq6fTkviqp_DSSBoRaOqp
 ZChM62Bcn.QZGtJbrBdpIm0bwTe_8ixa96gagTNrHE.Lh3C99OzXj1A7DIi4Ty2gM.P9jCP9xY0Q
 rnxQ027f.17VnDY4XmTmWKAlckwvihZUkr3L0oOuJAAX.SY8PlfGW_tAULKX.sEdt0a_SBgWq0nL
 BhcGdatZVr8bAryAQnfVIp.aFqZPztWFrns42QkEFFKbkd.1UU3XU4d5N1yfihlfLbGOMCeX5syE
 5uF9Wln6FkEzJA__ouJCuZ4.A2IEfKv8Hlvx6A.5kM0qWMjk9d07_XWplwh8FQ_rsHsr6.7Jpqir
 clasZRFPicaboeSWJaciNsdS8xIfoLAfF2_EFmNSZ0.jxlp_bVMU8OFmWC9wtx3e7m4xnjotCHWI
 telyi6qplRILDda_RphLs_BQJM.wD1mZNsn0rqasRqFrXwg--
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic307.consmr.mail.ne1.yahoo.com with HTTP; Wed, 11 Sep 2019 05:58:11 +0000
Received: by smtp426.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID eb6d57ddc44af5fcdf86d8632c91fff3; 
 Wed, 11 Sep 2019 05:58:06 +0000 (UTC)
From: Mark Millard <marklmi@yahoo.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: "cpuset -n prefer:?" --what values for "?" are supposed to be
 allowed? (only 1 is, despite two numa domains)
Message-Id: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
Date: Tue, 10 Sep 2019 22:58:05 -0700
To: FreeBSD Current <freebsd-current@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
X-Rspamd-Queue-Id: 46SrmP40Sbz4Sq9
X-Spamd-Bar: -
X-Spamd-Result: default: False [-1.50 / 15.00];
 R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[];
 TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[];
 MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.86)[-0.859,0];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.14)[-0.139,0];
 MIME_GOOD(-0.10)[text/plain];
 IP_SCORE(0.00)[ip: (3.85), ipnet: 66.163.184.0/21(1.30), asn: 36646(1.04),
 country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[];
 RCPT_COUNT_ONE(0.00)[1]; SUBJECT_HAS_QUESTION(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[33.190.163.66.list.dnswl.org : 127.0.5.0];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 05:58:14 -0000

In a context with:

# cpuset -g
pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, =
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
pid -1 domain policy: first-touch mask: 0, 1

I get:

# cpuset -l0 -n prefer:0 COMMAND
cpuset: setdomain: Invalid argument

# cpuset -l0 -n prefer:2 COMMAND
cpuset: setdomain: Invalid argument

But one prefer:? value does allow the COMMAND
to run:

# cpuset -l0 -n prefer:1 COMMAND

This seem odd to me. Am I missing something?

For reference: I'm using a ThreadRipper 1950X
with a head -r351227 based context for this
activity. The above happens to have been run
in a Windows 10 Pro HyperV session, instead
of in a native-boot of the same media. (A
native-boot would have had 32 CPUs.)


=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)


From owner-freebsd-current@freebsd.org  Wed Sep 11 06:28:44 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4E96BF056E
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Wed, 11 Sep 2019 06:28:44 +0000 (UTC)
 (envelope-from clay.daniels.jr@gmail.com)
Received: from mail-ua1-x941.google.com (mail-ua1-x941.google.com
 [IPv6:2607:f8b0:4864:20::941])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46SsRb3C42z4VHs
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 06:28:43 +0000 (UTC)
 (envelope-from clay.daniels.jr@gmail.com)
Received: by mail-ua1-x941.google.com with SMTP id n6so6408587uaq.3
 for <freebsd-current@freebsd.org>; Tue, 10 Sep 2019 23:28:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=FzaLkuM7iGciEFr7QmjPzfPSYFFgdY3+qWXSfX8B/gI=;
 b=OlDWOCFLR4tfUgb0aalv5Y44v+Fm25rsZSUdDCwFoBY6SIbk2zspNX4e0GiAJsDNZs
 6RqogL0wSc+m0HDQv2dJzHwTWf5JK4vtBCwncQZPS69ADgwCkldYRO2kFo370kXOlE6G
 UCKHRyvPxwpznV6HqSPaD7Thv+jsqZfAGJcShMg1ewFgYa0gOqjtHI9trFvwdYZI4goP
 5lNFqV3bJfZVDXl9SpVqmG8nnzTnEdcYjq2yZAnEm35Ivs/stmD9N0MedVVzHT+A4nk9
 avIv+COTjDVqGHaAfFJEQAPudHNeJSUsCef/AsEmtWCxij0vBdyNibIlnUWQ28JwZfqK
 LWUg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=FzaLkuM7iGciEFr7QmjPzfPSYFFgdY3+qWXSfX8B/gI=;
 b=aCtAItGzVcjkFLA6Hqj9kfsGsH7rSbDO7zAIUtXMsihfd4up8CJvYVYUEeurf50o3L
 ZHxYBVpQSHl16Hpp7VxKnbXhU4tdB2yeiS4pOHD12CQVE3fwY5+jshaFYqZ8owp54Vr9
 CyunXCdmbBD7KxvQz1eVaYjSItBS+lzaXvTnF4/MWl9hCSBE9xy4Jq76tzrKxEOkSR2V
 cvyvKYo0t5LY1arTGGpxDXEfw1b+/YCJQwkDgGulQtr1A7g4+YRKBYXbuhCCSYd8TtYz
 ETTWNAYk9/ujYHaz06MZ/VofFlshbgdrcSM2Cz/PHDdS4dANIWop7R9hLYzp/EBnfrvx
 rYrw==
X-Gm-Message-State: APjAAAWKzO/XiYdxNHCgKbSSJ+hBtIhX9qDNU9+HuCBiJYZdsgzTC/V2
 /zAaN45Cf4Bk2/DlBXeiezakKJySOzZOtEBuqw==
X-Google-Smtp-Source: APXvYqz1zev3kZT4DYzdcDYPQqnNOdaD935siS3BmEAn++iMYQNi1ZJDub834U2dqRwlcMBrCDGbPCO/0b7DbkxAWmU=
X-Received: by 2002:ab0:5b1a:: with SMTP id u26mr15849532uae.125.1568183321754; 
 Tue, 10 Sep 2019 23:28:41 -0700 (PDT)
MIME-Version: 1.0
References: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
In-Reply-To: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
From: "Clay Daniels Jr." <clay.daniels.jr@gmail.com>
Date: Wed, 11 Sep 2019 01:28:30 -0500
Message-ID: <CAGLDxTXWtH7LcgCBwK8T+4GRFvvhT5F-c=+_ZVFXz+ObFcHqZQ@mail.gmail.com>
Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be
 allowed? (only 1 is, despite two numa domains)
To: Mark Millard <marklmi@yahoo.com>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
X-Rspamd-Queue-Id: 46SsRb3C42z4VHs
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=OlDWOCFL;
 dmarc=pass (policy=none) header.from=gmail.com;
 spf=pass (mx1.freebsd.org: domain of claydanielsjr@gmail.com designates
 2607:f8b0:4864:20::941 as permitted sender)
 smtp.mailfrom=claydanielsjr@gmail.com
X-Spamd-Result: default: False [-2.00 / 15.00];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36];
 FREEMAIL_FROM(0.00)[gmail.com]; URI_COUNT_ODD(1.00)[7];
 TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 DMARC_POLICY_ALLOW(-0.50)[gmail.com,none];
 FREEMAIL_TO(0.00)[yahoo.com]; FROM_EQ_ENVFROM(0.00)[];
 IP_SCORE(0.00)[ip: (2.78), ipnet: 2607:f8b0::/32(-2.73), asn: 15169(-2.26),
 country: US(-0.05)]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 TAGGED_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[1.4.9.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 06:28:44 -0000

Mark, this is what I get on my machine:

root@new:~ # cpuset -g
pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
pid -1 domain policy: first-touch mask: 0
root@new:~ #  cpuset -l0 -n prefer:0 COMMAND
cpuset: COMMAND: No such file or directory
root@new:~ # cpuset -l0 -n prefer:2 COMMAND
cpuset: setdomain: Invalid argument
root@new:~ # cpuset -l0 -n prefer:1 COMMAND
cpuset: setdomain: Invalid argument

>From dmesg:
FreeBSD 13.0-CURRENT r351901 GENERIC amd64
CPU: AMD Ryzen 7 3700X 8-Core Processor(3600.08-MHz K8-class CPU)

Similar,
Clay

On Wed, Sep 11, 2019 at 12:58 AM Mark Millard <marklmi@yahoo.com> wrote:

> In a context with:
>
> # cpuset -g
> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17,
> 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
> pid -1 domain policy: first-touch mask: 0, 1
>
> I get:
>
> # cpuset -l0 -n prefer:0 COMMAND
> cpuset: setdomain: Invalid argument
>
> # cpuset -l0 -n prefer:2 COMMAND
> cpuset: setdomain: Invalid argument
>
> But one prefer:? value does allow the COMMAND
> to run:
>
> # cpuset -l0 -n prefer:1 COMMAND
>
> This seem odd to me. Am I missing something?
>
> For reference: I'm using a ThreadRipper 1950X
> with a head -r351227 based context for this
> activity. The above happens to have been run
> in a Windows 10 Pro HyperV session, instead
> of in a native-boot of the same media. (A
> native-boot would have had 32 CPUs.)
>
>
> ===
> Mark Millard
> marklmi at yahoo.com
> ( dsl-only.net went
> away in early 2018-Mar)
>
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
>

From owner-freebsd-current@freebsd.org  Wed Sep 11 13:15:44 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3BE97D37CC;
 Wed, 11 Sep 2019 13:15:44 +0000 (UTC)
 (envelope-from agapon@gmail.com)
Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com
 [209.85.208.171])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46T2TB6q1qz3Qyr;
 Wed, 11 Sep 2019 13:15:42 +0000 (UTC)
 (envelope-from agapon@gmail.com)
Received: by mail-lj1-f171.google.com with SMTP id 7so19933007ljw.7;
 Wed, 11 Sep 2019 06:15:42 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:to:from:subject:openpgp:autocrypt:message-id
 :date:user-agent:mime-version:content-language
 :content-transfer-encoding;
 bh=qQcFrJ7yQyPK1T9347wZLUod9c/b5wL2lS43WxHE1Wg=;
 b=EdAcXtHTDvB2cKcNdHMLcXhHgy1p1c7Si9eJWQUGSoivPS9FzT4Erg+bvHAdE1tptS
 2oPWYmrNgrr0EbWKPBT0pHxAbFpmnxInLsuvl3ptDucsdQ4I+sT/UGOCdfFRNnoxMKb0
 SrXNrBXNvpJnNXaBj6n5OgOBS/FSVXMGeh9bXzuAIR+pYEG6JfI1FVGzYlgksmCoF2JM
 YdZzQornnKgs3ESaKY07hc+yNXlRv8lfF1cQq/Y4F9trP6jQGIuQNOddZKvqotUHMim/
 1ORIE71Wyg+CaZ5jenAkM/RTYfdWtfgrcFGdyOQ6i0mtmnB8n+ASFBcK9ys1EWQhNvZZ
 1JyQ==
X-Gm-Message-State: APjAAAUCDJxyESjyI1wJVlBphAjlkefPE1fpBxUFh7XAx/y0cSIvOPRZ
 L7z9lGV93OnMy5gdVHaJazvEFU/p
X-Google-Smtp-Source: APXvYqxzaAH++zrLj4oRkGVLSjUjhFfJRiNPIq/ZbE053Nh6YZtZBa+ZH/woQ3M4Brv9/53Shd3ubA==
X-Received: by 2002:a2e:7318:: with SMTP id o24mr22831096ljc.111.1568207740728; 
 Wed, 11 Sep 2019 06:15:40 -0700 (PDT)
Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96])
 by smtp.googlemail.com with ESMTPSA id l3sm5155923lfc.31.2019.09.11.06.15.38
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 11 Sep 2019 06:15:39 -0700 (PDT)
To: freebsd-current@FreeBSD.org, freebsd-stable@FreeBSD.org
From: Andriy Gapon <avg@FreeBSD.org>
Subject: ixv + RSS
Openpgp: preference=signencrypt
Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata=
 mQINBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7
 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l
 N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z
 AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i
 gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ
 /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4
 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8
 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS
 ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx
 rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABtB5BbmRyaXkgR2Fw
 b24gPGF2Z0BGcmVlQlNELm9yZz6JAlQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC
 WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV
 l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL
 rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO
 LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj
 GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/
 QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT
 eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/
 psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o
 JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R
 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H
 lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryLkCDQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW
 be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+
 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY
 EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO
 jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye
 syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m
 WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI
 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+
 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN
 MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L
 HgZY3zfKOqcAEQEAAYkCPAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM
 BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i
 ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs
 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR
 XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk
 LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4
 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym
 dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6
 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0
 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx
 NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp
Message-ID: <11438596-7ac3-e5c2-b963-6e0690e94265@FreeBSD.org>
Date: Wed, 11 Sep 2019 16:15:38 +0300
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101
 Thunderbird/60.8.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46T2TB6q1qz3Qyr
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates
 209.85.208.171 as permitted sender) smtp.mailfrom=agapon@gmail.com
X-Spamd-Result: default: False [-3.22 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[];
 FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
 MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[];
 DMARC_NA(0.00)[FreeBSD.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 RCVD_COUNT_THREE(0.00)[3];
 IP_SCORE(-1.22)[ip: (-0.47), ipnet: 209.85.128.0/17(-3.32), asn: 15169(-2.25),
 country: US(-0.05)]; RCPT_COUNT_TWO(0.00)[2];
 RCVD_IN_DNSWL_NONE(0.00)[171.208.85.209.list.dnswl.org : 127.0.5.0];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com];
 RWL_MAILSPIKE_POSSIBLE(0.00)[171.208.85.209.rep.mailspike.net : 127.0.0.17];
 R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
 FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com];
 MID_RHS_MATCH_FROM(0.00)[];
 RECEIVED_SPAMHAUS_PBL(0.00)[96.151.72.93.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net
 : 127.0.0.10]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 13:15:44 -0000


Has anyone ever tested a kernel with option RSS and an ixv interface?
Just looking for some data points.
Thanks!

-- 
Andriy Gapon

From owner-freebsd-current@freebsd.org  Wed Sep 11 14:31:33 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4A6ACD5F13
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Wed, 11 Sep 2019 14:31:33 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com
 [IPv6:2607:f8b0:4864:20::d36])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46T48h27Dqz41MN
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 14:31:32 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: by mail-io1-xd36.google.com with SMTP id r4so46266452iop.4
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 07:31:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:date:from:to:cc:subject:message-id:references:mime-version
 :content-disposition:in-reply-to:user-agent;
 bh=HY78RraKo+uBYn5dUskyEQdRVQaBrexZduCSjVPl7Pk=;
 b=EVH9196pfG6gN81SzWOYMNV2fpjfxOWVcizKAhad+tAl/tnnfMkHf6ARdj8rNjt6GZ
 q/ixw9o7wfZY8me8m/aF5PeFxkJm7Qhjh2D3WuUInkqzoDeGKgltI6bYF+rklk9DWkiW
 qTOFKJntN1zlOXr3dYWqTSgpTo8yqkcLrl86AANKvanpTOfxjaBleuS0s+k6TVBvUiFX
 ojVpEtbQztjfUOMidE4TZwzkymWPYETykHGafTPJ0EiXgf8Rz9cAyq6O3LEOeIw9+6RA
 pDFisGrsj4iWneZCLX5OBR4h7h/gmtJDiUbaRsWe+2FyeAL9BB8gg1/aprSRuly52KFo
 TIrw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
 :references:mime-version:content-disposition:in-reply-to:user-agent;
 bh=HY78RraKo+uBYn5dUskyEQdRVQaBrexZduCSjVPl7Pk=;
 b=dFpE4xWJon6cvQVXHS/EXsKbFVFQ759nN+tlIrbXvuy/QqBr/yPh4VFgtoHu/6wjjY
 oFiQjahnw5XCn7umjMxhLZGVk8vliRr5xY5WaMAiaVmWWQlVt2ELVX60Wm2xaKk7Jpya
 quix5SWvGXYmtLhC66xXwjwHc5WmOKGwYh8gCnXKieNlHHofavu6652Vi8BTCWE4Sqwb
 oRQE7hyVUkjSxedOVNbEN0hlGcvyRvrLU9GyHp+aWmeZcFpZguJFE10uuDFn6NctHMe6
 JfiLygdkh3rlqWH2fz4nQ7IuQddtpLDEjsgjC6BgTJi99t5nJfKCZ6lDuS7doCiM98tT
 bThg==
X-Gm-Message-State: APjAAAUkhgZRTb5Ntfj/OJNzDJGFfK/QOi8Tl8l/9oY/cYAwfrEbdKhN
 XFtVIb28CtD4eDC/wMm4uEA=
X-Google-Smtp-Source: APXvYqwRPFDKIRNDh8Ej3i3HFEnN1QnzympQ6jjcObqHecAeLwjWEumxq64UVwGj0IUCTScKCbahVA==
X-Received: by 2002:a05:6638:928:: with SMTP id
 8mr37214168jak.124.1568212290972; 
 Wed, 11 Sep 2019 07:31:30 -0700 (PDT)
Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca.
 [69.159.39.167])
 by smtp.gmail.com with ESMTPSA id c4sm16575099ioa.76.2019.09.11.07.31.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 11 Sep 2019 07:31:30 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Wed, 11 Sep 2019 10:31:25 -0400
From: Mark Johnston <markj@freebsd.org>
To: Mark Millard <marklmi@yahoo.com>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be
 allowed? (only 1 is, despite two numa domains)
Message-ID: <20190911143125.GA17992@raichu>
References: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Rspamd-Queue-Id: 46T48h27Dqz41MN
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=EVH9196p;
 dmarc=none;
 spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates
 2607:f8b0:4864:20::d36 as permitted sender) smtp.mailfrom=markjdb@gmail.com
X-Spamd-Result: default: False [-3.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c];
 RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2];
 FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com];
 FREEMAIL_TO(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+];
 IP_SCORE(-2.10)[ip: (-5.45), ipnet: 2607:f8b0::/32(-2.72), asn: 15169(-2.25),
 country: US(-0.05)]; FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com];
 SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[6.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 14:31:33 -0000

On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote:
> In a context with:
> 
> # cpuset -g
> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
> pid -1 domain policy: first-touch mask: 0, 1
> 
> I get:
> 
> # cpuset -l0 -n prefer:0 COMMAND
> cpuset: setdomain: Invalid argument
> 
> # cpuset -l0 -n prefer:2 COMMAND
> cpuset: setdomain: Invalid argument
> 
> But one prefer:? value does allow the COMMAND
> to run:
> 
> # cpuset -l0 -n prefer:1 COMMAND
> 
> This seem odd to me. Am I missing something?
> 
> For reference: I'm using a ThreadRipper 1950X
> with a head -r351227 based context for this
> activity. The above happens to have been run
> in a Windows 10 Pro HyperV session, instead
> of in a native-boot of the same media. (A
> native-boot would have had 32 CPUs.)

Can you please show the output of "sysctl vm.phys_segs" from this
setup?

From owner-freebsd-current@freebsd.org  Wed Sep 11 14:57:31 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4F4B5D6804
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Wed, 11 Sep 2019 14:57:31 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic310-21.consmr.mail.gq1.yahoo.com
 (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46T4kf1x5lz42Vq
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 14:57:29 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: RQgq3n8VM1lZeiY4pzg2EFgC1q1Jd7cvbMH3BSJfbpkB85rba3nYa5nTf8eCaJz
 Yscsdvua0WCzj6Nv0roRgPJ3BAcm2R3CnsXqMezgc4ei.ahBNhRd9heHPx.vX8V8vudO9ZFw1hYn
 1fBboZQlT6EIbtFNipmeArs2M38DXPhaPCt52B77_1POxikg78K.j3gPYyt3jVhMB.33DijBZsQh
 adojktyIRYvB_x8z5bJNSvAB4CiRh66D0FG.qbYa_pGQfvfTXhHOpALwj0_bvAlaiDgnCAmREeIF
 NM73qL4O8n7Bh7fzoi1s4KpIA36k4EF6zR5EAINX5WWrGrC2Wqt0nLwUPi8ygB9Hsqi3z1_efzAP
 2IpijJ3x215IG8F1jZWb.3J.4tpU8bqja837nXSmlIG_rkqG1ghDD0Ztmssbj946kk4vhntmrgcB
 7HXGfxwBo_l7hB8Rw2dmfwwIykQhHrinZ6vMiu1l10B3MW_GzehwxYOgATqAZiNWjJCxp1yq7IKs
 a.cVXyrKVh_BA8urIaBlh3cKNLt1NoEfYu4uftjcsqA.ISYHN_UqCrwPEJSWWNsiOA_3Xm9x6qN7
 Sf2.wi4z.1xnj813WLKEhcCfYFUsiJdRVqb91qrUW8cKZryHFfW87x_FdZ3sVc2PkHO.fCXpm1s6
 fObmNBEmvgfsB4lpISR2DYAb60hIw.Ml7Cnjb6XdWxOQPvQfW6akdpjsCj05SwCAoq_.eRpBh1zi
 UqYgphuhtU6n1R69fA.91uuOWzuG9tDMRo2MXXVbgth0BQY184XI8Hwn1PI0zW.gObRdMfHLDfVO
 UlgM3Xg0FNnst1NwQVz0pTuRT0xyKQ9a2_68sSBbiBSyVzgOVObtiDI9DwJP9opTEUrUaR9ZRfxx
 jJr2XJZ0FVl4wzlaJECXu78UEiaEG5KzV.xn0VnQgCjqW8vNodMo3WqHU9SxVqo45UG3nWM94qTO
 kARzAA2nHqxW2ocPBZxYOjxnzPaYqjQpKZNm2.uoYzqlhpwTHE0Sg7NrALWbT0.l.twZJFx83kka
 HPI9gGwLynBfVx_T9Ms6nKPt1EM2vWnhOttBfGyqscYP_N_k7bZsnYhliLQyM4PbcCUs59Tsk3YT
 LeDivAwCtlUoL596yJsqPiGUBXYAE53gqGuw2._Vk.FX7RVuKXGsQytuW8vQpDrjSjPIyJp.F1b3
 Yq1wnvY1nVf3MgiD1SKqgbrGy9rPDOm6AFrknoeKw0MPgWVCe7R5M6EkA6E_ryHpymSzJMtPkYNz
 n_gaVYeprl_eq.up45sMlFIp988UjY_9kGnRnatyb0gpMaX96Gn2wjmuUse4-
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic310.consmr.mail.gq1.yahoo.com with HTTP; Wed, 11 Sep 2019 14:57:28 +0000
Received: by smtp430.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID 40a50c2fca182dcd8f557fc7a02c9985; 
 Wed, 11 Sep 2019 14:57:27 +0000 (UTC)
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be
 allowed? (only 1 is, despite two numa domains)
From: Mark Millard <marklmi@yahoo.com>
In-Reply-To: <20190911143125.GA17992@raichu>
Date: Wed, 11 Sep 2019 07:57:26 -0700
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com>
References: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
 <20190911143125.GA17992@raichu>
To: Mark Johnston <markj@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
X-Rspamd-Queue-Id: 46T4kf1x5lz42Vq
X-Spamd-Bar: --
X-Spamd-Result: default: False [-2.14 / 15.00];
 R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[];
 TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 FROM_EQ_ENVFROM(0.00)[];
 IP_SCORE(0.00)[ip: (3.91), ipnet: 98.137.64.0/21(0.92), asn: 36647(0.74),
 country: US(-0.05)]; MIME_TRACE(0.00)[0:+];
 FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.85)[-0.851,0];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 NEURAL_HAM_LONG(-0.79)[-0.793,0]; MIME_GOOD(-0.10)[text/plain];
 RCVD_TLS_LAST(0.00)[]; IP_SCORE_FREEMAIL(0.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[147.69.137.98.list.dnswl.org : 127.0.5.0];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 14:57:31 -0000



On 2019-Sep-11, at 07:31, Mark Johnston <markj at freebsd.org> wrote:

> On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote:
>> In a context with:
>>=20
>> # cpuset -g
>> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, =
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
>> pid -1 domain policy: first-touch mask: 0, 1
>>=20
>> I get:
>>=20
>> # cpuset -l0 -n prefer:0 COMMAND
>> cpuset: setdomain: Invalid argument
>>=20
>> # cpuset -l0 -n prefer:2 COMMAND
>> cpuset: setdomain: Invalid argument
>>=20
>> But one prefer:? value does allow the COMMAND
>> to run:
>>=20
>> # cpuset -l0 -n prefer:1 COMMAND
>>=20
>> This seem odd to me. Am I missing something?
>>=20
>> For reference: I'm using a ThreadRipper 1950X
>> with a head -r351227 based context for this
>> activity. The above happens to have been run
>> in a Windows 10 Pro HyperV session, instead
>> of in a native-boot of the same media. (A
>> native-boot would have had 32 CPUs.)
>=20
> Can you please show the output of "sysctl vm.phys_segs" from this
> setup?

Sure:

# sysctl vm.phys_segs
vm.phys_segs:=20
SEGMENT 0:

start:     0x1000
end:       0x9f000
domain:    0
free list: 0xffffffff8281daa0

SEGMENT 1:

start:     0x103000
end:       0x1000000
domain:    0
free list: 0xffffffff8281daa0

SEGMENT 2:

start:     0x1000000
end:       0x2ee1000
domain:    0
free list: 0xffffffff8281d830

SEGMENT 3:

start:     0x2eea000
end:       0x2f23000
domain:    0
free list: 0xffffffff8281d830

SEGMENT 4:

start:     0x3000000
end:       0xf7ff0000
domain:    0
free list: 0xffffffff8281d830

SEGMENT 5:

start:     0x100002000
end:       0x9c5562000
domain:    0
free list: 0xffffffff8281d5c0

SEGMENT 6:

start:     0xa07c00000
end:       0xa07d50000
domain:    0
free list: 0xffffffff8281d5c0

SEGMENT 7:

start:     0xa08001000
end:       0xf9ee00000
domain:    1
free list: 0xffffffff8281dd10

SEGMENT 8:

start:     0x1000000000
end:       0x1427fe6000
domain:    1
free list: 0xffffffff8281dd10


And confirming the oddity is still the case
(I'd rebooted since the report):

# cpuset -g
pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, =
17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
pid -1 domain policy: first-touch mask: 0, 1

# cpuset -l0 -n prefer:0 COMMAND
cpuset: setdomain: Invalid argument

# cpuset -l0 -n prefer:2 COMMAND
cpuset: setdomain: Invalid argument

# cpuset -l0 -n prefer:1 COMMAND
cpuset: COMMAND: No such file or directory


=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)


From owner-freebsd-current@freebsd.org  Wed Sep 11 15:15:18 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 57A16D6FEC
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Wed, 11 Sep 2019 15:15:18 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com
 [IPv6:2607:f8b0:4864:20::d34])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46T5791dVHz43M4
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 15:15:16 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: by mail-io1-xd34.google.com with SMTP id f4so45935979ion.2
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 08:15:16 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:date:from:to:cc:subject:message-id:references:mime-version
 :content-disposition:in-reply-to:user-agent;
 bh=gz3AXwftLEOIeRZOrHwHQgPzlZX9AlkUhYF7P1FtR/g=;
 b=RsketEWgKSLyOwOOiksc6zGYnHn0COjpulD+GP8qi31ISajuvnpB2yekTrJAn/jMDf
 hVkHwMnF3JQyTM/Xent6MXLow3UntpBMmh7PLNnNsVAcvUskif6+8mbyEBFUncRkH6/K
 QOUeG+SjwukokJO6y4s++PLv97BfxZNO6IH/HJ/n0gAuYdNFm6M2HeEsO7oN5jE2jx+I
 p1aFSSjaCFPj97FQDSuM3wUxnZ43C6mo1+hPCiq3kw0XCe7kv6jHP5ks+NUCgY5zZdWq
 MrvvZ8QUvkjvMx0GuOOc7SfnWuuUGjy1KSWeRLUDPFOowHrFuPMRXonuOHaC25luEfS9
 ortw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
 :references:mime-version:content-disposition:in-reply-to:user-agent;
 bh=gz3AXwftLEOIeRZOrHwHQgPzlZX9AlkUhYF7P1FtR/g=;
 b=drXRN+8rl/PC3ltc9K5CjqgamE4+oVxct884AWuLsepTKhbziPG2RkOgsvTmhA7LOb
 0AiyCoMuv0NUe2BeSBJaekboLfmlOkNgQIu0aoPqChmxd9frEzuOAY4OZrTDTGA0FS83
 YeKfREP1FiTRzgl0CCO28SQ+qU5HlKP9rS2dJosPrga8MlPnrsBow9mpomIekEsmm6Lp
 nBi1sGTKX/O1yeubnGMoq17L5liP5eIsAG8OzckMZi1LITX0xirwXFEDJV/1Q3WliieT
 JV2Iz0uKdkmPZ8k13BUD3O0ybg5tRw2YKMO5cxrutUO14v/xT7EZOyrMhpw3ysTizqnn
 qEOQ==
X-Gm-Message-State: APjAAAVKRkSg9xnG2QaSejYmuhSv4ZCAGcfmLBtQcmsN2uIoWbgYSaJS
 tfTZl7c1E/Vw6ld5clOzbIVITK6C
X-Google-Smtp-Source: APXvYqypTQhylPboRqz13W/wSraaCAB0se7lzKOMf6iizx4Dzfy6Kw1Div7QJHOAAQvkixyMruZpLw==
X-Received: by 2002:a02:6601:: with SMTP id k1mr39161122jac.47.1568214915680; 
 Wed, 11 Sep 2019 08:15:15 -0700 (PDT)
Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca.
 [69.159.39.167])
 by smtp.gmail.com with ESMTPSA id f7sm16080128ioc.31.2019.09.11.08.15.14
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 11 Sep 2019 08:15:14 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Wed, 11 Sep 2019 11:15:12 -0400
From: Mark Johnston <markj@freebsd.org>
To: Mark Millard <marklmi@yahoo.com>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be
 allowed? (only 1 is, despite two numa domains)
Message-ID: <20190911151512.GB17992@raichu>
References: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
 <20190911143125.GA17992@raichu>
 <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Rspamd-Queue-Id: 46T5791dVHz43M4
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=RsketEWg;
 dmarc=none;
 spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates
 2607:f8b0:4864:20::d34 as permitted sender) smtp.mailfrom=markjdb@gmail.com
X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c];
 RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2];
 FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com];
 FREEMAIL_TO(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+];
 IP_SCORE(-1.80)[ip: (-3.95), ipnet: 2607:f8b0::/32(-2.72), asn: 15169(-2.25),
 country: US(-0.05)]; FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com];
 SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[4.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 15:15:18 -0000

On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote:
> 
> 
> On 2019-Sep-11, at 07:31, Mark Johnston <markj at freebsd.org> wrote:
> 
> > On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote:
> >> In a context with:
> >> 
> >> # cpuset -g
> >> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
> >> pid -1 domain policy: first-touch mask: 0, 1
> >> 
> >> I get:
> >> 
> >> # cpuset -l0 -n prefer:0 COMMAND
> >> cpuset: setdomain: Invalid argument
> >> 
> >> # cpuset -l0 -n prefer:2 COMMAND
> >> cpuset: setdomain: Invalid argument
> >> 
> >> But one prefer:? value does allow the COMMAND
> >> to run:
> >> 
> >> # cpuset -l0 -n prefer:1 COMMAND
> >> 
> >> This seem odd to me. Am I missing something?
> >> 
> >> For reference: I'm using a ThreadRipper 1950X
> >> with a head -r351227 based context for this
> >> activity. The above happens to have been run
> >> in a Windows 10 Pro HyperV session, instead
> >> of in a native-boot of the same media. (A
> >> native-boot would have had 32 CPUs.)
> > 
> > Can you please show the output of "sysctl vm.phys_segs" from this
> > setup?
> 
> Sure:

I was wondering if you had only one domain populated, but it seems not
to be the case.  Could you try updating to r351672 or later and see if
the behaviour persists?

From owner-freebsd-current@freebsd.org  Wed Sep 11 17:11:52 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id A08F1DA134
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Wed, 11 Sep 2019 17:11:52 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic306-2.consmr.mail.bf2.yahoo.com
 (sonic306-2.consmr.mail.bf2.yahoo.com [74.6.132.41])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46T7jg6B0vz4BHT
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 17:11:51 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: .Curh5wVM1la4tlyPFhbOv2vzZbxo9MFdbsLAT0MwfcRQSYHaVSjSdGjEC515nc
 6vWJB5Xeflzk27VRPCgNaPG8aH352tSMTL3OWydjdCZc2LAAHVPr2NHUyCDUq0g5o72uJHXRL4lX
 S33aJZob3GgS_.RCUNk6L0rG8QvWe9y4J7lFj1Recj_tn.43nuOlgmz1xrE0YtSabrd3ocMCHeSC
 ZtmESgYxB1ZtjHkimSZOq05aVSXNH5hASUJOVDYATVJeiojnB49itF68.k0h2QFw_EpD5FJDxCAC
 wkV3MJoHKkdpyaEhRxr7vJl2KwWN6mT2kM7XvAm1ct11zbbBiugIP31Nx.mHIR.YtxMIoXo35ZA8
 qOnTw1Un7ZhxCfNhH_hAFQTs8851HasL8I4pfsDRVtK6q5F9lUjvps5VjtSLpTKTZkM2ZaGTmfBJ
 VxdAgF6TucKTvv302R37bEAGgyOnHgEf9bGlzkHvgJnkJV.6e1Azs3Qav0y8yUWHZqiMW34bsq8L
 Y8YC7StIsl1jLzynKmwdTgDpchvQ2MJvY49sglz84uYKInYKufKk1iRATPzKD6.nKEHucxl_fcWC
 sFlo3F3MWA6LK.zIdv.wV3202bfkYj9HgkGy4FfO0tzCZIqal1JWca32QBqzAU7vIMzTIMcUB5fr
 I73z_jbune5S8mAcHTXKnSm6Jx3w1T.5Uz0ZFyJyE9Ykx.OC.gsqUkhzmvK2WMSJk8x7.suCtmjW
 DjlIiRtxG0P9pmee5pREMuPDfLLsXwpZV8kgrbxp_K3Qec6NSi9Nh.LujTbJ_88EX0woBa7Ui7GV
 Q.SFDFNmUQkJkva8tNKcw2ijx6Vh5Mi4n3RJ2yq26h95qQO1cE6RueDpDGXEDWvlseOOYRCUzzIM
 gEVyJqT6tS1xtPhjJEa7hA15xYnfPColOgmQwPoGBR2.mLSZ_D2bKiRwS9uPki1dWEGF2F2luCZe
 RLi6_PRan3FmSY4zp1txLVpWp85olci3MlWkOvJTEh_VIpf7Xj2pEaiYZXmSK4wmw.ff_3k9JCb7
 OYhV.CttP7C9GxjK1_gWbIG1laKPpcrgUwPGQkHMmlGh0CezNo2ADVuJ_3DzYakpONe9_er.7_5V
 MGBt7_DvJG9xeaXDioENu3mUUIzxUcGw.zXf12qmFvmYSfrLQfzfZV1ywCM7tijYRNxdkrQKo3t_
 UyPPW0SeTVR1di.3xlteJYPTd.RqX4_hMO_aMcRMn_LmJh4Y9UfjgzXDjKq6nC3ngzsFZpk7VnNk
 8CEOXhbAwtbyOSK4cGrmPvBaQ6PdtKkyvwXbn8cTE5ZFDKdu6uH9PMbq7NYLS0Q--
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic306.consmr.mail.bf2.yahoo.com with HTTP; Wed, 11 Sep 2019 17:11:50 +0000
Received: by smtp406.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID c844bdba0cb144dafb0c5a2a48b13cca; 
 Wed, 11 Sep 2019 17:11:47 +0000 (UTC)
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be
 allowed? (only 1 is, despite two numa domains)
From: Mark Millard <marklmi@yahoo.com>
In-Reply-To: <20190911151512.GB17992@raichu>
Date: Wed, 11 Sep 2019 10:11:45 -0700
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <3CE4AEB7-E32C-49BD-8C75-71AB8739BAEC@yahoo.com>
References: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
 <20190911143125.GA17992@raichu>
 <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com>
 <20190911151512.GB17992@raichu>
To: Mark Johnston <markj@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
X-Rspamd-Queue-Id: 46T7jg6B0vz4BHT
X-Spamd-Bar: -
X-Spamd-Result: default: False [-1.82 / 15.00];
 R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[];
 TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 FROM_EQ_ENVFROM(0.00)[];
 IP_SCORE(0.00)[ip: (3.75), ipnet: 74.6.128.0/21(1.45), asn: 26101(1.16),
 country: US(-0.05)]; MIME_TRACE(0.00)[0:+];
 FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.74)[-0.739,0];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 NEURAL_HAM_LONG(-0.58)[-0.583,0]; MIME_GOOD(-0.10)[text/plain];
 RCVD_TLS_LAST(0.00)[]; IP_SCORE_FREEMAIL(0.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[41.132.6.74.list.dnswl.org : 127.0.5.0];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 17:11:52 -0000



On 2019-Sep-11, at 08:15, Mark Johnston <markj at freebsd.org> wrote:

> On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote:
>>=20
>>=20
>> On 2019-Sep-11, at 07:31, Mark Johnston <markj at freebsd.org> wrote:
>>=20
>>> On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote:
>>>> In a context with:
>>>>=20
>>>> # cpuset -g
>>>> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, =
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
>>>> pid -1 domain policy: first-touch mask: 0, 1
>>>>=20
>>>> I get:
>>>>=20
>>>> # cpuset -l0 -n prefer:0 COMMAND
>>>> cpuset: setdomain: Invalid argument
>>>>=20
>>>> # cpuset -l0 -n prefer:2 COMMAND
>>>> cpuset: setdomain: Invalid argument
>>>>=20
>>>> But one prefer:? value does allow the COMMAND
>>>> to run:
>>>>=20
>>>> # cpuset -l0 -n prefer:1 COMMAND
>>>>=20
>>>> This seem odd to me. Am I missing something?
>>>>=20
>>>> For reference: I'm using a ThreadRipper 1950X
>>>> with a head -r351227 based context for this
>>>> activity. The above happens to have been run
>>>> in a Windows 10 Pro HyperV session, instead
>>>> of in a native-boot of the same media. (A
>>>> native-boot would have had 32 CPUs.)
>>>=20
>>> Can you please show the output of "sysctl vm.phys_segs" from this
>>> setup?
>>=20
>> Sure:
>=20
> I was wondering if you had only one domain populated, but it seems not
> to be the case.  Could you try updating to r351672 or later and see if
> the behaviour persists?

It may be a bit before I do that.

FYI: I had set MAXMEMDOM to match the number of
actual domains for the context:

/usr/src/sys/amd64/conf/GENERIC-DBG:options     MAXMEMDOM=3D2
/usr/src/sys/amd64/conf/GENERIC-NODBG:options   MAXMEMDOM=3D2

(These kernel configuration files include GENERIC.)

=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)


From owner-freebsd-current@freebsd.org  Wed Sep 11 18:14:51 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 35ED5DB6F4
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Wed, 11 Sep 2019 18:14:51 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic308-9.consmr.mail.ne1.yahoo.com
 (sonic308-9.consmr.mail.ne1.yahoo.com [66.163.187.32])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46T96L2mNlz4FYW
 for <freebsd-current@freebsd.org>; Wed, 11 Sep 2019 18:14:50 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: VDTiFN8VM1nza.XakPnpb2SIZP1vQILOgi6Zfj06XgE3jc0b3rDDr7j5kQoAUKc
 M46R6UTTtJwEix_vrou9byobso_tVYKtHDF3xUhIF52XIbhp4nPk_rVG9iUwLqlDGF3rlWYCnjPQ
 khrD7cK1J2_7goy8ZpOh9iJ_4kRFXg6HM187qyAIPE.76LYlO64MqdBvNVKCOpkplS_ypwr13ylN
 J34UrGzqps1ZPGsxgeR3JeRgZ.VIaNG5ZI_u6mU1WPaDoqMnlCY4a0Qkd5k7ma9zHd3Sdhk9Kdwm
 KCZQqgk77HwgYesxxynpM3EJsnhRlDxEGXEX.WKreQIgidvs8jNkG.87DiS9iQKuyTvU7yW9VAo1
 fAVYcQOCyTgNKg5f1EIbnSJsTPUwHu3j8_gXkuWmpBrJ6MI69DBuREQE.NUgRO2GEQ5h0UwY9LvM
 NZRCRrrZVsn0n.vdqMZM_ANnLPcxXpYSJ5J0KOIg1VbMUf9kGYIn9IcHm33C4eg8NmjgIY0sFub1
 gHjn_7rbWBWAWdIrmf0Of6qjFifrbZziFSSvKKpP5b8Stc8sjhmHkV.FgcBRCLcX9hSgq0UD_tLh
 uoSLETf8BIEUF8.yBPu2MSKrhPDpcQ1DsrQaWACwu9RS60eyPCieCsxwA.OOMhnpvqS4SX.BgMF6
 lX9XgmOCWXmdy88G7k.QMUkf0aX5xex_bWvAHarSmQHnpU6MddemBPxMhQTyGWBTaNaNkDuZgJti
 _qr5l3Lvz66TffilCVwaKdVNg5t2ebJMHYAY8bQScUXHx5yHIYy.6YxFI1tgqkEKyqVK2nLtYK1g
 25eqHLbNjUtWf2EKqjDClU2SgwsnWZe3G8IhqaLKZNa7B3GQNDyptU7m0cxcqWeQ5nEuV4cNyfzA
 lHD4UyEdUdlxipqyMjGrGnzZ1ifdnKP.PE22.pIajrKV1haexoJiOg2HISorEF3w9ayyfibSqiaO
 YGWXSBMlsa49rQghBisTlS3eIYOApqekv4GJYzCwcjgGY77ZDwNFea93JqWMkykVMApjOkepiXgp
 teKQXba7SbzZzVvGX2BpKBcxubqrGjc6TMBm7RLoX8NoCGBZ1o.G_6YauJq7ukk_FKubJZDosqFQ
 LOUAZsmszWOls0Y7.lTSwuD5ibZ8kjXLMXHfwAJSRjLu0oNqxTgo3uEEpuBYRLaXv7eIoVMSIDsY
 N.CIfLVO8fxVnn6tSLiGZsRz08KQgTgddplwhgrcAfsMFJUQZ3WIT5IefZuO6YeieIB8wpxf1J35
 uVj6tECKMmDYd0Cr7qOBSVHOQ1RUUo8PBMf4D1ZNFyL.D3Ff8BjMzuq1a0fZAIGcyYE6R
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic308.consmr.mail.ne1.yahoo.com with HTTP; Wed, 11 Sep 2019 18:14:48 +0000
Received: by smtp430.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID df8eb85ede1bb96da59f2499f775d362; 
 Wed, 11 Sep 2019 18:14:44 +0000 (UTC)
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be
 allowed? (only 1 is, despite two numa domains)
From: Mark Millard <marklmi@yahoo.com>
In-Reply-To: <3CE4AEB7-E32C-49BD-8C75-71AB8739BAEC@yahoo.com>
Date: Wed, 11 Sep 2019 11:14:42 -0700
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <54F53CA8-6BEC-4B31-9662-C6854CDE0A08@yahoo.com>
References: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
 <20190911143125.GA17992@raichu>
 <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com>
 <20190911151512.GB17992@raichu>
 <3CE4AEB7-E32C-49BD-8C75-71AB8739BAEC@yahoo.com>
To: Mark Johnston <markj@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
X-Rspamd-Queue-Id: 46T96L2mNlz4FYW
X-Spamd-Bar: -
X-Spamd-Result: default: False [-1.64 / 15.00];
 R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[];
 TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 FROM_EQ_ENVFROM(0.00)[];
 IP_SCORE(0.00)[ip: (4.65), ipnet: 66.163.184.0/21(1.30), asn: 36646(1.04),
 country: US(-0.05)]; MIME_TRACE(0.00)[0:+];
 FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.69)[-0.687,0];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 NEURAL_HAM_LONG(-0.46)[-0.458,0]; MIME_GOOD(-0.10)[text/plain];
 RCVD_TLS_LAST(0.00)[]; IP_SCORE_FREEMAIL(0.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[32.187.163.66.list.dnswl.org : 127.0.5.0];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 18:14:51 -0000



On 2019-Sep-11, at 10:11, Mark Millard <marklmi at yahoo.com> wrote:



> On 2019-Sep-11, at 08:15, Mark Johnston <markj at freebsd.org> wrote:
>=20
>> On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote:
>>>=20
>>>=20
>>> On 2019-Sep-11, at 07:31, Mark Johnston <markj at freebsd.org> =
wrote:
>>>=20
>>>> On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote:
>>>>> In a context with:
>>>>>=20
>>>>> # cpuset -g
>>>>> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, =
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
>>>>> pid -1 domain policy: first-touch mask: 0, 1
>>>>>=20
>>>>> I get:
>>>>>=20
>>>>> # cpuset -l0 -n prefer:0 COMMAND
>>>>> cpuset: setdomain: Invalid argument
>>>>>=20
>>>>> # cpuset -l0 -n prefer:2 COMMAND
>>>>> cpuset: setdomain: Invalid argument
>>>>>=20
>>>>> But one prefer:? value does allow the COMMAND
>>>>> to run:
>>>>>=20
>>>>> # cpuset -l0 -n prefer:1 COMMAND
>>>>>=20
>>>>> This seem odd to me. Am I missing something?
>>>>>=20
>>>>> For reference: I'm using a ThreadRipper 1950X
>>>>> with a head -r351227 based context for this
>>>>> activity. The above happens to have been run
>>>>> in a Windows 10 Pro HyperV session, instead
>>>>> of in a native-boot of the same media. (A
>>>>> native-boot would have had 32 CPUs.)
>>>>=20
>>>> Can you please show the output of "sysctl vm.phys_segs" from this
>>>> setup?
>>>=20
>>> Sure:
>>=20
>> I was wondering if you had only one domain populated, but it seems =
not
>> to be the case.  Could you try updating to r351672 or later and see =
if
>> the behaviour persists?
>=20
> It may be a bit before I do that.
>=20
> FYI: I had set MAXMEMDOM to match the number of
> actual domains for the context:
>=20
> /usr/src/sys/amd64/conf/GENERIC-DBG:options     MAXMEMDOM=3D2
> /usr/src/sys/amd64/conf/GENERIC-NODBG:options   MAXMEMDOM=3D2
>=20
> (These kernel configuration files include GENERIC.)

Not that the below is the problem that I reported, but
cpuset_modify_domain has an oddity. In the below, note
the "root->" use followed by the "root &&" test: the
root-> use would have failed first. Should the && be
"dset &&" instead? Should "root &&" just be removed for
being redundant?

793	                root =3D cpuset_getroot(set);
794	                mtx_lock_spin(&cpuset_lock);
795	                dset =3D root->cs_domain;
796	                /*
797	                 * Verify that we have access to this set of =
domains.
798	                 */
799	                if (root && !domainset_valid(dset, domain)) {
800	                        error =3D EINVAL;
801	                        goto out;
802	                }

=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)


From owner-freebsd-current@freebsd.org  Wed Sep 11 18:26:46 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 22EA8DC08F;
 Wed, 11 Sep 2019 18:26:46 +0000 (UTC)
 (envelope-from lwhsu@freebsd.org)
Received: from freefall.freebsd.org (freefall.freebsd.org
 [IPv6:2610:1c1:1:6074::16:84])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (4096 bits) client-digest SHA256)
 (Client CN "freefall.freebsd.org",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46T9N607WRz4GV2;
 Wed, 11 Sep 2019 18:26:46 +0000 (UTC)
 (envelope-from lwhsu@freebsd.org)
Received: by freefall.freebsd.org (Postfix, from userid 1129)
 id D7B3712E65; Wed, 11 Sep 2019 18:26:45 +0000 (UTC)
Date: Wed, 11 Sep 2019 18:26:45 +0000
From: Li-Wen Hsu <lwhsu@freebsd.org>
To: freebsd-testing@freebsd.org
Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org
Subject: FreeBSD CI Weekly Report 2019-09-08
Message-ID: <20190911182645.GA64832@freefall.freebsd.org>
Reply-To: freebsd-testing@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.11.4 (2019-03-13)
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Sep 2019 18:26:46 -0000

(Please send the followup to freebsd-testing@ and note Reply-To is set.)

FreeBSD CI Weekly Report 2019-09-08
===================================

Here is a summary of the FreeBSD Continuous Integration results for the period
from 2019-09-02 to 2019-09-08.

During this period, we have:

* 2122 builds (99.4% (+2) passed, 0.6% (-2) failed) were executed on
  aarch64, amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64,
  powerpcspe, riscv64, sparc64 architectures for head, stable/12, stable/11
  branches.
* 373 test runs (59.6% (-0.2) passed, 39.1% (-1.1) unstable, 1.3% (+1.3)
  exception) were executed on amd64, i386, riscv64 architectures for head,
  stable/12, stable/11 branches.
* 50 doc builds (100% (+2) passed)

Test case status (on 2019-09-08 23:59):
| Branch/Architecture | Total      | Pass       | Fail      | Skipped  |
| ------------------- | ---------- | ---------- | --------- | -------- |
| head/amd64          | 7556 (+6)  | 7495 (+11) | 0 (-1)    | 61 (-4)  |
| head/i386           | 7554 (+6)  | 7484 (+7)  | 2 (0)     | 68 (-1)  |
| 12-STABLE/amd64     | 7393 (+1)  | 7341 (-7)  | 11 (+11)  | 41 (-3)  |
| 12-STABLE/i386      | 7391 (+1)  | 7329 (-10) | 11 (+11)  | 51 (0)   |
| 11-STABLE/amd64     | 6846 (+1)  | 6802 (+4)  | 0 (0)     | 44 (-3)  |
| 11-STABLE/i386      | 6844 (+1)  | 6767 (-7)  | 34 (0)    | 43 (-6)  |

(The statistics from experimental jobs are omitted)

If any of the issues found by CI are in your area of interest or expertise
please investigate the PRs listed below.

The latest web version of this report is available at
https://hackmd.io/@FreeBSD-CI/report-20190908 and archive is available at
https://hackmd.io/@FreeBSD-CI/, any help is welcome.

## News

* Weekly CI report archive has been moved to https://hackmd.io/@FreeBSD-CI

* [FCP 20190401-ci_policy: CI policy](https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md)
  is in "feedback" state, please check and provide comments on -fcp@ and -hackers@ mailing lists.

## Fixed Tests

* https://ci.freebsd.org/job/FreeBSD-head-amd64-test/
    * (flakey) usr.bin.procstat.procstat_test.environment
    * (flakey) usr.bin.procstat.procstat_test.command_line_arguments
      Fixed in https://svnweb.freebsd.org/changeset/base/351819

## Failing Tests

* https://ci.freebsd.org/job/FreeBSD-stable-12-amd64-test/
* https://ci.freebsd.org/job/FreeBSD-stable-12-i386-test/
    * sys.netipsec.tunnel.aes_cbc_128_hmac_sha1.v6
    * sys.netipsec.tunnel.aes_cbc_256_hmac_sha2_256.v6
    * sys.netipsec.tunnel.aes_gcm_128.v6
    * sys.netipsec.tunnel.aes_gcm_256.v6
    * sys.netipsec.tunnel.aesni_aes_cbc_128_hmac_sha1.v6
    * sys.netipsec.tunnel.aesni_aes_cbc_256_hmac_sha2_256.v6
    * sys.netipsec.tunnel.aesni_aes_gcm_128.v6
    * sys.netipsec.tunnel.aesni_aes_gcm_256.v6
    * sys.netipsec.tunnel.empty.v6
    * sys.netpfil.pf.fragmentation.v6
    * sys.netpfil.pf.pass_block.v6
    * https://bugs.freebsd.org/240511

* https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/
    * local.kyua.* (31 cases)
    * local.lutok.* (3 cases)

## Failing and Flaky Tests (from experimental jobs)

* https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/
    * cddl.usr.sbin.dtrace.common.misc.t_dtrace_contrib.tst_dynopt_d
        * https://bugs.freebsd.org/237641
    * cddl.usr.sbin.dtrace.amd64.arrays.t_dtrace_contrib.tst_uregsarray_d
        * https://bugs.freebsd.org/240358

* https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/
    * There are ~60 failing cases, including flakey ones, see https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/ for more details

## Disabled Tests

* lib.libc.sys.mmap_test.mmap_truncate_signal
  https://bugs.freebsd.org/211924
  Patch available: https://reviews.freebsd.org/D21566
* sys.fs.tmpfs.mount_test.large
  https://bugs.freebsd.org/212862
* sys.fs.tmpfs.link_test.kqueue
  https://bugs.freebsd.org/213662
* sys.kqueue.libkqueue.kqueue_test.main
  https://bugs.freebsd.org/233586
* sys.kern.ptrace_test.ptrace__PT_KILL_competing_stop
  https://bugs.freebsd.org/220841
* lib.libc.regex.exhaust_test.regcomp_too_big (i386 only)
  https://bugs.freebsd.org/237450
* sys.netinet.socket_afinet.socket_afinet_bind_zero (new)
  https://bugs.freebsd.org/238781
* sys.netpfil.pf.names.names
* sys.netpfil.pf.synproxy.synproxy
  https://bugs.freebsd.org/238870
* sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger 
  https://bugs.freebsd.org/239292
* sys.netpfil.pf.forward.v4 (i386 only)
* sys.netpfil.pf.forward.v6 (i386 only)
* sys.netpfil.pf.set_tos.v4 (i386 only)
  https://bugs.freebsd.org/239380 (updating net/scapy to 2.4.3 may fix this)
* sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger	
  https://bugs.freebsd.org/239397
* sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger	
  https://bugs.freebsd.org/239399
* sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger
  https://bugs.freebsd.org/239425
* lib.libc.gen.getmntinfo_test.getmntinfo_test
  https://bugs.freebsd.org/240049
* sys.sys.qmath_test.qdivq_s64q
  https://bugs.freebsd.org/240219

## Issues

### Cause build fails
* https://bugs.freebsd.org/233735
  Possible build race: genoffset.o /usr/src/sys/sys/types.h: error: machine/endian.h: No such file or directory
* https://bugs.freebsd.org/233769
  Possible build race: ld: error: unable to find library -lgcc_s

### Cause kernel panics
* https://bugs.freebsd.org/238870
  sys.netpfil.pf.names.names and sys.netpfil.pf.synproxy.synproxy cause panic
  Patch exists:
    * https://reviews.freebsd.org/D20868
    * https://reviews.freebsd.org/D20869

### Open
* https://bugs.freebsd.org/237403
  Tests in sys/opencrypto should be converted to Python3
* https://bugs.freebsd.org/237641
  Flakey test case: common.misc.t_dtrace_contrib.tst_dynopt_d
* https://bugs.freebsd.org/237656
  "Freed UMA keg (rtentry) was not empty (18 items). Lost 1 pages of memory." seen when running sys/netipsec tests
* https://bugs.freebsd.org/238781
  sys.netinet.socket_afinet.socket_afinet_bind_zero does not work when mac_portacl(4) loaded
* https://bugs.freebsd.org/239292
  Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger 
* https://bugs.freebsd.org/239380
  sys.netpfil.pf.forward.{v4,v6} and sys.netpfil.pf.set_tos.v4 fail on i386
* https://bugs.freebsd.org/239397
  Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger
* https://bugs.freebsd.org/239399
  Flakey test case: sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger
* https://bugs.freebsd.org/239425
  Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger
* https://bugs.freebsd.org/240085
  Failing test: sys.netpfil.common.forward.pf_v4 on i386
* https://bugs.freebsd.org/240086
  Failing test: sys.netpfil.common.tos.pf_tos on i386

### Others

* [Tickets related to testing@](https://preview.tinyurl.com/y9maauwg)


From owner-freebsd-current@freebsd.org  Thu Sep 12 04:40:14 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 17787EA8E5
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Thu, 12 Sep 2019 04:40:14 +0000 (UTC)
 (envelope-from ohartmann@walstatt.org)
Received: from mout.gmx.net (mout.gmx.net [212.227.15.19])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client CN "mout.gmx.net",
 Issuer "TeleSec ServerPass Class 2 CA" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46TQzw4mfcz3Jyx
 for <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 04:40:12 +0000 (UTC)
 (envelope-from ohartmann@walstatt.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net;
 s=badeba3b8450; t=1568263201;
 bh=EX4hJECsDx6GLN9rGvl97z+dh9WIHCf4bA1SYWHtrio=;
 h=X-UI-Sender-Class:Date:From:To:Subject;
 b=U/ofKeyKTbc++9CYT4RZ0KWW9iXb/zqYHT7RykFrbvaG/U8MBXbX989oZYv2PgmUD
 vcntQBDqqkHIRZcMAoBWOO7vOR9ZPpx/C1YHX+4M2H4teHPR9P3ALBrFY6rbLaprwW
 OZPrNau4lVeeSmlL3f5zARF7JgIRVTOAYk3Zpw1Q=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from freyja ([46.88.81.15]) by mail.gmx.com (mrgmx002
 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MPlsg-1i4RCr20o6-0055GI for
 <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 06:27:07 +0200
Date: Thu, 12 Sep 2019 06:27:00 +0200
From: "O. Hartmann" <ohartmann@walstatt.org>
To: freebsd-current <freebsd-current@freebsd.org>
Subject: r352239: install failure: make[10]: exec(btxld) failed (No such
 file or directory)
Message-ID: <20190912062656.7afa3816@freyja>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Provags-ID: V03:K1:vupM7kRy1J4A8HBijF5v+7Tp2BMiHLbPNDCYcpbB4f5DANRKr5B
 CbERc9Fnr7QFJmuSKneKHIYMUKa+Wb5L+8bf/YveN+n9QXgFe88fY2YS7YQ5cHfv83H66ZY
 hsqErgeifJFh/b/6I299hBl3M3OdP3WNzsrOSWDncnscFjGZqDJuoXg5hpWLAYzDMRuE+XM
 dihAIjHSfI4PQTA+0kK/Q==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V03:K0:V/Xs9PJPaMA=:ijeGU6bsFiE2mSZH2bah77
 YNLYX9RbPUEZSNRY+M4MmNMDaUAcoqrZnRsVINwJHPWjF42kjlD67ZF/1GyN3C8eqtZmH+BlW
 p9aQTlhUmuIEX5dotZFkwnWCYsv/JxPqPCt6VbaoueSdibSJ7JgAov85blgGBox3n9atbgklB
 fSmvPqCbGrhL+cf8FIc0J+TzfOKaeCWZJeu7pQBWl+nUk2Ud6N+oblbNq/Iha/QzePdumFORI
 vp22lksmsuetYj69Eh5krioxaMKLGhy10kVcj2pzOEgH3tstDUp8sCzYPNfI0Yp7YPshCFc4c
 UJ/7oAbaKdPmE6IpsyKcXxxb75QjZyyvJUdjdN5BTihlgTw4zPUcVpVVjj9DKu6RygtnKUn6p
 tHH8zXQd4IySDpdSYZxkWOUvlnPwM8dVoW4p+bzio3VZVhChIJsMo5y+kBTBBSdnKjxtSoWz9
 f5vhELb0a6mPMYDwBO2tSG3hxOdQ04pYpiMmRFceqlKG2zQ5hY6hhyuO/WLeCSolKfJhO9nrq
 c6eCia4AMzbKxuoL2ZdAZBkuWBrp+j6oqQwmd55hcFLwKF/8XKxtNNuWYCRBxmUpxrA6pjS1w
 Gw4wEKsidOICBArV2OOXNPtXTi7ax+q0jqQlcr9R5jn8eb1ee5oz1ZM9LKcTX/8LrEqd18CtI
 CX3Btw+gBF3nC+6PG9W0fEOphzqE7U5a25SUvcEYc6F2oNpXJqfHS71ojMsxj3DJFZ8xzKFRg
 AablgItid84dHZBQzPaYJKz2pWkP5wZl9G3G5g9a7I+WTf4ZHraXC6GEBMGC4T+syFQahaSY3
 15hS7WvrD5uD5MWsk2PUwDYc47yIv+12fsr/Rvjkjj/o5wHG6TXnFvdus4bDrT3IAehipAqGx
 fpRVcOacG7/Q6bo/XDeZEl5hGPbMVKLKRHX1QYJNAxGzyBfPk1HGQ0IqYYGupBQd2UHU2XQsL
 2/gA0niBzZyLXXCPaptZkXrXKpR41b26EHqIqjxrLTAJRD23Z7fxlaXP4AKa+S3PM1eYvsjQA
 BDNZyvmWyZJSsKh+AicqeMC+V325x/ljcribHlrm7YQ2V3N8AYvFA6ypK3UzxV/Cwey6lYu8E
 MAPAo5DoUmu4agDz147En3b6t5+Uoy6QKI3AFpQLDnSO0JQ1UFdBCf3xFqrvfrUU88VX+aomM
 XqItO9aWNFCa8jQ89ftCbtwrXzJ27exf2+Outht+nppqEjGlYNdS4rLr2zvJr2E2D63Sz+gOq
 lMYoxvlJk7woaMEoD
X-Rspamd-Queue-Id: 46TQzw4mfcz3Jyx
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=U/ofKeyK;
 dmarc=none;
 spf=none (mx1.freebsd.org: domain of ohartmann@walstatt.org has no SPF policy
 when checking 212.227.15.19) smtp.mailfrom=ohartmann@walstatt.org
X-Spamd-Result: default: False [-2.92 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[];
 R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
 TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[walstatt.org]; RCPT_COUNT_ONE(0.00)[1];
 IP_SCORE(-1.12)[ip: (-6.38), ipnet: 212.227.0.0/16(-1.37), asn: 8560(2.14),
 country: DE(-0.01)]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[gmx.net:+]; R_SPF_NA(0.00)[];
 FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE];
 RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[];
 RECEIVED_SPAMHAUS_PBL(0.00)[15.81.88.46.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net
 : 127.0.0.10]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Sep 2019 04:40:14 -0000

Hello,

we install several pkg-based systems and poudriere from a dedicated tree of
sources, instead of /usr/src it is in our case /pool/sources/CURRENT/src and
12-STABLE/src. Compilation of the sources is done within a JAIL!

For a couple of days now, both trees, CURRENT (r352239 now) and 12-STABLE
(r352239) fail at the exact same point, when compiling and further packaging:

[...]
install -U -M
/pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstage//METALOG
-D /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstage -T
package=utilities -d -m 0755 -o root  -g wheel
/pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstage/boot
objcopy -S -O binary boot2.out boot2.bin btxld -v -E 0x2000 -f bin -b
/pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/stand/i386/btx/btx/btx
-l boot2.ldr  -o boot2.ld -P 1 boot2.bin make[10]: exec(btxld) failed (No such
file or directory) *** Error code 1
[...]

For reduction of the installed binaries and stuff, we use customized src.conf
and each build process is delegated to its appropriate src.conf by setting the
variabel SRCCONF accordingly; poudriere also uses the same src.conf by linking
the jailname-src.conf file into poudriere's config folder; the content of
src.conf is as follows:

[...]
WITH_OFED=                              YES
#WITH_CTF=                              YES
#
#WITH_BEARSSL=                          YES
#
WITH_SVN=                               YES
#
WITH_SORT_THREADS=                      YES
#
MALLOC_PRODUCTION=                      YES
#
#WITHOUT_ASSERT_DEBUG=                  YES
#WITHOUT_DEBUG_FILES=        YES
#WITHOUT_TESTS=              YES
WITHOUT_PROFILE=            YES
#
WITHOUT_REPRODUCIBLE_BUILD=     YES
#
#  mitigation for CVE-2017-5715 in the kernel build
WITH_RETPOLINE=                         YES

[...]

Building poudriere jails from such sources also fails since a couple of days on
all platforms with a weird message thata folder and/or file atf-check is
missing (this happens when using command sequence: poudriere jail -j jailname
-u -b and the install method of the appropriate jail is src=/path/to/source/src.

Thanks for helping,

oh

From owner-freebsd-current@freebsd.org  Thu Sep 12 11:38:57 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 05354F33A0
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Thu, 12 Sep 2019 11:38:57 +0000 (UTC)
 (envelope-from ohartmann@walstatt.org)
Received: from mout.gmx.net (mout.gmx.net [212.227.17.21])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client CN "mout.gmx.net",
 Issuer "TeleSec ServerPass Class 2 CA" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46TcH36v7xz48fD
 for <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 11:38:55 +0000 (UTC)
 (envelope-from ohartmann@walstatt.org)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net;
 s=badeba3b8450; t=1568288325;
 bh=p/XzAgmdRPgfsJu25R2INsLgZFQxapYmGKrzeDs8udI=;
 h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References;
 b=eAFJJC5BuyF+as/npP4hiN70s8Fo0X9wiqCBX3KXQHQNXhD24rJVUk/mzKv7Izb3G
 riyn4rtboemOYUa11+1zrvHkBYH5CF2i2tA8gLi3rSeVl2dfNt2lbWcaSMTmDLFP9N
 sgAx+VzYBETD/HKSu9UjEtyPYMsrL7MoQSO33HuQ=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from freyja ([46.88.81.15]) by mail.gmx.com (mrgmx103
 [212.227.17.168]) with ESMTPSA (Nemesis) id 0Le69A-1iTOBo1tj2-00pvsW; Thu, 12
 Sep 2019 13:26:00 +0200
Date: Thu, 12 Sep 2019 13:25:55 +0200
From: "O. Hartmann" <ohartmann@walstatt.org>
To: "O. Hartmann" <ohartmann@walstatt.org>
Cc: freebsd-current <freebsd-current@freebsd.org>
Subject: Re: r352239: install failure: make[10]: exec(btxld) failed (No such
 file or directory)
Message-ID: <20190912132555.7126215c@freyja>
In-Reply-To: <20190912062656.7afa3816@freyja>
References: <20190912062656.7afa3816@freyja>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K1:Y+3eYBUoevXwFVdbbV0+3vdPpXj/VWbN37WSo/vKtfeyUd/ZkPt
 TJRmNHlNVonVd3JqC2c1w/RfhgQZ/hkxZECkQb+t6kn4yZqha8BFAEo7OHxIvZ0P6b2wdxs
 6xp71iok1qMplFlCJHKrrWBdEnxcfRhfDTORReLKhyoSZuxO8z0l12o7j7sKuPc0OhNgplz
 oLvCARQGqY64mbUNZtJWQ==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V03:K0:UUKVZRs6HSA=:85AQ0mcCwJALB5gEpZUZ4L
 uYsESGSIX4RszAIOp8SZR+nu4RbYfJfBpkyVrG9lyn3p0MsItQxtfJGdf5e+OL7RkB1kbAUzn
 oFJkZUJD0AjgXFea/OAsX82pfcCYrb13F/b4HCjhHnhlkLOZDTH1BPo3nqL+sYyD3KZUH1S9A
 VCP1gTt1tysdGVgQHx2ijvDxAMghc9bNUK8IHHq+nAqAsiZQo8qi32elLXxz62H8FEC5wL2lU
 2urZOBK6IkiwqfxYiTJ1dmCUvVeqnjkd99Iv9kw9Sc883oN98Hy9+rn3y7wWCjjoQz0yMbMJs
 1yBM6LgRd/4DgcErHZH5IsoztPkkraxiuwFZ13AS1u9QAt2samf9pE0b3VN+zlNPl/OuC5pRd
 Sy6ajjZT9Nk+4PcPvCTsC79voYCaagRTW3JcGvNnpBGRwYhrAyx/u4/6rkY5ueyCj6tToOLMP
 2bxGsM2yWT3Y7XoD4Nb+8IGqp0BYCdh+kUZQajJL3Dbju4SLhSoaqUqtlehWaerRb5CRRpjVb
 IpQtzY8lG8xv7/uYjt2XdtWJNNDCy/15if+79lFj51mLzfDXnYgjh4dhyw+Vur07slabdHnIA
 akumSvkDsdVO4JB0xxDZssH9j9fqKmQZrEgKqY75jO7MgVCMnQ3oyYCG5UB7daVNGFBZgeCdY
 GX+6Q3hvioFFI0pbxWtgUR8ZOHF5x3oB73ZDQzT0azarIDrYtGTtQROEZfhgYTjdYEOvSI7bq
 enneJvMwmEGtmRYv0mW4tbo6NxScIRkvcFo5sZoq2+UC9WMKEYurRnOtGKDdUMgbmj2OY3AEH
 kntEKSPVMoGHxIDKwwBU9+Icy0V+bfq7F9hPtJidBeiwUQNUFReOIyzJHLR+fh133/RlyQ5UN
 vFXs9QBhkvyYnDMHIxYT0p8ZeMdarfDJKrABBoef7fZbAKqClS6A3W2+5xz7ZvWQvbltopXx0
 R1OdxmNgg3rkJ9KKUttHUfMnNWo2haUy/N2DmhGMV8olT1/pSouZSuNGNa2RMT/Ua0i46CRW5
 nkU/Hn64syN5y5TG/HjMCYJQW8zQNO0yaNH0kyhqucfnYmesnh9AFtgCcP2E0Mevomf9YW1cL
 DIw+5hsKEMeqLZQxFjm9JeS2KXjx7Iqh3fI5ZDcbi4ykZ8jo8aJEXhcH7W/cwFgJ8pbXlhz1/
 M3jTwYzyCoJFnua4nf+8sWInlh8f37a+9W/TZY+7ZRvemqfzteCbAWXnfs+QJVE5MdZUoCQ3n
 U/NRKsXAH+10b3vUzgw+KceK5JdusdXGSKFe5HCbvjRv04jGCioYYVRW1CrQ=
X-Rspamd-Queue-Id: 46TcH36v7xz48fD
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=eAFJJC5B;
 dmarc=none;
 spf=none (mx1.freebsd.org: domain of ohartmann@walstatt.org has no SPF policy
 when checking 212.227.17.21) smtp.mailfrom=ohartmann@walstatt.org
X-Spamd-Result: default: False [-3.17 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[];
 R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450];
 RECEIVED_SPAMHAUS_PBL(0.00)[15.81.88.46.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net
 : 127.0.0.10]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 IP_SCORE(-1.27)[ip: (-7.12), ipnet: 212.227.0.0/16(-1.37), asn: 8560(2.16),
 country: DE(-0.01)]; MIME_GOOD(-0.10)[text/plain];
 DMARC_NA(0.00)[walstatt.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[gmx.net:+]; RCPT_COUNT_TWO(0.00)[2];
 R_SPF_NA(0.00)[];
 RCVD_IN_DNSWL_LOW(-0.10)[21.17.227.212.list.dnswl.org : 127.0.3.1];
 FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[];
 ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE];
 MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Sep 2019 11:38:57 -0000

On Thu, 12 Sep 2019 06:27:00 +0200
"O. Hartmann" <ohartmann@walstatt.org> wrote:

> Hello,
>
> we install several pkg-based systems and poudriere from a dedicated tree=
 of
> sources, instead of /usr/src it is in our case /pool/sources/CURRENT/src=
 and
> 12-STABLE/src. Compilation of the sources is done within a JAIL!
>
> For a couple of days now, both trees, CURRENT (r352239 now) and 12-STABL=
E
> (r352239) fail at the exact same point, when compiling and further packa=
ging:
>
> [...]
> install -U -M
> /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstag=
e//METALOG
> -D /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worlds=
tage
> -T package=3Dutilities -d -m 0755 -o root  -g wheel
> /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/worldstag=
e/boot
> objcopy -S -O binary boot2.out boot2.bin btxld -v -E 0x2000 -f bin -b
> /pool/sources/CURRENT/obj/pool/sources/CURRENT/src/amd64.amd64/stand/i38=
6/btx/btx/btx
> -l boot2.ldr  -o boot2.ld -P 1 boot2.bin make[10]: exec(btxld) failed (N=
o such
> file or directory) *** Error code 1
> [...]
>
> For reduction of the installed binaries and stuff, we use customized src=
.conf
> and each build process is delegated to its appropriate src.conf by setti=
ng the
> variabel SRCCONF accordingly; poudriere also uses the same src.conf by l=
inking
> the jailname-src.conf file into poudriere's config folder; the content o=
f
> src.conf is as follows:
>
> [...]
> WITH_OFED=3D                              YES
> #WITH_CTF=3D                              YES
> #
> #WITH_BEARSSL=3D                          YES
> #
> WITH_SVN=3D                               YES
> #
> WITH_SORT_THREADS=3D                      YES
> #
> MALLOC_PRODUCTION=3D                      YES
> #
> #WITHOUT_ASSERT_DEBUG=3D                  YES
> #WITHOUT_DEBUG_FILES=3D        YES
> #WITHOUT_TESTS=3D              YES
> WITHOUT_PROFILE=3D            YES
> #
> WITHOUT_REPRODUCIBLE_BUILD=3D     YES
> #
> #  mitigation for CVE-2017-5715 in the kernel build
> WITH_RETPOLINE=3D                         YES
>
> [...]
>
> Building poudriere jails from such sources also fails since a couple of =
days
> on all platforms with a weird message thata folder and/or file atf-check=
 is
> missing (this happens when using command sequence: poudriere jail -j jai=
lname
> -u -b and the install method of the appropriate jail is
> src=3D/path/to/source/src.
>
> Thanks for helping,
>
> oh
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.or=
g"

After today's update of CURRENT's source tree and "poudriere jail -j jailn=
ame
-u -b", which should result in a successful build, the error is:

[...]
cc -target x86_64-unknown-freebsd13.0
=2D-sysroot=3D/pool/sources/CURRENT/obj/pool/poudriere/jails/headamd64/usr=
/src/amd64.amd64/tmp
-B/pool/sources/CURRENT/obj/pool/poudriere/jails/headamd64/usr/src/amd64.a=
md64/tmp/usr/bin
 -O2 -pipe -O3      -DNDEBUG   -I.
-I/pool/poudriere/jails/headamd64/usr/src/contrib/elftoolchain/libelf
-I/pool/poudriere/jails/headamd64/usr/src/contrib/elftoolchain/common
-mretpoline -g -MD  -MF.depend.elf_update.o -MTelf_update.o -std=3Dgnu99
-Wno-format-zero-length -fstack-protector-strong -Wsystem-headers -Werror =
-Wall
-Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes
-Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-str=
ings
-Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winli=
ne
-Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sig=
n
-Wmissing-variable-declarations -Wthread-safety -Wno-empty-body
-Wno-string-plus-int -Wno-unused-const-variable  -Qunused-arguments    -c
/pool/poudriere/jails/headamd64/usr/src/contrib/elftoolchain/libelf/elf_up=
date.c
-o elf_update.o
/pool/poudriere/jails/headamd64/usr/src/contrib/elftoolchain/libelf/elf_up=
date.c:841:67:
error: unused parameter 'ex' [-Werror,-Wunused-parameter]
_libelf_write_ehdr(Elf *e, unsigned char *nf, struct _Elf_Extent *ex)
[...]

As I mentioned earlier, the build is performed within a jail considered to=
 run
poudriere and the task has been performed successful earlier (a couple of =
days
ago, but didn't memorised the revision number).

On non-jailed hosts, this task works as expected, both on 12-STABLE (recen=
t
version 12.1-PRE) and CURRENT. Also did I remove the object's path and sta=
rted
a fresh build, so  remnants from an earlier build can be excluded.
/usr/local/etc/poudriere.d/jailname-poudriere.conf has a valid setting lik=
e
this:

export   MAKEOBJDIRPREFIX=3D/pool/sources/CURRENT/obj/

Another strategy also fails. Building all the binaries under ./obj from th=
e
base host with an objetctree of a valid and successful build and then jexe=
'ing
into the poudriere jail, which has all the infrastructure on ZFS already
mounted and typing there

poudriere jail -j jailname -u

which should result in a correct and successfuil installation, fails
immediately with

install -N /pool/sources/CURRENT/src/etc  -C -o root -g wheel -m 444
libpythagoras.a /pool/poudriere/jails/headamd64/usr/tests/libexec/rtld-elf=
/
install -N /pool/sources/CURRENT/src/etc  -C -o root -g wheel -m 444
libpythagoras_p.a /pool/poudriere/jails/headamd64/usr/tests/libexec/rtld-e=
lf/
=2D-- ld_library_pathfds.install --- --- _proginstall --- ---
realinstall_subdir_libexec/rtld-elf/tests/libpythagoras --- install:
libpythagoras_p.a: No such file or directory *** [_libinstall] Error code =
71

I feel a bit lost here ... is there something special to jails?

Kind regards,

oh

From owner-freebsd-current@freebsd.org  Thu Sep 12 15:54:36 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7889FD24E7
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Thu, 12 Sep 2019 15:54:36 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com
 [IPv6:2607:f8b0:4864:20::d2f])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Tjy34mWXz4QXv
 for <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 15:54:35 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: by mail-io1-xd2f.google.com with SMTP id j4so55895793iog.11
 for <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 08:54:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:date:from:to:cc:subject:message-id:references:mime-version
 :content-disposition:in-reply-to:user-agent;
 bh=UjayjPjs2x6sLHh4cI8QCU2bPWZJMuQjTyXQA7vJvW8=;
 b=p954PMDk1ETBqhRprS9GP6j9fZ3GlIE2aMR+0RmphN/q4wz6+diJXJW9ps9EYZyjm2
 OGGSXzXILzf32YUce4CF6ZXA+QsJeLLQHRUWq5kDpeSvPvTG1OAvrwd6OCsys19CWX9V
 WZRsWVIaNJVrUTCvxQ0vJmYWv9iaJD2oBPopa3kN3sRt7hfORkxdYce3d6IWO+L6sYhM
 LlC2eQF77M1IKut9ES5p2l11jdFb8rLqUkuQZapWxLB9u4RmUoQrq5KWdHYcf9PrKvrP
 RcBaGoZXVQKezfsdJlWNIoGQPRFJlVRpj/vHwycO99oa6QSJdTZnzHr1NyvzPJ3W8tD+
 Y0XA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
 :references:mime-version:content-disposition:in-reply-to:user-agent;
 bh=UjayjPjs2x6sLHh4cI8QCU2bPWZJMuQjTyXQA7vJvW8=;
 b=FqpXQneCo1GqVf+SxkW4xIwica7uxVYkKC5bXThz5Ovd2Z/En+Rht6b4aacbBAmJ/H
 9QD5Dz5HSnALjnAUM+TRyDgly390wfzy4U0Otq6xgbffLAe114Zci7kh+NpOO8ZlD89Q
 Mbmdnz1Un8KDlh9iLnmkHZZ2ljjIXd6gOEhFjgBu9JPzjZ9PDucrH0I/Wv9nkGGzPZUV
 Ux6LcWPFhaD9ei2O3U2x+eegH/CToADOU0nIYw5HmsebSvy9TGU0ln9EHL3ScxpGwJup
 LGbJ2S1FO7HIXHnTTkqBEUEsfY3+SD/sc5b07km8bXtyNl7u/zBZ+XeKEouiEHMU/0Hc
 y36A==
X-Gm-Message-State: APjAAAVebRfydLbTpPBIe2ZdKdndaradc0Zpl/gM7aTcKpehwX7EHe/3
 ARVqQ+t1wcMZ3/pT/Acbp9U5yngo
X-Google-Smtp-Source: APXvYqycYrXQtuPCB45KUaw2Dp5P7+tOMgzcSndDr4+D+J+RmJ38ql8I2YN7zKo1NZ6mkfTsyNTGIA==
X-Received: by 2002:a05:6602:2256:: with SMTP id
 o22mr2311350ioo.104.1568303674164; 
 Thu, 12 Sep 2019 08:54:34 -0700 (PDT)
Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca.
 [69.159.39.167])
 by smtp.gmail.com with ESMTPSA id y19sm23423476ioq.69.2019.09.12.08.54.32
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 12 Sep 2019 08:54:33 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Thu, 12 Sep 2019 11:54:28 -0400
From: Mark Johnston <markj@freebsd.org>
To: Mark Millard <marklmi@yahoo.com>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Subject: Re: "cpuset -n prefer:?" --what values for "?" are supposed to be
 allowed? (only 1 is, despite two numa domains)
Message-ID: <20190912155428.GA8397@raichu>
References: <C230AB3E-2FCF-4837-A4FD-A29F037647FC@yahoo.com>
 <20190911143125.GA17992@raichu>
 <99BB5653-1F42-4309-9892-24029FD02E39@yahoo.com>
 <20190911151512.GB17992@raichu>
 <3CE4AEB7-E32C-49BD-8C75-71AB8739BAEC@yahoo.com>
 <54F53CA8-6BEC-4B31-9662-C6854CDE0A08@yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <54F53CA8-6BEC-4B31-9662-C6854CDE0A08@yahoo.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Rspamd-Queue-Id: 46Tjy34mWXz4QXv
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=p954PMDk;
 dmarc=none;
 spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates
 2607:f8b0:4864:20::d2f as permitted sender) smtp.mailfrom=markjdb@gmail.com
X-Spamd-Result: default: False [-3.92 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36];
 RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2];
 FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com];
 FREEMAIL_TO(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+];
 IP_SCORE(-2.22)[ip: (-6.09), ipnet: 2607:f8b0::/32(-2.71), asn: 15169(-2.25),
 country: US(-0.05)]; FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com];
 SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[f.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Sep 2019 15:54:36 -0000

On Wed, Sep 11, 2019 at 11:14:42AM -0700, Mark Millard wrote:
> 
> 
> On 2019-Sep-11, at 10:11, Mark Millard <marklmi at yahoo.com> wrote:
> 
> 
> 
> > On 2019-Sep-11, at 08:15, Mark Johnston <markj at freebsd.org> wrote:
> > 
> >> On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote:
> >>> 
> >>> 
> >>> On 2019-Sep-11, at 07:31, Mark Johnston <markj at freebsd.org> wrote:
> >>> 
> >>>> On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote:
> >>>>> In a context with:
> >>>>> 
> >>>>> # cpuset -g
> >>>>> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
> >>>>> pid -1 domain policy: first-touch mask: 0, 1
> >>>>> 
> >>>>> I get:
> >>>>> 
> >>>>> # cpuset -l0 -n prefer:0 COMMAND
> >>>>> cpuset: setdomain: Invalid argument
> >>>>> 
> >>>>> # cpuset -l0 -n prefer:2 COMMAND
> >>>>> cpuset: setdomain: Invalid argument
> >>>>> 
> >>>>> But one prefer:? value does allow the COMMAND
> >>>>> to run:
> >>>>> 
> >>>>> # cpuset -l0 -n prefer:1 COMMAND
> >>>>> 
> >>>>> This seem odd to me. Am I missing something?
> >>>>> 
> >>>>> For reference: I'm using a ThreadRipper 1950X
> >>>>> with a head -r351227 based context for this
> >>>>> activity. The above happens to have been run
> >>>>> in a Windows 10 Pro HyperV session, instead
> >>>>> of in a native-boot of the same media. (A
> >>>>> native-boot would have had 32 CPUs.)
> >>>> 
> >>>> Can you please show the output of "sysctl vm.phys_segs" from this
> >>>> setup?
> >>> 
> >>> Sure:
> >> 
> >> I was wondering if you had only one domain populated, but it seems not
> >> to be the case.  Could you try updating to r351672 or later and see if
> >> the behaviour persists?
> > 
> > It may be a bit before I do that.
> > 
> > FYI: I had set MAXMEMDOM to match the number of
> > actual domains for the context:
> > 
> > /usr/src/sys/amd64/conf/GENERIC-DBG:options     MAXMEMDOM=2
> > /usr/src/sys/amd64/conf/GENERIC-NODBG:options   MAXMEMDOM=2
> > 
> > (These kernel configuration files include GENERIC.)

Ok, that helps.  I believe you are hitting a bug that will be fixed by
r351672 and a couple of preceding commits to the same area.

> Not that the below is the problem that I reported, but
> cpuset_modify_domain has an oddity. In the below, note
> the "root->" use followed by the "root &&" test: the
> root-> use would have failed first. Should the && be
> "dset &&" instead? Should "root &&" just be removed for
> being redundant?

Good catch.  I believe cpusets are not allowed to have a NULL domain
set, so dset should never be NULL either.

From owner-freebsd-current@freebsd.org  Thu Sep 12 19:37:38 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 684A0D83AC
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Thu, 12 Sep 2019 19:37:38 +0000 (UTC)
 (envelope-from rysto32@gmail.com)
Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com
 [IPv6:2607:f8b0:4864:20::72c])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46TpvP53wmz4fn1
 for <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 19:37:37 +0000 (UTC)
 (envelope-from rysto32@gmail.com)
Received: by mail-qk1-x72c.google.com with SMTP id u184so22809013qkd.4
 for <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 12:37:37 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:from:date:message-id:subject:to;
 bh=nBW+ufIGymAIdGqAv2jrxFR4u0H5dz8jvADK0YDjK2U=;
 b=rDmmqF565oSelmPNFMcZkXrbiW/t4zqBl0nessFDCL0O+XdX5srFWhf4niALdVct5b
 A8KtWouJUKDvT03CtpXMO937R2e0W6BespNvCR3ClPHJuv8RLlALGqQM8B5iBDRWXkvb
 IOhbZx05hSvgRknsH4ohjjq8mVEbLMORl4w16DklPTLo9BX8SiIglZjMWKmVrMKMeG9d
 KoV+xYIcjnhwsrM08jFy0VTIZUJk7FskTmh4wDIP8YUdf2bSk0O/EcVglThWedco01ej
 RRiVsa5HcDPyzivhHzGWSyTYZUHGHkNuPF3XFRQx54pGD6Tf+f8fVFruNL+bwhDVZhNn
 JdYQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=nBW+ufIGymAIdGqAv2jrxFR4u0H5dz8jvADK0YDjK2U=;
 b=JnGd+hlkgiI7bL3huL9CXnY+B/liSLR5fKszB4InaMiERu6HmffKgsm+uOuHND67/A
 8KBw/mpxBT+eKoHL/W1ep+e8wevv8IvBHwOk8bpBZraWmWTVfWRT9ZJCtbagsQ8xwelX
 0mOVFItM4iD2SkDwYgqNYx/sRL+Ckfod+lBlvQcZTg8saqwADAFK9Wf90CLPEYRc9+px
 81lAShDGrhV7IhE93ZhTEK7DznkHuP2IX7pWrtZxQsIJMeGn/X0fi004onY7VP+n2dFD
 AhlL5eHKwfFHFXk6vmo2niq5+jkB94qrZoGUYVYrmLpWLwBpaHW3qXV0KOX/DfbjDPnq
 nt4Q==
X-Gm-Message-State: APjAAAXWQw2o+K656+VBM0V8/T402p91Jv+pwfrEsFUCRoi0NRihXgrx
 jEZevdarJosOqIgPwhsl0BQHPNY8uPIWiet299FcL4UG
X-Google-Smtp-Source: APXvYqws6+yHBERuAnLhKZmFRpeJVJuvzgVpQQZ1p2KRyLfUhFG/ojLwMvujhykXM9D78IGYIsPOLLj2bpN1kEMnQ8k=
X-Received: by 2002:a37:b045:: with SMTP id z66mr19402085qke.428.1568317056311; 
 Thu, 12 Sep 2019 12:37:36 -0700 (PDT)
MIME-Version: 1.0
From: Ryan Stone <rysto32@gmail.com>
Date: Thu, 12 Sep 2019 15:37:25 -0400
Message-ID: <CAFMmRNwRWZsQhMSY7MwS2kU-qNJ0+Pk676mi59_Y3QsbcEmPVQ@mail.gmail.com>
Subject: Deadlock involving truss -f, pdfork() and wait4()
To: FreeBSD Current <freebsd-current@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 46TpvP53wmz4fn1
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=rDmmqF56;
 dmarc=pass (policy=none) header.from=gmail.com;
 spf=pass (mx1.freebsd.org: domain of rysto32@gmail.com designates
 2607:f8b0:4864:20::72c as permitted sender) smtp.mailfrom=rysto32@gmail.com
X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c];
 FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[];
 RCPT_COUNT_ONE(0.00)[1];
 IP_SCORE(0.00)[ip: (-9.32), ipnet: 2607:f8b0::/32(-2.71), asn: 15169(-2.25),
 country: US(-0.05)]; TO_DN_ALL(0.00)[];
 DKIM_TRACE(0.00)[gmail.com:+];
 DMARC_POLICY_ALLOW(-0.50)[gmail.com,none];
 RCVD_IN_DNSWL_NONE(0.00)[c.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[];
 FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[];
 DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Sep 2019 19:37:38 -0000

I've hit an issue with a simple use of pdfork().  I have a process
that calls pdfork() and the parent immediately does a wait4() on the
child pid.  This works fine under normal conditions, but if the parent
is run under truss -f, the three processes deadlock.  If I switch out
pdfork() for fork(), the deadlock does not occur.

This C file demonstrates the issue:

https://people.freebsd.org/~rstone/pdfork.c

If I run "truss -f ./pdfork", which uses fork(), it completes within a
second.  If I run "truss -f ./pdfork -p", which uses pdfork(), the
processes deadlock.  If I run "./pdfork -p" without truss, it
completes normally.

procstat reports the following kernel stacks:

27572 102043 truss               -                   mi_switch+0xe2
sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf
kern_wait6+0x695 sys_wait6+0x9f amd64_syscall+0x36e
fast_syscall_common+0x101
27573 102469 pdfork              -                   mi_switch+0xe2
sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf
kern_wait6+0x695 sys_wait4+0x78 amd64_syscall+0x36e
fast_syscall_common+0x101
27574 102053 pdfork              -                   mi_switch+0xe2
thread_suspend_switch+0xd4 ptracestop+0x13b fork_return+0x14e
fork_exit+0x83 fork_trampoline+0xe

As near as I can tell, truss is blocked waiting for ptrace events, the
parent process is blocked in wait4, and the child process is perhaps
waiting for its parent to exit the kernel so it can send the ptrace
event?

I really don't see anything obvious in the pdfork() code path that
would cause this to happen when fork() doesn't have the problem.  It
may be that pdfork() just changes the timing enough to expose a latent
bug.

I'm seeing this on a recentish current (r351363).

From owner-freebsd-current@freebsd.org  Thu Sep 12 23:00:19 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id DEB06DCE84
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Thu, 12 Sep 2019 23:00:19 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (4096 bits) client-digest SHA256)
 (Client CN "smtp.freebsd.org",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46TvPH5brKz3NHS
 for <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 23:00:19 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Received: from mousie.catspoiler.org (unknown [76.212.85.177])
 (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 (Authenticated sender: truckman)
 by smtp.freebsd.org (Postfix) with ESMTPSA id 53913627B
 for <freebsd-current@freebsd.org>; Thu, 12 Sep 2019 23:00:19 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Date: Thu, 12 Sep 2019 16:00:17 -0700 (PDT)
From: Don Lewis <truckman@FreeBSD.org>
Subject: spurious out of swap kills
To: FreeBSD Current <freebsd-current@freebsd.org>
Message-ID: <tkrat.84b3295682c83162@FreeBSD.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; CHARSET=us-ascii
Content-Disposition: INLINE
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Sep 2019 23:00:19 -0000

My poudriere machine is running 13.0-CURRENT and gets updated to the
latest version of -CURRENT periodically.  At least in the last week or
so, I've been seeing occasional port build failures when building my
default set of ports, and I finally had some time to do some
investigation.

It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap.
Poudriere is configured with
  USE_TMPFS="wrkdir data localbase"
and I have
  .if ${.CURDIR:M*/www/chromium}
  MAKE_JOBS_NUMBER=16
  .else
  MAKE_JOBS_NUMBER=7
  .endif
in /usr/local/etc/poudriere.d/make.conf, since this gives me the best
overall build time for my set of ports.  This hits memory pretty hard,
especially when chromium, firefox, libreoffice, and both versions of
openoffice are all building at the same time.  During this time, the
amount of space consumed by tmpfs for /wrkdir gets large when building
these large ports.  There is not enough RAM to hold it all, so some of
the older data spills over to swap.  Swap usage peaks at about 10 GB,
leaving about 30 GB of free swap.  Nevertheless, I see these errors,
with rustc being the usual victim:

Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space
Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space

Top shows the size of rustc being about 2 GB, so I doubt that it
suddenly needs an additional 30 GB of swap.

I'm wondering if there might be a transient kmem shortage that is
causing a malloc(..., M_NOWAIT) failure in the swap allocation path
that is the cause of the problem.


From owner-freebsd-current@freebsd.org  Fri Sep 13 00:06:40 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8F528DEDDA
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 00:06:40 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com
 [IPv6:2607:f8b0:4864:20::d35])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Twsr3H0fz3R57;
 Fri, 13 Sep 2019 00:06:40 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: by mail-io1-xd35.google.com with SMTP id n197so59154951iod.9;
 Thu, 12 Sep 2019 17:06:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:date:from:to:cc:subject:message-id:references:mime-version
 :content-disposition:in-reply-to:user-agent;
 bh=zXSDpPl0zFUKkD0PMalBOt4ILr01VMLqlspB62yRWIk=;
 b=A+wDDJ1H9RwWBg1mX4BM23Yut5RYAmvPixKV2uW3jbSNUXgB7qO3g+TXFPSo9nSwJ2
 C93FUBylNy9m1wlE+aY72gg938XYPS+oho+NeJNAp+kRk/8xKXeMtRYSybls6GewaLZM
 tEu3vRDqZ5T7b0S521K60Wre3O4K1VUDt4hhiVDAI452fEQcT+Ls0AUtS/8T/oqIHpX1
 FwT+9M96OymiRm0bYnIL/1XRVUOKeQDkDfBQfdvEfdZjlqUnhEZ+JeTFNku437tGF1Gy
 agF1BzR9Q3jZYybzKF/8gEQ+VSvNZz9UGler3miy9jgKhaUNEgstL7OntF/elkHGgtr2
 tMpw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
 :references:mime-version:content-disposition:in-reply-to:user-agent;
 bh=zXSDpPl0zFUKkD0PMalBOt4ILr01VMLqlspB62yRWIk=;
 b=dlhZucB5cUwHcG2NsKjjDdQICjwUaZp53zbnxROlPalLbt4tU7KVUf69GGgF5/lhgh
 rYakLrnSm+fsce/F6EChZk01DIMD8GRYB5wu2V7vS1EMg4hsEzRCqABk+TQWqUCHPRK9
 BMk1PBOBC9VcGAPCQrrbGiEN8+4F+dXSAaYiPnl95OaPRTHlGd1G1r9goN+epEbMTXzo
 nPdYPWj3ryfAttRJqG7bkmNE1G4GuBRJpj54QbgwVqIeMr8LIcY/j7cHQa/9tbi0BaZb
 Bwj+BEYW9aXrI9ZqY7tj+4+KnW4Xn/DjHiJbIyBTh1lGbFIv2TKH67yLs7GjA/cwYhzr
 nGLQ==
X-Gm-Message-State: APjAAAX5OKK/IkpElwkhiKC4dvPP5Sw9JFydAH63iPusaWeEJkKhwVno
 1pqXNzA0ATw2wOkF+QR0+kiarVMN
X-Google-Smtp-Source: APXvYqzybUUCiuxCOywauynyBxalm0VYYIqxWmO5db61Hi7pezxb06eSUAF5ckSLU9HwydAv1FIyJA==
X-Received: by 2002:a6b:db0e:: with SMTP id t14mr7719780ioc.93.1568333198655; 
 Thu, 12 Sep 2019 17:06:38 -0700 (PDT)
Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca.
 [69.159.39.167])
 by smtp.gmail.com with ESMTPSA id x26sm20937946ioa.37.2019.09.12.17.06.37
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Thu, 12 Sep 2019 17:06:37 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Thu, 12 Sep 2019 20:06:35 -0400
From: Mark Johnston <markj@freebsd.org>
To: Don Lewis <truckman@freebsd.org>
Cc: FreeBSD Current <freebsd-current@freebsd.org>, kib@freebsd.org
Subject: Re: spurious out of swap kills
Message-ID: <20190913000635.GG8397@raichu>
References: <tkrat.84b3295682c83162@FreeBSD.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <tkrat.84b3295682c83162@FreeBSD.org>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Rspamd-Queue-Id: 46Twsr3H0fz3R57
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-6.00 / 15.00];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; REPLY(-4.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 00:06:40 -0000

On Thu, Sep 12, 2019 at 04:00:17PM -0700, Don Lewis wrote:
> My poudriere machine is running 13.0-CURRENT and gets updated to the
> latest version of -CURRENT periodically.  At least in the last week or
> so, I've been seeing occasional port build failures when building my
> default set of ports, and I finally had some time to do some
> investigation.
> 
> It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap.
> Poudriere is configured with
>   USE_TMPFS="wrkdir data localbase"
> and I have
>   .if ${.CURDIR:M*/www/chromium}
>   MAKE_JOBS_NUMBER=16
>   .else
>   MAKE_JOBS_NUMBER=7
>   .endif
> in /usr/local/etc/poudriere.d/make.conf, since this gives me the best
> overall build time for my set of ports.  This hits memory pretty hard,
> especially when chromium, firefox, libreoffice, and both versions of
> openoffice are all building at the same time.  During this time, the
> amount of space consumed by tmpfs for /wrkdir gets large when building
> these large ports.  There is not enough RAM to hold it all, so some of
> the older data spills over to swap.  Swap usage peaks at about 10 GB,
> leaving about 30 GB of free swap.  Nevertheless, I see these errors,
> with rustc being the usual victim:
> 
> Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space
> Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space
> 
> Top shows the size of rustc being about 2 GB, so I doubt that it
> suddenly needs an additional 30 GB of swap.
> 
> I'm wondering if there might be a transient kmem shortage that is
> causing a malloc(..., M_NOWAIT) failure in the swap allocation path
> that is the cause of the problem.

Perhaps this is a consequence of r351114?  To confirm this, you might
try increasing the value of vm.pfault_oom_wait to a larger value, like
20 or 30, and see if the OOM kills still occur.

From owner-freebsd-current@freebsd.org  Fri Sep 13 00:42:02 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id CAF01E00A2
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 00:42:02 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (4096 bits) client-digest SHA256)
 (Client CN "smtp.freebsd.org",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Txff52GJz3xvd;
 Fri, 13 Sep 2019 00:42:02 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Received: from mousie.catspoiler.org (unknown [76.212.85.177])
 (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 (Authenticated sender: truckman)
 by smtp.freebsd.org (Postfix) with ESMTPSA id 156FF6F67;
 Fri, 13 Sep 2019 00:42:01 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Date: Thu, 12 Sep 2019 17:42:00 -0700 (PDT)
From: Don Lewis <truckman@FreeBSD.org>
Subject: Re: spurious out of swap kills
To: Mark Johnston <markj@freebsd.org>
cc: FreeBSD Current <freebsd-current@freebsd.org>, kib@freebsd.org
In-Reply-To: <20190913000635.GG8397@raichu>
Message-ID: <tkrat.1a0e98a230c1a223@FreeBSD.org>
References: <tkrat.84b3295682c83162@FreeBSD.org> <20190913000635.GG8397@raichu>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; CHARSET=us-ascii
Content-Disposition: INLINE
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 00:42:02 -0000

On 12 Sep, Mark Johnston wrote:
> On Thu, Sep 12, 2019 at 04:00:17PM -0700, Don Lewis wrote:
>> My poudriere machine is running 13.0-CURRENT and gets updated to the
>> latest version of -CURRENT periodically.  At least in the last week or
>> so, I've been seeing occasional port build failures when building my
>> default set of ports, and I finally had some time to do some
>> investigation.
>> 
>> It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap.
>> Poudriere is configured with
>>   USE_TMPFS="wrkdir data localbase"
>> and I have
>>   .if ${.CURDIR:M*/www/chromium}
>>   MAKE_JOBS_NUMBER=16
>>   .else
>>   MAKE_JOBS_NUMBER=7
>>   .endif
>> in /usr/local/etc/poudriere.d/make.conf, since this gives me the best
>> overall build time for my set of ports.  This hits memory pretty hard,
>> especially when chromium, firefox, libreoffice, and both versions of
>> openoffice are all building at the same time.  During this time, the
>> amount of space consumed by tmpfs for /wrkdir gets large when building
>> these large ports.  There is not enough RAM to hold it all, so some of
>> the older data spills over to swap.  Swap usage peaks at about 10 GB,
>> leaving about 30 GB of free swap.  Nevertheless, I see these errors,
>> with rustc being the usual victim:
>> 
>> Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space
>> Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space
>> 
>> Top shows the size of rustc being about 2 GB, so I doubt that it
>> suddenly needs an additional 30 GB of swap.
>> 
>> I'm wondering if there might be a transient kmem shortage that is
>> causing a malloc(..., M_NOWAIT) failure in the swap allocation path
>> that is the cause of the problem.
> 
> Perhaps this is a consequence of r351114?  To confirm this, you might
> try increasing the value of vm.pfault_oom_wait to a larger value, like
> 20 or 30, and see if the OOM kills still occur.

I wonder if increasing vm.pfault_oom_attempts might also be a good idea.


From owner-freebsd-current@freebsd.org  Fri Sep 13 05:53:40 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id F1E48E639F
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 05:53:40 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46V4ZD57rjz4BNT;
 Fri, 13 Sep 2019 05:53:40 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from tom.home (kib@localhost [127.0.0.1])
 by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x8D5rW4q048713
 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO);
 Fri, 13 Sep 2019 08:53:35 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x8D5rW4q048713
Received: (from kostik@localhost)
 by tom.home (8.15.2/8.15.2/Submit) id x8D5rWw9048712;
 Fri, 13 Sep 2019 08:53:32 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
 using -f
Date: Fri, 13 Sep 2019 08:53:32 +0300
From: Konstantin Belousov <kostikbel@gmail.com>
To: Don Lewis <truckman@FreeBSD.org>
Cc: Mark Johnston <markj@freebsd.org>,
 FreeBSD Current <freebsd-current@freebsd.org>, kib@freebsd.org
Subject: Re: spurious out of swap kills
Message-ID: <20190913055332.GN2559@kib.kiev.ua>
References: <tkrat.84b3295682c83162@FreeBSD.org> <20190913000635.GG8397@raichu>
 <tkrat.1a0e98a230c1a223@FreeBSD.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <tkrat.1a0e98a230c1a223@FreeBSD.org>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM,
 NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home
X-Rspamd-Queue-Id: 46V4ZD57rjz4BNT
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-5.99 / 15.00];
 NEURAL_HAM_MEDIUM(-0.99)[-0.995,0];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 05:53:41 -0000

On Thu, Sep 12, 2019 at 05:42:00PM -0700, Don Lewis wrote:
> On 12 Sep, Mark Johnston wrote:
> > On Thu, Sep 12, 2019 at 04:00:17PM -0700, Don Lewis wrote:
> >> My poudriere machine is running 13.0-CURRENT and gets updated to the
> >> latest version of -CURRENT periodically.  At least in the last week or
> >> so, I've been seeing occasional port build failures when building my
> >> default set of ports, and I finally had some time to do some
> >> investigation.
> >> 
> >> It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap.
> >> Poudriere is configured with
> >>   USE_TMPFS="wrkdir data localbase"
> >> and I have
> >>   .if ${.CURDIR:M*/www/chromium}
> >>   MAKE_JOBS_NUMBER=16
> >>   .else
> >>   MAKE_JOBS_NUMBER=7
> >>   .endif
> >> in /usr/local/etc/poudriere.d/make.conf, since this gives me the best
> >> overall build time for my set of ports.  This hits memory pretty hard,
> >> especially when chromium, firefox, libreoffice, and both versions of
> >> openoffice are all building at the same time.  During this time, the
> >> amount of space consumed by tmpfs for /wrkdir gets large when building
> >> these large ports.  There is not enough RAM to hold it all, so some of
> >> the older data spills over to swap.  Swap usage peaks at about 10 GB,
> >> leaving about 30 GB of free swap.  Nevertheless, I see these errors,
> >> with rustc being the usual victim:
> >> 
> >> Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space
> >> Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space
> >> 
> >> Top shows the size of rustc being about 2 GB, so I doubt that it
> >> suddenly needs an additional 30 GB of swap.
> >> 
> >> I'm wondering if there might be a transient kmem shortage that is
> >> causing a malloc(..., M_NOWAIT) failure in the swap allocation path
> >> that is the cause of the problem.
> > 
> > Perhaps this is a consequence of r351114?  To confirm this, you might
> > try increasing the value of vm.pfault_oom_wait to a larger value, like
> > 20 or 30, and see if the OOM kills still occur.
> 
> I wonder if increasing vm.pfault_oom_attempts might also be a good idea.
If you are sure that you cannot exhaust your swap space, set
attempts to -1 to disable this mechanism.

Basically, page fault handler waits for vm.pfault_oom_wait *
vm.pfault_oom_attempts for a page allocation before killing the process.
Default is 30 secs, and if you cannot get a page for 30 secs, there is
something very wrong with the machine.

From owner-freebsd-current@freebsd.org  Fri Sep 13 14:05:37 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5A4DF26A9
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 14:05:37 +0000 (UTC)
 (envelope-from oshogbo.vx@gmail.com)
Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com
 [209.85.215.193])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VHTq5MMlz3ByX
 for <freebsd-current@freebsd.org>; Fri, 13 Sep 2019 14:05:35 +0000 (UTC)
 (envelope-from oshogbo.vx@gmail.com)
Received: by mail-pg1-f193.google.com with SMTP id u72so15303590pgb.10
 for <freebsd-current@freebsd.org>; Fri, 13 Sep 2019 07:05:35 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=a0ctwTlwL0MXYKXjkQGU3EGj8KmPHjmsxjsU5sqXwAA=;
 b=JrDDnuT9C0cdHB31/jqFjBetXNNyi3r3xA/06a0BBLXKOQyrLcPlGGzqNxw+WqrOAN
 hdyTWxw+A0cQ6yDSon3UNsxSITVcGc5sDZacbIqOE0v9PP0yYq7V0YFmtbVlJV43JjnF
 LP5hhqwDzFl8hRZP5Ki/8jX3UtTt085FczkM5sDGv0Z7nplH8960WkSphxFYSX6ALhzb
 hWYVJ+zLC9smecs/fyaxmMvU/nwVVCWjVVTsp4zK+oBU0NaUEnQTBQow3lcmIzOtZogp
 X71HxMOOYnmot8HjQl/I/EUoQn/yOzSiW0SsUOFJ6Z7sA7tUgRxZOqlOo/LkaRFHB9Jj
 ipXg==
X-Gm-Message-State: APjAAAWVMtiJGkSdAkSaqTHa9Rbp2mUDRGFMX9OjvQ8tnO3sOUy2s5Nh
 bmMrDnscgOtGVfc+/qHL96YCmaz18y0AzypO8Ec=
X-Google-Smtp-Source: APXvYqwDSQU1EvYVtl7f4bMOB0WfyxP7QtP8h0P6wN2l7QYHE3Kj8CuD0VXedBPnGY5BCaoWH9v/zLiaN6kX5vhvhR4=
X-Received: by 2002:a17:90a:340d:: with SMTP id
 o13mr5336079pjb.19.1568383533909; 
 Fri, 13 Sep 2019 07:05:33 -0700 (PDT)
MIME-Version: 1.0
References: <CAFMmRNwRWZsQhMSY7MwS2kU-qNJ0+Pk676mi59_Y3QsbcEmPVQ@mail.gmail.com>
In-Reply-To: <CAFMmRNwRWZsQhMSY7MwS2kU-qNJ0+Pk676mi59_Y3QsbcEmPVQ@mail.gmail.com>
From: Mariusz Zaborski <oshogbo@freebsd.org>
Date: Fri, 13 Sep 2019 16:05:21 +0200
Message-ID: <CAGOYWV-v1-F3dWQovdj5TZMZeBFifDm020AXnPYdZSHPKLG4SA@mail.gmail.com>
Subject: Re: Deadlock involving truss -f, pdfork() and wait4()
To: Ryan Stone <rysto32@gmail.com>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 46VHTq5MMlz3ByX
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of oshogbovx@gmail.com designates
 209.85.215.193 as permitted sender) smtp.mailfrom=oshogbovx@gmail.com
X-Spamd-Result: default: False [-3.12 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_TLS_ALL(0.00)[];
 FROM_HAS_DN(0.00)[];
 RWL_MAILSPIKE_GOOD(0.00)[193.215.85.209.rep.mailspike.net : 127.0.0.18];
 R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org];
 DMARC_NA(0.00)[freebsd.org]; MIME_TRACE(0.00)[0:+];
 TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2];
 RCVD_IN_DNSWL_NONE(0.00)[193.215.85.209.list.dnswl.org : 127.0.5.0];
 TO_MATCH_ENVRCPT_SOME(0.00)[];
 IP_SCORE(-1.12)[ipnet: 209.85.128.0/17(-3.32), asn: 15169(-2.24), country:
 US(-0.05)]; 
 FORGED_SENDER(0.30)[oshogbo@freebsd.org,oshogbovx@gmail.com];
 FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
 TAGGED_FROM(0.00)[];
 FROM_NEQ_ENVFROM(0.00)[oshogbo@freebsd.org,oshogbovx@gmail.com];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 14:05:38 -0000

Hello Ryan,

Can you verify is this patch fix your issue:
https://reviews.freebsd.org/D20362

Thanks,
Mariusz

On Thu, 12 Sep 2019 at 21:37, Ryan Stone <rysto32@gmail.com> wrote:
>
> I've hit an issue with a simple use of pdfork().  I have a process
> that calls pdfork() and the parent immediately does a wait4() on the
> child pid.  This works fine under normal conditions, but if the parent
> is run under truss -f, the three processes deadlock.  If I switch out
> pdfork() for fork(), the deadlock does not occur.
>
> This C file demonstrates the issue:
>
> https://people.freebsd.org/~rstone/pdfork.c
>
> If I run "truss -f ./pdfork", which uses fork(), it completes within a
> second.  If I run "truss -f ./pdfork -p", which uses pdfork(), the
> processes deadlock.  If I run "./pdfork -p" without truss, it
> completes normally.
>
> procstat reports the following kernel stacks:
>
> 27572 102043 truss               -                   mi_switch+0xe2
> sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf
> kern_wait6+0x695 sys_wait6+0x9f amd64_syscall+0x36e
> fast_syscall_common+0x101
> 27573 102469 pdfork              -                   mi_switch+0xe2
> sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf
> kern_wait6+0x695 sys_wait4+0x78 amd64_syscall+0x36e
> fast_syscall_common+0x101
> 27574 102053 pdfork              -                   mi_switch+0xe2
> thread_suspend_switch+0xd4 ptracestop+0x13b fork_return+0x14e
> fork_exit+0x83 fork_trampoline+0xe
>
> As near as I can tell, truss is blocked waiting for ptrace events, the
> parent process is blocked in wait4, and the child process is perhaps
> waiting for its parent to exit the kernel so it can send the ptrace
> event?
>
> I really don't see anything obvious in the pdfork() code path that
> would cause this to happen when fork() doesn't have the problem.  It
> may be that pdfork() just changes the timing enough to expose a latent
> bug.
>
> I'm seeing this on a recentish current (r351363).
> _______________________________________________
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"

From owner-freebsd-current@freebsd.org  Fri Sep 13 16:24:57 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 36EE3F5827
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 16:24:57 +0000 (UTC)
 (envelope-from fbsd@www.zefox.net)
Received: from www.zefox.net (www.zefox.net [50.1.20.27])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "www.zefox.org", Issuer "www.zefox.org" (not verified))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VLZc153Cz3Mx7;
 Fri, 13 Sep 2019 16:24:55 +0000 (UTC)
 (envelope-from fbsd@www.zefox.net)
Received: from www.zefox.net (localhost [127.0.0.1])
 by www.zefox.net (8.15.2/8.15.2) with ESMTPS id x8DGOiNU029093
 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
 Fri, 13 Sep 2019 09:24:45 -0700 (PDT)
 (envelope-from fbsd@www.zefox.net)
Received: (from fbsd@localhost)
 by www.zefox.net (8.15.2/8.15.2/Submit) id x8DGOiUG029092;
 Fri, 13 Sep 2019 09:24:44 -0700 (PDT) (envelope-from fbsd)
Date: Fri, 13 Sep 2019 09:24:44 -0700
From: bob prohaska <fbsd@www.zefox.net>
To: Konstantin Belousov <kostikbel@gmail.com>
Cc: Don Lewis <truckman@FreeBSD.org>, Mark Johnston <markj@freebsd.org>,
 FreeBSD Current <freebsd-current@freebsd.org>, kib@freebsd.org,
 bob prohaska <fbsd@www.zefox.net>
Subject: Re: spurious out of swap kills
Message-ID: <20190913162444.GA28886@www.zefox.net>
References: <tkrat.84b3295682c83162@FreeBSD.org> <20190913000635.GG8397@raichu>
 <tkrat.1a0e98a230c1a223@FreeBSD.org>
 <20190913055332.GN2559@kib.kiev.ua>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190913055332.GN2559@kib.kiev.ua>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-Rspamd-Queue-Id: 46VLZc153Cz3Mx7
X-Spamd-Bar: ++
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=none (mx1.freebsd.org: domain of fbsd@www.zefox.net has no SPF policy when
 checking 50.1.20.27) smtp.mailfrom=fbsd@www.zefox.net
X-Spamd-Result: default: False [2.79 / 15.00]; ARC_NA(0.00)[];
 WWW_DOT_DOMAIN(0.50)[]; RCVD_TLS_ALL(0.00)[];
 FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[];
 IP_SCORE(0.08)[ip: (0.34), ipnet: 50.1.16.0/20(0.17), asn: 7065(-0.05),
 country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain];
 DMARC_NA(0.00)[zefox.net]; AUTH_NA(1.00)[];
 NEURAL_SPAM_MEDIUM(0.57)[0.573,0]; RCPT_COUNT_FIVE(0.00)[6];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.24)[0.237,0];
 R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com];
 FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US];
 MID_RHS_MATCH_FROM(0.00)[]; MID_RHS_WWW(0.50)[];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 16:24:57 -0000

Not sure this is relevant, but in compiling chromium on a Pi3 with 6 GB
of swap the job completed successfully some months ago, with peak swap 
use around 3.5 GB. The swap layout was sub-optimal, with a 2 GB partition
combined with a 4 GB partition. A little over 4GB total seems usable. 

A few days ago the same attempt stopped with a series of OOMA kills,
but in each case simply restarting allowed the compile to pick up
where it left off and continue, eventually finishing with a runnable
version of chromium. In this case swap use peaked a little over 4 GB.

Might this suggest the machine isn't freeing swap in a timely manner?

Thanks for reading,

bob prohaska


From owner-freebsd-current@freebsd.org  Fri Sep 13 18:13:08 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6D8EBF7458
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 18:13:08 +0000 (UTC)
 (envelope-from rysto32@gmail.com)
Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com
 [IPv6:2607:f8b0:4864:20::843])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VNzS2Mzgz3xqP;
 Fri, 13 Sep 2019 18:13:08 +0000 (UTC)
 (envelope-from rysto32@gmail.com)
Received: by mail-qt1-x843.google.com with SMTP id g16so1413199qto.9;
 Fri, 13 Sep 2019 11:13:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=R0jxhCKH/ykKWRzx1nsZPZVQKqlCqmbFf6nq9D07Rvc=;
 b=rIKvbvDmFxIpKY1c3bg8XRfFW7LHSFcr+COjLUbZoUd5/tXQMmOD0q3BudZD3vz1Pj
 GaS0t93yplaY9AD8V7pV7nq5gNx5DuYChYwryHHpAh1it0mx8bcbYWhWHG81WLyCB0nM
 P/xWG2+OoyfYUXFkMg7Wv4qAd/tnzF2wZ2grnj2igTntLDOHCyyl+GgXGqe9AcupSN73
 qakFqWVHvPJLHW2BcTP1nv8fzM5AViB7FcLj/yslhy0ZDOE+vz6yngrXTMT7Qch4mx68
 rxMvyDpDtpAez+ONkNFxtSSkcvqCZiHL5Zgq5nTe2Er3IF/oUw2IRAEAERupGt9JOJRX
 kNww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=R0jxhCKH/ykKWRzx1nsZPZVQKqlCqmbFf6nq9D07Rvc=;
 b=FaPGw8ip4LF9csKuN55RrpgAJRIwYrqBaXw9nevvrXwl9o2DkGQ4jGzGji6i3yKb8u
 Q1LOm696XVRguAm0f3ILy10TYBI5a9J9WUuh+83wAAdUvMUeIoplL3/Xn6K//1yegA5i
 DhqX6wWD3aeoVFW68moDfug87nA1NFuQqp/GrF0KEH2KZe33qWeGUGDN63vYF07XuXfM
 lKLqqcdS2ns4zOJR198O2WMKWKk4mq/nULigqKdANyJp6D0i5Hx8BcY4NcvXHN3XjXR4
 0JUJ+FmHYRmNgYSiH3+tz3RSR8TcB+XJPdA+Entasg4HSRiAZezWHH6STwvdd1sPdvmy
 k+3A==
X-Gm-Message-State: APjAAAVilOikH1XBiQJGRwDJbdgN+tfBeOwm46HLlgQjd3FqLN+XYTd6
 tnFfGGb64SDAMS3c5oYX/5s9xujAs5hHwD0//6/mVwUnoWk=
X-Google-Smtp-Source: APXvYqzL2HOnOw5TETrgwQE87xHs5U1IuVhM76OT4tgZ1ELif7X1E7TRuFBNbjyaEhibUs2koJIqtAMGa9pwLX/roQk=
X-Received: by 2002:ac8:845:: with SMTP id x5mr4477819qth.42.1568398386792;
 Fri, 13 Sep 2019 11:13:06 -0700 (PDT)
MIME-Version: 1.0
References: <CAFMmRNwRWZsQhMSY7MwS2kU-qNJ0+Pk676mi59_Y3QsbcEmPVQ@mail.gmail.com>
 <CAGOYWV-v1-F3dWQovdj5TZMZeBFifDm020AXnPYdZSHPKLG4SA@mail.gmail.com>
In-Reply-To: <CAGOYWV-v1-F3dWQovdj5TZMZeBFifDm020AXnPYdZSHPKLG4SA@mail.gmail.com>
From: Ryan Stone <rysto32@gmail.com>
Date: Fri, 13 Sep 2019 14:12:56 -0400
Message-ID: <CAFMmRNzhsk-1BO6iyJz7_qtgnRqC+fPpYSuQnxEZyno+O1K71Q@mail.gmail.com>
Subject: Re: Deadlock involving truss -f, pdfork() and wait4()
To: Mariusz Zaborski <oshogbo@freebsd.org>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 46VNzS2Mzgz3xqP
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-6.00 / 15.00];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; REPLY(-4.00)[];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 18:13:08 -0000

This gets me a little further but now the wait4 call by the parent
never reaps the child and instead blocks forever:

# truss -f ./pdfork -p
 706: mmap(0x0,131072,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34361
970688 (0x800221000)
 706: issetugid()                               = 0 (0x0)
 706: open("/etc/libmap.conf",O_RDONLY|O_CLOEXEC,010440020) = 3 (0x3)
 706: fstat(3,{ mode=-rw-r--r-- ,inode=241058,size=47,blksize=32768 }) = 0 (0x0
)
 706: read(3,"# $FreeBSD$\nincludedir /usr/loc"...,47) = 47 (0x2f)
 706: close(3)                                  = 0 (0x0)
 706: open("/usr/local/etc/libmap.d",O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC,
0165) ERR#2 'No such file or directory'
 706: open("/var/run/ld-elf.so.hints",O_RDONLY|O_CLOEXEC,010002355) = 3 (0x3)
 706: read(3,"Ehnt\^A\0\0\0\M^@\0\0\0A\0\0\0\0"...,128) = 128 (0x80)
 706: fstat(3,{ mode=-r--r--r-- ,inode=72,size=193,blksize=4096 }) = 0 (0x0)
 706: pread(3,"/lib:/usr/lib:/usr/lib/compat:/u"...,65,0x80) = 65 (0x41)
 706: close(3)                                  = 0 (0x0)
 706: open("/lib/libc.so.7",O_RDONLY|O_CLOEXEC|O_VERIFY,057400) = 3 (0x3)
 706: fstat(3,{ mode=-r--r--r-- ,inode=402754,size=1915744,blksize=32768 }) = 0
(0x0)
 706: mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 3436210176
0 (0x800241000)
 706: mmap(0x0,4169728,PROT_NONE,MAP_GUARD,-1,0x0) = 34362105856 (0x800242000)
 706: mmap(0x800242000,524288,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PR
EFAULT_READ,3,0x0) = 34362105856 (0x800242000)
 706: mmap(0x8002c2000,1327104,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NO
CORE|MAP_PREFAULT_READ,3,0x80000) = 34362630144 (0x8002c2000)
 706: mmap(0x800406000,61440,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PRE
FAULT_READ,3,0x1c4000) = 34363957248 (0x800406000)
 706: mmap(0x800415000,2256896,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_A
NON,-1,0x0) = 34364018688 (0x800415000)
 706: munmap(0x800241000,4096)                  = 0 (0x0)
 706: close(3)                                  = 0 (0x0)
 706: mprotect(0x80040c000,36864,PROT_READ)     = 0 (0x0)
 706: sysarch(AMD64_SET_FSBASE,0x7fffffffe110)  = 0 (0x0)
 706: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG
TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS
Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
 706: mprotect(0x80040c000,36864,PROT_READ|PROT_WRITE) = 0 (0x0)
 706: sigprocmask(SIG_SETMASK,{ },0x0)          = 0 (0x0)
 706: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG
TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS
Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
 706: mprotect(0x80040c000,36864,PROT_READ)     = 0 (0x0)
 706: sigprocmask(SIG_SETMASK,{ },0x0)          = 0 (0x0)
 706: readlink("/etc/malloc.conf",0x7fffffffd830,1024) ERR#2 'No such file or d
irectory'
 706: issetugid()                               = 0 (0x0)
 706: __sysctl(0x7fffffffd7e0,0x2,0x7fffffffd7dc,0x7fffffffd7d0,0x0,0x0) = 0 (0
x0)
 706: mmap(0x0,2097152,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(21
),-1,0x0) = 34368126976 (0x800800000)
 706: cap_getmode({ 0 })                        = 0 (0x0)
 706: open("/dev/hpet0",O_RDONLY,00)            = 3 (0x3)
 706: mmap(0x0,4096,PROT_READ,MAP_SHARED,3,0x0) = 34362101760 (0x800241000)
 706: close(3)                                  = 0 (0x0)
 706: mmap(0x0,32768,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),
-1,0x0) = 34366275584 (0x80063c000)
 706: mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(21
),-1,0x0) = 34370224128 (0x800a00000)
 706: mmap(0x0,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-
1,0x0) = 34366308352 (0x800644000)
 706: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG
TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS
Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
 706: sigprocmask(SIG_SETMASK,{ },0x0)          = 0 (0x0)
 706: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG
TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS
Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
 706: mprotect(0x203000,4096,PROT_READ)         = 0 (0x0)
 706: sigprocmask(SIG_SETMASK,{ },0x0)          = 0 (0x0)
 706: pdfork(0x7fffffffeba8,0x0)                = 708 (0x2c4)
 708: <new process>
 708: nanosleep({ 1.000000000 })                = 0 (0x0)
 708: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG
TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS
Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
 708: sigprocmask(SIG_SETMASK,{ },0x0)          = 0 (0x0)
 708: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG
TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS
Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
 708: sigprocmask(SIG_SETMASK,{ },0x0)          = 0 (0x0)
 708: sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIG
TERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFS
Z|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
 708: sigprocmask(SIG_SETMASK,{ },0x0)          = 0 (0x0)
load: 0.27  cmd: pdfork 706 [wait] 18.20r 0.00u 0.00s 0% 2072k

# ps
PID TT  STAT    TIME COMMAND
698 u0  Is   0:00.01 login [pam] (login)
700 u0  I    0:00.04 -sh (sh)
705 u0  I+   0:00.10 truss -f ./pdfork -p
706 u0  IX+  0:00.01 ./pdfork -p
708 u0  Z+   0:00.00 <defunct>
714  0  S    0:00.01 su
715  0  S    0:00.01 su (sh)
716  0  R+   0:00.00 ps
# procstat -kk 708
 PID    TID COMM                TDNAME              KSTACK
# procstat -kk 706
 PID    TID COMM                TDNAME              KSTACK
 706 100095 pdfork              -                   mi_switch+0x174
sleepq_switch+0x110 sleepq_catch_signals+0x417 slee
pq_wait_sig+0xf _sleep+0x2d0 kern_wait6+0x48f sys_wait4+0x78
amd64_syscall+0x337 fast_syscall_common+0x101
# procstat -kk 705
 PID    TID COMM                TDNAME              KSTACK
 705 100077 truss               -                   mi_switch+0x174
sleepq_switch+0x110 sleepq_catch_signals+0x417 slee
pq_wait_sig+0xf _sleep+0x2d0 kern_wait6+0x48f sys_wait6+0x9f
amd64_syscall+0x337 fast_syscall_common+0x101

On Fri, Sep 13, 2019 at 10:05 AM Mariusz Zaborski <oshogbo@freebsd.org> wrote:
>
> Hello Ryan,
>
> Can you verify is this patch fix your issue:
> https://reviews.freebsd.org/D20362
>
> Thanks,
> Mariusz
>
> On Thu, 12 Sep 2019 at 21:37, Ryan Stone <rysto32@gmail.com> wrote:
> >
> > I've hit an issue with a simple use of pdfork().  I have a process
> > that calls pdfork() and the parent immediately does a wait4() on the
> > child pid.  This works fine under normal conditions, but if the parent
> > is run under truss -f, the three processes deadlock.  If I switch out
> > pdfork() for fork(), the deadlock does not occur.
> >
> > This C file demonstrates the issue:
> >
> > https://people.freebsd.org/~rstone/pdfork.c
> >
> > If I run "truss -f ./pdfork", which uses fork(), it completes within a
> > second.  If I run "truss -f ./pdfork -p", which uses pdfork(), the
> > processes deadlock.  If I run "./pdfork -p" without truss, it
> > completes normally.
> >
> > procstat reports the following kernel stacks:
> >
> > 27572 102043 truss               -                   mi_switch+0xe2
> > sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf
> > kern_wait6+0x695 sys_wait6+0x9f amd64_syscall+0x36e
> > fast_syscall_common+0x101
> > 27573 102469 pdfork              -                   mi_switch+0xe2
> > sleepq_catch_signals+0x425 sleepq_wait_sig+0xf _sleep+0x1bf
> > kern_wait6+0x695 sys_wait4+0x78 amd64_syscall+0x36e
> > fast_syscall_common+0x101
> > 27574 102053 pdfork              -                   mi_switch+0xe2
> > thread_suspend_switch+0xd4 ptracestop+0x13b fork_return+0x14e
> > fork_exit+0x83 fork_trampoline+0xe
> >
> > As near as I can tell, truss is blocked waiting for ptrace events, the
> > parent process is blocked in wait4, and the child process is perhaps
> > waiting for its parent to exit the kernel so it can send the ptrace
> > event?
> >
> > I really don't see anything obvious in the pdfork() code path that
> > would cause this to happen when fork() doesn't have the problem.  It
> > may be that pdfork() just changes the timing enough to expose a latent
> > bug.
> >
> > I'm seeing this on a recentish current (r351363).
> > _______________________________________________
> > freebsd-current@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"

From owner-freebsd-current@freebsd.org  Fri Sep 13 18:37:23 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 88012F7A96
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 18:37:23 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com
 [IPv6:2607:f8b0:4864:20::d30])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VPWQ6T1jz3yl3;
 Fri, 13 Sep 2019 18:37:22 +0000 (UTC)
 (envelope-from markjdb@gmail.com)
Received: by mail-io1-xd30.google.com with SMTP id d17so42791449ios.13;
 Fri, 13 Sep 2019 11:37:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:date:from:to:cc:subject:message-id:references:mime-version
 :content-disposition:in-reply-to:user-agent;
 bh=5Qx0GXouBf82DYf/SnRPVZZBk6ja8twCzszm05JDo9M=;
 b=bSzODw+d0MlFt74H7nV4JPlYdLIwAcYSqcSqrLw+Cxh17Qz9gRR0MF/fPWEwmIOEcR
 SI/5jWDxv7rH7GgIpc4wDVMxOKPCW1LWRpuPAzEn5ga4PaGoxU2iPzIFzsudZFBt2Lgy
 I3mAxf1rTJYKPSu80TPEhJF08NfeGSRQqW7E4NSpH35rRf7VLzSg/XgBwS7wt2HjiGBj
 2McIAC9Qp8p+RPXBpqLKM5E4ORIbNJph3F8Y1UIQzjCha+SMC5ULneigoLs2Aas4W4lu
 7QcTWNo1Nd4ZS/fXCJYejFr+jUMt2+pLaViNOVyibxpB4wc+O+6d0+HFjf74/6BPzMaB
 iy9Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
 :references:mime-version:content-disposition:in-reply-to:user-agent;
 bh=5Qx0GXouBf82DYf/SnRPVZZBk6ja8twCzszm05JDo9M=;
 b=tlSYLcrFQCZUsaBtjqTSGzhLPipAwsustcSATU8G4n13fZ967k4O+mPJ+ZO2QbEWIY
 ItfvV24TDuOGJhlsAaTbhAEY/be/Y8rUMD+uiqUq1/0/Jn5wj17GWWt4dsgTKu+CcvY7
 QXpS/Xe87bdH1pGmcKRggcr48HeIKGGQ793Jf7WFJzzXRdGpjblIYtY29/jSiefVUFBz
 PcXwIta1ZtFOAAAvpQVLQbMINFvYCXDifmA/5RVwfZAenPW+CiGgwgWWC7N4XQBofzI+
 5EsrFzjzK43lINLNx17kawoQTApJJd5J+RYo+NTaVb3UJK1H5dhhYTQzGmzlmAAm7l6P
 BblA==
X-Gm-Message-State: APjAAAUXKWmOvHwEwGOqUQNtcKnlbx0OEewOn0tK/M6op8VrxK+zW8vW
 IlqekhWeJDGX6V+cRWmwHQ0s2dFQ
X-Google-Smtp-Source: APXvYqzV9mkyzjw9IRU0NGPnEVRlY/EZPXTy4hTk4ZcC+tGKK9Zi5y/rknuAra33YuBiQF8iCdvJXw==
X-Received: by 2002:a02:712b:: with SMTP id n43mr24250607jac.2.1568399841986; 
 Fri, 13 Sep 2019 11:37:21 -0700 (PDT)
Received: from raichu (toroon0560w-lp140-01-69-159-39-167.dsl.bell.ca.
 [69.159.39.167])
 by smtp.gmail.com with ESMTPSA id k22sm20409152iom.45.2019.09.13.11.37.20
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 13 Sep 2019 11:37:20 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Fri, 13 Sep 2019 14:37:18 -0400
From: Mark Johnston <markj@freebsd.org>
To: Ryan Stone <rysto32@gmail.com>
Cc: Mariusz Zaborski <oshogbo@freebsd.org>,
 FreeBSD Current <freebsd-current@freebsd.org>
Subject: Re: Deadlock involving truss -f, pdfork() and wait4()
Message-ID: <20190913183718.GC84156@raichu>
References: <CAFMmRNwRWZsQhMSY7MwS2kU-qNJ0+Pk676mi59_Y3QsbcEmPVQ@mail.gmail.com>
 <CAGOYWV-v1-F3dWQovdj5TZMZeBFifDm020AXnPYdZSHPKLG4SA@mail.gmail.com>
 <CAFMmRNzhsk-1BO6iyJz7_qtgnRqC+fPpYSuQnxEZyno+O1K71Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAFMmRNzhsk-1BO6iyJz7_qtgnRqC+fPpYSuQnxEZyno+O1K71Q@mail.gmail.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
X-Rspamd-Queue-Id: 46VPWQ6T1jz3yl3
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=gmail.com header.s=20161025 header.b=bSzODw+d;
 dmarc=none;
 spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates
 2607:f8b0:4864:20::d30 as permitted sender) smtp.mailfrom=markjdb@gmail.com
X-Spamd-Result: default: False [-3.73 / 15.00]; ARC_NA(0.00)[];
 RCVD_VIA_SMTP_AUTH(0.00)[];
 R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
 RCPT_COUNT_THREE(0.00)[3];
 R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.org];
 RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+];
 RCVD_IN_DNSWL_NONE(0.00)[0.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
 : 127.0.5.0]; 
 IP_SCORE(-2.03)[ip: (-5.17), ipnet: 2607:f8b0::/32(-2.71), asn: 15169(-2.24),
 country: US(-0.05)]; 
 FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com];
 FREEMAIL_TO(0.00)[gmail.com]; MID_RHS_NOT_FQDN(0.50)[];
 FREEMAIL_ENVFROM(0.00)[gmail.com];
 ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
 FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com];
 RCVD_TLS_ALL(0.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 18:37:23 -0000

On Fri, Sep 13, 2019 at 02:12:56PM -0400, Ryan Stone wrote:
> This gets me a little further but now the wait4 call by the parent
> never reaps the child and instead blocks forever:

Does it perform a wildcarded wait(), or does it explicitly specify the
PID of the child?  By design, the former will not return children
created by pdfork().

From owner-freebsd-current@freebsd.org  Fri Sep 13 18:56:02 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC452F7FE0
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 18:56:02 +0000 (UTC)
 (envelope-from cse.cem@gmail.com)
Received: from mail-io1-f47.google.com (mail-io1-f47.google.com
 [209.85.166.47])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VPwy46Gqz40kD;
 Fri, 13 Sep 2019 18:56:02 +0000 (UTC)
 (envelope-from cse.cem@gmail.com)
Received: by mail-io1-f47.google.com with SMTP id n197so64870990iod.9;
 Fri, 13 Sep 2019 11:56:02 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:reply-to
 :from:date:message-id:subject:to:cc;
 bh=1kro4TsNc2u5gD9ePloPjURn3UaksDqqER6TId0HiZk=;
 b=LQTBFD5qGrRr5sOkJakqQqzpwW0aMuyDEMXz9Nb+QpWkMwnAGgj4JoAAXQUbhNJigi
 Px/IinK0dYjFoDP5Ui0r3w6F1NC7UjJoNlwIA+s3Po4g/8p3YELwE4imKzxYzTCRb4kg
 ZqzytIi8VHJ2bbpqt+PWWCCTrAX5vHNsX5iD6/efHkiXKpOmBif5k9ajSMpQGCWTGXfW
 vFL8BKKM+xq2lPubF5tZzkKYYU3frX3IsU+hWBt49UvAkqI1VQZ/dGvpTWYwIhPyCVVG
 B44bDK8o1rHSMws7NaZoN2vouN5a3EkfKgcEVmIVp3BQABKEnUlg0H4/qpWYAABEgegd
 at1g==
X-Gm-Message-State: APjAAAXwMOr7eMdDypAUpQWmv0HIFIoiMvY/ilbKQJlxuZJ/9NDDOUEC
 0Go2aioE8OT1SskmvtStsxpiDoog
X-Google-Smtp-Source: APXvYqxfmURrSVSZO00sjwMGAA7tSdg3li/81csPuPTYvsqTjPlJD5qA6EIIMgApxpzcq+FkKeA5yQ==
X-Received: by 2002:a02:7009:: with SMTP id f9mr24052jac.81.1568400961147;
 Fri, 13 Sep 2019 11:56:01 -0700 (PDT)
Received: from mail-io1-f48.google.com (mail-io1-f48.google.com.
 [209.85.166.48])
 by smtp.gmail.com with ESMTPSA id h4sm27073904iok.1.2019.09.13.11.56.00
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 13 Sep 2019 11:56:00 -0700 (PDT)
Received: by mail-io1-f48.google.com with SMTP id r26so64691440ioh.8;
 Fri, 13 Sep 2019 11:56:00 -0700 (PDT)
X-Received: by 2002:a5e:c74d:: with SMTP id g13mr1408215iop.65.1568400960791; 
 Fri, 13 Sep 2019 11:56:00 -0700 (PDT)
MIME-Version: 1.0
References: <CAFMmRNwRWZsQhMSY7MwS2kU-qNJ0+Pk676mi59_Y3QsbcEmPVQ@mail.gmail.com>
 <CAGOYWV-v1-F3dWQovdj5TZMZeBFifDm020AXnPYdZSHPKLG4SA@mail.gmail.com>
 <CAFMmRNzhsk-1BO6iyJz7_qtgnRqC+fPpYSuQnxEZyno+O1K71Q@mail.gmail.com>
 <20190913183718.GC84156@raichu>
In-Reply-To: <20190913183718.GC84156@raichu>
Reply-To: cem@freebsd.org
From: Conrad Meyer <cem@freebsd.org>
Date: Fri, 13 Sep 2019 11:55:49 -0700
X-Gmail-Original-Message-ID: <CAG6CVpV03BQfu8f5Tc-bkfASCB_wiR-SNhQpsLLZf=1EHsEJdQ@mail.gmail.com>
Message-ID: <CAG6CVpV03BQfu8f5Tc-bkfASCB_wiR-SNhQpsLLZf=1EHsEJdQ@mail.gmail.com>
Subject: Re: Deadlock involving truss -f, pdfork() and wait4()
To: Mark Johnston <markj@freebsd.org>
Cc: Ryan Stone <rysto32@gmail.com>,
 FreeBSD Current <freebsd-current@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 46VPwy46Gqz40kD
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-6.00 / 15.00];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; REPLY(-4.00)[];
 TAGGED_FROM(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 18:56:02 -0000

On Fri, Sep 13, 2019 at 11:37 AM Mark Johnston <markj@freebsd.org> wrote:
>
> On Fri, Sep 13, 2019 at 02:12:56PM -0400, Ryan Stone wrote:
> > This gets me a little further but now the wait4 call by the parent
> > never reaps the child and instead blocks forever:
>
> Does it perform a wildcarded wait(), or does it explicitly specify the
> PID of the child?  By design, the former will not return children
> created by pdfork().

Explicit PID:

https://people.freebsd.org/~rstone/pdfork.c

Best,
Conrad

From owner-freebsd-current@freebsd.org  Fri Sep 13 19:04:09 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4A7D3D03B7
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 19:04:09 +0000 (UTC)
 (envelope-from rysto32@gmail.com)
Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com
 [IPv6:2607:f8b0:4864:20::732])
 (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (2048 bits) client-digest SHA256)
 (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VQ6K18Shz41DP;
 Fri, 13 Sep 2019 19:04:08 +0000 (UTC)
 (envelope-from rysto32@gmail.com)
Received: by mail-qk1-x732.google.com with SMTP id u184so26334035qkd.4;
 Fri, 13 Sep 2019 12:04:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=vzlFZh13uypuw6jx1qKWMeDx8zged9i5dMQzrKz/t/M=;
 b=rmm8Mzv2uwd870Y2zYQ/0178OKr/zaA+33/X2Ca/y/HXpiGGy7CNHtuGwChisxqxUv
 bkBHgWNTjnvS9HKgS2WqPkXY+IgUg1tSVSL4cZTdPEGGXUsgmNIu/y9zLMaDL11xuof0
 o619WwaSfDkrbX2B3uDc0tNXW7DGgkJ5oNM2j7M68VYeZPXOdmvH7sbzf/pkfe+X4Zyo
 wGIxodboUxPYEiF1zRgvJVX84WhSKiocq10ZPwa9qsYdwODUCV+EZpGa8+7xta7MG11p
 b9wYsXvPIPnAXvMNlxaA6oudHfxCoFl30o68YK1ujgQh8tpGYEp36dyzWG+9Xvk+jlC5
 0G7g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=vzlFZh13uypuw6jx1qKWMeDx8zged9i5dMQzrKz/t/M=;
 b=KQMOGB1i1xOjOBQEiIBb0OJ4hCzqDdf5eU8lXXlrN23gYhCBrBe+5RgkYCXJojvbGU
 WV2i8Dz6GjsHpyypdeKd5fi+pbLoKsC/IC1NKSO64bPHS3Ha10EqLlTpYJ0aapkB79D8
 z9brk70xufe+4wvSSSoenkttrzN8ESU15MgSOLs3ADg+7n4WEHcWKlSiqs5TV85xkcg/
 ISkBM9fdKaWVYzYrdKfMW2BhRU3PK4/YlKNmlcUWD5zLWa3R6ld5Nge8TKeqMr6W8/E8
 AseyItSyIYIsYy1qsYOZJkI0cl2v6+Rrx2PFZ5qzJthooIU39m0+yJa+zAk0Tzpn2Sht
 hNMw==
X-Gm-Message-State: APjAAAXDsvu4I0ph02QJwEzyqJbXSZxhKvzqnVRKZSxjfMztTVa/SHXr
 UIMFv3ssWojOMUjsGZcUOQyuufYW2q06GRZpMSp6m+YX
X-Google-Smtp-Source: APXvYqwCMWiFMpzXYoW9uvCi3JYhfgcHlqMILGJdcYy8oBsypg0SST/PapIWOWU2uFPzTvZxZu/2CuMbvSpo2eC/8JY=
X-Received: by 2002:a37:a843:: with SMTP id r64mr45063413qke.363.1568401447743; 
 Fri, 13 Sep 2019 12:04:07 -0700 (PDT)
MIME-Version: 1.0
References: <CAFMmRNwRWZsQhMSY7MwS2kU-qNJ0+Pk676mi59_Y3QsbcEmPVQ@mail.gmail.com>
 <CAGOYWV-v1-F3dWQovdj5TZMZeBFifDm020AXnPYdZSHPKLG4SA@mail.gmail.com>
 <CAFMmRNzhsk-1BO6iyJz7_qtgnRqC+fPpYSuQnxEZyno+O1K71Q@mail.gmail.com>
 <20190913183718.GC84156@raichu>
In-Reply-To: <20190913183718.GC84156@raichu>
From: Ryan Stone <rysto32@gmail.com>
Date: Fri, 13 Sep 2019 15:03:56 -0400
Message-ID: <CAFMmRNzZT4LFLEUY6viNttLaJs14xm1JZBzKTmpKgAhjNZOUtw@mail.gmail.com>
Subject: Re: Deadlock involving truss -f, pdfork() and wait4()
To: Mark Johnston <markj@freebsd.org>
Cc: Mariusz Zaborski <oshogbo@freebsd.org>,
 FreeBSD Current <freebsd-current@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 46VQ6K18Shz41DP
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-6.00 / 15.00];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 19:04:09 -0000

As Conrad has pointed out, it's an explicit PID.  The test completes
successfully when not run under truss -f.

On Fri, Sep 13, 2019 at 2:37 PM Mark Johnston <markj@freebsd.org> wrote:
>
> On Fri, Sep 13, 2019 at 02:12:56PM -0400, Ryan Stone wrote:
> > This gets me a little further but now the wait4 call by the parent
> > never reaps the child and instead blocks forever:
>
> Does it perform a wildcarded wait(), or does it explicitly specify the
> PID of the child?  By design, the former will not return children
> created by pdfork().

From owner-freebsd-current@freebsd.org  Fri Sep 13 23:39:14 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3DE07D64FB
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Fri, 13 Sep 2019 23:39:14 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits)
 client-signature RSA-PSS (4096 bits) client-digest SHA256)
 (Client CN "smtp.freebsd.org",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VXCk0hZJz4FnS;
 Fri, 13 Sep 2019 23:39:14 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Received: from mousie.catspoiler.org (unknown [76.212.85.177])
 (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 (Authenticated sender: truckman)
 by smtp.freebsd.org (Postfix) with ESMTPSA id 7448911144;
 Fri, 13 Sep 2019 23:39:13 +0000 (UTC)
 (envelope-from truckman@FreeBSD.org)
Date: Fri, 13 Sep 2019 16:39:10 -0700 (PDT)
From: Don Lewis <truckman@FreeBSD.org>
Subject: Re: spurious out of swap kills
To: Mark Johnston <markj@freebsd.org>
cc: FreeBSD Current <freebsd-current@freebsd.org>, kib@freebsd.org
In-Reply-To: <tkrat.1a0e98a230c1a223@FreeBSD.org>
Message-ID: <tkrat.e0c4dd73c3a5b2fe@FreeBSD.org>
References: <tkrat.84b3295682c83162@FreeBSD.org>
 <20190913000635.GG8397@raichu> <tkrat.1a0e98a230c1a223@FreeBSD.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; CHARSET=us-ascii
Content-Disposition: INLINE
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Sep 2019 23:39:14 -0000

On 12 Sep, Don Lewis wrote:
> On 12 Sep, Mark Johnston wrote:
>> On Thu, Sep 12, 2019 at 04:00:17PM -0700, Don Lewis wrote:
>>> My poudriere machine is running 13.0-CURRENT and gets updated to the
>>> latest version of -CURRENT periodically.  At least in the last week or
>>> so, I've been seeing occasional port build failures when building my
>>> default set of ports, and I finally had some time to do some
>>> investigation.
>>> 
>>> It's a 16-thread Ryzen machine, with 64 GB of RAM and 40 GB of swap.
>>> Poudriere is configured with
>>>   USE_TMPFS="wrkdir data localbase"
>>> and I have
>>>   .if ${.CURDIR:M*/www/chromium}
>>>   MAKE_JOBS_NUMBER=16
>>>   .else
>>>   MAKE_JOBS_NUMBER=7
>>>   .endif
>>> in /usr/local/etc/poudriere.d/make.conf, since this gives me the best
>>> overall build time for my set of ports.  This hits memory pretty hard,
>>> especially when chromium, firefox, libreoffice, and both versions of
>>> openoffice are all building at the same time.  During this time, the
>>> amount of space consumed by tmpfs for /wrkdir gets large when building
>>> these large ports.  There is not enough RAM to hold it all, so some of
>>> the older data spills over to swap.  Swap usage peaks at about 10 GB,
>>> leaving about 30 GB of free swap.  Nevertheless, I see these errors,
>>> with rustc being the usual victim:
>>> 
>>> Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, was killed: out of swap space
>>> Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, was killed: out of swap space
>>> 
>>> Top shows the size of rustc being about 2 GB, so I doubt that it
>>> suddenly needs an additional 30 GB of swap.
>>> 
>>> I'm wondering if there might be a transient kmem shortage that is
>>> causing a malloc(..., M_NOWAIT) failure in the swap allocation path
>>> that is the cause of the problem.
>> 
>> Perhaps this is a consequence of r351114?  To confirm this, you might
>> try increasing the value of vm.pfault_oom_wait to a larger value, like
>> 20 or 30, and see if the OOM kills still occur.
> 
> I wonder if increasing vm.pfault_oom_attempts might also be a good idea.

sysctl vm.pfault_oom_attempts=10 by itself seemed to work well.



From owner-freebsd-current@freebsd.org  Sat Sep 14 06:00:07 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id C4835E8700
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 06:00:07 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic309-15.consmr.mail.bf2.yahoo.com
 (sonic309-15.consmr.mail.bf2.yahoo.com [74.6.129.125])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VhgB5FRBz4Y9l
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 06:00:06 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: LzuxDPwVM1nZysMYI6qdNL2WhSIxhlL6.iUUJjOg2q50oQkptBuQaaahKUmBGlW
 JLE2ErGM.knpr88moy6hcVfpqQafBjkfCVk3bOaV.CrWYoAeGJQbmd6F70vgnRQNx6Dlizh.jt0Q
 cYJbUGvVmoGxuGTcCeYpqpgd5CROPUI8jyakTLP1jRsia9XmrWmmY0GecnHEZzj89y4m5nim07tj
 QJLX64nffurNcRMRDoOzzwev_udCthDxACHr9f7BW_e2IA2aGihifob0JC.5Z036UuDNuZQeQwhI
 F_KzDEHyQSgTPMfT8zluSQCQQmAex9KPjEzau4VzJY3mxjLjvR4DglTyxbFBYu5H8kkKf5HSZPvG
 o881Ro.3mLKohXSdrGL.RUjA2B5nTAMbrWu1MRxwdtMvPDFsEWTox.8PvqIz9UWdSKLckszrtTGf
 4vqOUerIvzB7lfcuomC7lQpb4xBTTY49ehI0RDwGlTfWWnRXSsKsGP31f5ue_WyiH0NG1i.8kf2B
 DsTM75XNawRd7riAt7jcjLo0N58szHjb.9EYPOVQpZQnJeon98deQBPIsds9FstgMXDr2WaVivZM
 Q0ZDRT44NZh4CGYcrP1MrwlgdR4elCf_JZ8WiG0l7G9ngmrdSpXUuQGwfg1pYViRPEfKEd1pXeHs
 6UimJBt0ePFo0hu_Q26TvmoNuaDuIs0A7X09d_pIXzMuz19D.hKg6LbWLL2iPpjOtEwyefC0Ghxm
 6VJ25pM0585Smv4MrSPDsj9G0x9OLkqRvGgo9BPgCT1NHS7Jpou7psffsuOliQrRdvTLxqc6.DOL
 zqfNlt2AJFaIZz0N0q0F30z3TL_7lw0QYUcP4UMqcYSunzz_fymioJjg3ZgzaZPls00QPC6K8yIq
 kuqWz9vGbH5aIR5guN_CGkOGizRnoH0nUh4UBO.Nij9iS5QhK53LayCgNZ_3Q_VbFtQVSvKBfPAo
 aNh3khxvjbQpbsX3oRgI.cSGsUSsseaKL0JT6C8z41XpDbwvyve0V6FNuse6RfZR9D68ZPT_IffY
 iyZfPWP1b9XfxrW9LkxVNFiRoo1tHlrC8G1lwSWvuCWn.5V.mAVQRolpuC8wm6_3fKKzcCO5tJFa
 Z_SOhKnqas4pyccK6l3nu99v2rHaZXje5GOVWCE.2HlLF2AAYJHdRycwzHT.8LfeL7u7Jjvo5rCv
 2qelU7IpPOKEnEB8GJG45ahh6sXPSB2hJjeAG8h5IJtg_TxPZwJP9OfNg0TSibyccn.XCksyCHJc
 garohi2RkN3uBxhit8XITTbFHdLQZff02FZeMfD7PIfMxpDUJ1Z2GSW4j8A--
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic309.consmr.mail.bf2.yahoo.com with HTTP; Sat, 14 Sep 2019 06:00:05 +0000
Received: by smtp407.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID 6ba0ac3b58b1275bd760445b71d0f5e0; 
 Sat, 14 Sep 2019 06:00:00 +0000 (UTC)
From: Mark Millard <marklmi@yahoo.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: spurious out of swap kills
Message-Id: <28BF21DA-8B16-4CD8-8E5E-C1B596FE3684@yahoo.com>
Date: Fri, 13 Sep 2019 22:59:58 -0700
To: bob prohaska <fbsd@www.zefox.net>,
 FreeBSD Current <freebsd-current@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
X-Rspamd-Queue-Id: 46VhgB5FRBz4Y9l
X-Spamd-Bar: /
X-Spamd-Result: default: False [0.72 / 15.00];
 R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[];
 TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[];
 MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; FAKE_REPLY(1.00)[];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048];
 NEURAL_HAM_MEDIUM(-0.04)[-0.039,0]; FROM_HAS_DN(0.00)[];
 MIME_GOOD(-0.10)[text/plain];
 IP_SCORE(0.00)[ip: (3.36), ipnet: 74.6.128.0/21(1.45), asn: 26101(1.16),
 country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.26)[0.256,0];
 RCVD_IN_DNSWL_NONE(0.00)[125.129.6.74.list.dnswl.org : 127.0.5.0];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 06:00:07 -0000

bob prohaska fbsd at www.zefox.net wrote on
Fri Sep 13 16:24:57 UTC 2019 :

> Not sure this is relevant, but in compiling chromium on a Pi3 with 6 GB
> of swap the job completed successfully some months ago, with peak swap 
> use around 3.5 GB. The swap layout was sub-optimal, with a 2 GB partition
> combined with a 4 GB partition. A little over 4GB total seems usable. 
> 
> A few days ago the same attempt stopped with a series of OOMA kills,
> but in each case simply restarting allowed the compile to pick up
> where it left off and continue, eventually finishing with a runnable
> version of chromium. In this case swap use peaked a little over 4 GB.
> 
> Might this suggest the machine isn't freeing swap in a timely manner?

Are you saying that your increases to:

vm.pageout_oom_seq

no longer prove sufficient? What value for vm.pageout_oom_seq were
you using that got the recent failures?

(Mark Johnston's slow_swap.diff patch [and related] for investigations
of some OOM killing contributions never became official and has not
been updated to apply to updated code. It has been over a year since
those patches were used for the arm small board computer investigations
that lead to my learning about vm.pageout_oom_seq .)

If more or different configuration/tuning is required, I'm going to
eventually want to learn about it as well.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)


From owner-freebsd-current@freebsd.org  Sat Sep 14 06:14:59 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 80C75E8FB4
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 06:14:59 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic303-1.consmr.mail.bf2.yahoo.com
 (sonic303-1.consmr.mail.bf2.yahoo.com [74.6.131.40])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46Vj0L2qYlz4ZJ2
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 06:14:58 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: 2CDQbBUVM1kB8cZRdmfDXalU5zBJAVMcClER97ud17rlTarm9uOuRKQD6KXGrOI
 svjQzNla8SdUD12RjmNOmGMtyfTlzC3wu3NpawZiCg3OT.s_JJwM5S69_V6iu4r6J7ROjfKTpyEG
 CzK07I7Hg3f.NW2gtNycgh_cwogv45UyRqs17EuLXXK6xISr_NgpuAkD1ZjjRQcaaFJFpyZsnYyd
 bJbcciPIx4PnhGvAyv34y_YDt1ny0FAQWPJBuKU0WghoIT5L37kZ4w1ywLye1Cf97KcqH6pFJajT
 X7XdFWZ1r2rE_pyCE_ttxecEBgFt2scAt1F8qnwHNIr2d6KJuT7.Okp1scMw5B54zGOLibXjdFc3
 VDyvqz5D9wIrAqzc_HkdNOp115cq_830suQ2Lu01_Bz3bdbO6GPAawSoR.uT9H.BZwBpaaYYlmmG
 oeV1qyEImtnVfJ0rpwM.qmDoCKVnXyZTo1XDbzxhuLXT82VFEqa5TUZPQT2GuhMZNsZEIGiG.gy9
 9oUxmjVzmPABBFFL9.BcDNkh0_.8F4W06VOtZzO0cScaVymsViBW4r0wTTedAKHc28wp9MLIbdlJ
 GArj3VOrJTOPSB1cidDmPbavCUlSkEqN1epsdcw37mXxMyTdRUte9FKa3BQ.dy_86xe6PCJ2kpDp
 DXfIuEZ_MAHu9IuhorbYLUUrwMxFmlV1_0SfgvmrX6xVKamnX2pslULj4NahCJvpnm6XIC.Yej2N
 C1vaZD8t1tc.B6WMgL_6hEtfRxhLoGZ83.EAAQ8ujXI1ghV8.LQTcUSBUB7ubQ4o9dHmZpNWGlSB
 qICcHqXRaoAx0HqmDbyGafDImdCIKFffApe5YAoEMaIyPwED9o2hUeDdH0jolAwxGaQPM1lGjjLf
 N_nmdP.q4uWtYWkv9vw9YFyJgJ6O3ynWpBwXhZLEvRcKkg4Vj9mOohiXgLjn3wYTazW6GwElPeif
 Py1zsWl5NNKCb2iG2H_LUJERaAJjpKZ4PyBQHJFj.GMmsYw7dir8lsXdtiAjszrGVNlNaP9cVqxO
 nfj.xLtZp1dfAs1apnAgJMdZqt7Ng4_bgNI.tsTQUQ4ptUbVzEPFldOY1Uv0JaOLC6_PO2WI5_eS
 taHBejvi7D9yCaztZksT2UxOCzn1VyM6h.uHNB3HrUjCjIijKKEn2JPVthifnZsYN7AQ2ZlPxQqK
 _f5EYy48FfxE.x3ii_uue6CsPzXBk2jGoVnDxLnn6sX5SwdZf7.e0WgFM.TQ_Ph36N_ZbUju5q2O
 GzYEKOTqQ0pfeUrUib3e7pmopL5P0GMe3bWHfkwh6kwrFevHPINA8_O_nFLM-
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic303.consmr.mail.bf2.yahoo.com with HTTP; Sat, 14 Sep 2019 06:14:57 +0000
Received: by smtp426.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID 73493a2f06f187fec621f15ed484febd; 
 Sat, 14 Sep 2019 06:14:53 +0000 (UTC)
From: Mark Millard <marklmi@yahoo.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: spurious out of swap kills
Message-Id: <9A2342FC-D168-485A-9918-0109F9CD1646@yahoo.com>
Date: Fri, 13 Sep 2019 23:14:50 -0700
To: " truckman@freebsd.org " <truckman@FreeBSD.org>,
 FreeBSD Current <freebsd-current@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
X-Rspamd-Queue-Id: 46Vj0L2qYlz4ZJ2
X-Spamd-Bar: /
X-Spamd-Result: default: False [0.65 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[];
 TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[];
 DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[];
 MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; FAKE_REPLY(1.00)[];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048];
 NEURAL_HAM_MEDIUM(-0.08)[-0.075,0]; FROM_HAS_DN(0.00)[];
 MIME_GOOD(-0.10)[text/plain];
 IP_SCORE(0.00)[ip: (3.46), ipnet: 74.6.128.0/21(1.45), asn: 26101(1.16),
 country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.23)[0.230,0];
 RCVD_IN_DNSWL_NONE(0.00)[40.131.6.74.list.dnswl.org : 127.0.5.0];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 06:14:59 -0000

Don Lewis truckman at FreeBSD.org wrote on
Thu Sep 12 23:00:19 UTC 2019 :

. . .
> Nevertheless, I see these errors,
> with rustc being the usual victim:
>=20
> Sep 11 23:21:43 zipper kernel: pid 16581 (rustc), jid 43, uid 65534, =
was killed: out of swap space
> Sep 12 02:48:23 zipper kernel: pid 1209 (rustc), jid 62, uid 65534, =
was killed: out of swap space
. . .

Unfortunately, the wording of this type of message is a misnomer for
what drives the kills: it is actually driven by being unable to gain
more free memory but FreeBSD will not swap-out processes that stay
runnable (or are running), only ones that are waiting. Even a single
process that stays runnable and keeps lots of RAM in the active
category can lead to kills when swap is unused or little used. So the
kill-behavior is very workload dependent.

Real "out of swap" conditions tend to also have messages
similar to:

Aug  5 17:54:01 sentinel kernel: swap_pager_getswapspace(32): failed

If you are not seeing such messages, then it is likely that
the mount of swap space still free is not the actual thing
driving the kills.

Are you seeing "swap_pager_getswapspace(32): failed" messages?

(It used to be that the system simply leaves the dirty pages in
memory when a swap_pager_getswapspace failed message is produced.
Of itself, it did not cause a kill. I do not know about now.)

=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)


From owner-freebsd-current@freebsd.org  Sat Sep 14 16:10:19 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id E906BF8A48;
 Sat, 14 Sep 2019 16:10:19 +0000 (UTC)
 (envelope-from fbsd@www.zefox.net)
Received: from www.zefox.net (www.zefox.net [50.1.20.27])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "www.zefox.org", Issuer "www.zefox.org" (not verified))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46VyCG4jnSz48ck;
 Sat, 14 Sep 2019 16:10:18 +0000 (UTC)
 (envelope-from fbsd@www.zefox.net)
Received: from www.zefox.net (localhost [127.0.0.1])
 by www.zefox.net (8.15.2/8.15.2) with ESMTPS id x8EGAFWq033684
 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
 Sat, 14 Sep 2019 09:10:16 -0700 (PDT)
 (envelope-from fbsd@www.zefox.net)
Received: (from fbsd@localhost)
 by www.zefox.net (8.15.2/8.15.2/Submit) id x8EGAFoo033683;
 Sat, 14 Sep 2019 09:10:15 -0700 (PDT) (envelope-from fbsd)
Date: Sat, 14 Sep 2019 09:10:14 -0700
From: bob prohaska <fbsd@www.zefox.net>
To: Mark Millard <marklmi@yahoo.com>
Cc: FreeBSD Current <freebsd-current@freebsd.org>, freebsd-arm@freebsd.org
Subject: Re: spurious out of swap kills
Message-ID: <20190914161014.GA33442@www.zefox.net>
References: <28BF21DA-8B16-4CD8-8E5E-C1B596FE3684@yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <28BF21DA-8B16-4CD8-8E5E-C1B596FE3684@yahoo.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-Rspamd-Queue-Id: 46VyCG4jnSz48ck
X-Spamd-Bar: ++
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=none (mx1.freebsd.org: domain of fbsd@www.zefox.net has no SPF policy when
 checking 50.1.20.27) smtp.mailfrom=fbsd@www.zefox.net
X-Spamd-Result: default: False [2.01 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-0.15)[-0.150,0]; WWW_DOT_DOMAIN(0.50)[];
 RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[];
 RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[];
 IP_SCORE(0.08)[ip: (0.34), ipnet: 50.1.16.0/20(0.17), asn: 7065(-0.05),
 country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain];
 DMARC_NA(0.00)[zefox.net]; AUTH_NA(1.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.18)[0.178,0];
 R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[yahoo.com];
 FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US];
 MID_RHS_MATCH_FROM(0.00)[]; MID_RHS_WWW(0.50)[];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 16:10:20 -0000

On Fri, Sep 13, 2019 at 10:59:58PM -0700, Mark Millard wrote:
> bob prohaska fbsd at www.zefox.net wrote on
> Fri Sep 13 16:24:57 UTC 2019 :
> 
> > Not sure this is relevant, but in compiling chromium on a Pi3 with 6 GB
> > of swap the job completed successfully some months ago, with peak swap 
> > use around 3.5 GB. The swap layout was sub-optimal, with a 2 GB partition
> > combined with a 4 GB partition. A little over 4GB total seems usable. 
> > 
> > A few days ago the same attempt stopped with a series of OOMA kills,
> > but in each case simply restarting allowed the compile to pick up
> > where it left off and continue, eventually finishing with a runnable
> > version of chromium. In this case swap use peaked a little over 4 GB.
> > 
> > Might this suggest the machine isn't freeing swap in a timely manner?
> 
> Are you saying that your increases to:
> 
> vm.pageout_oom_seq
> 
> no longer prove sufficient? What value for vm.pageout_oom_seq were
> you using that got the recent failures?
> 
Correct. Initial value was 2048, later raised to 4096. Far as I could
tell the change didn't help. No explict j value was set for make, but
no more than four jobs were observed in top 

A log of storage activity along with swap total and the last two 
console messages is at
http://www.zefox.net/~fbsd/rpi3/swaptests/r351586/swapscript.log
along with a sorted list of total swap use, which can be used as
a sort of index to the log file. 

The initial "out of swap space" at the very beginning
is a relic from before logging started. 

Da0 is a Sandisk SDCZ80 usb 3.0 device, mmcsd0 is a Samsung
Evo + 128 GB device.

The two points of curiosity to me are:
1. Why did swap use increase from 3.5 GB months ago to 4.2 GB now?
2. Why does stopping and restarting make (which would seem to free
un-needed swap) allow the job to finish?

> If more or different configuration/tuning is required, I'm going to
> eventually want to learn about it as well.
> 
You will have some company.

Thanks for reading,

bob prohaska


From owner-freebsd-current@freebsd.org  Sat Sep 14 17:38:17 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49DC3FA65C
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 17:38:17 +0000 (UTC) (envelope-from lists@opsec.eu)
Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46W08m3jzNz4DrN
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 17:38:16 +0000 (UTC)
 (envelope-from lists@opsec.eu)
Received: from pi by home.opsec.eu with local (Exim 4.92.2 (FreeBSD))
 (envelope-from <lists@opsec.eu>) id 1i9Bzp-000MoT-ID
 for freebsd-current@freebsd.org; Sat, 14 Sep 2019 19:38:05 +0200
Date: Sat, 14 Sep 2019 19:38:05 +0200
From: Kurt Jaeger <lists@opsec.eu>
To: FreeBSD Current <freebsd-current@freebsd.org>
Subject: poudriere, swap full and top says memory is free ?
Message-ID: <20190914173805.GC2863@home.opsec.eu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-Rspamd-Queue-Id: 46W08m3jzNz4DrN
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of lists@opsec.eu designates
 2001:14f8:200::1 as permitted sender) smtp.mailfrom=lists@opsec.eu
X-Spamd-Result: default: False [-4.94 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
 R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[];
 MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[];
 DMARC_NA(0.00)[opsec.eu]; RCPT_COUNT_ONE(0.00)[1];
 NEURAL_HAM_LONG(-1.00)[-1.000,0];
 IP_SCORE(-3.64)[ip: (-9.62), ipnet: 2001:14f8::/32(-4.77), asn: 12502(-3.82),
 country: DE(-0.01)]; TO_DN_ALL(0.00)[];
 FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
 SUBJECT_ENDS_QUESTION(1.00)[];
 ASN(0.00)[asn:12502, ipnet:2001:14f8::/32, country:DE];
 MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 17:38:17 -0000

Hi!

I'm running

- a poudriere build
- of a list of ports
- on 12.0-RELEASE-p10
- on a 4 core+4 hyperthreads CPU, an Intel(R) Xeon(R) CPU E3-1230 v6
  @ 3.50GHz
- with 32 GB RAM
- zpool with 2x 500 GB SSDs as a mirror

and right now, this can be seen:

last pid: 90922;  load averages:  5.02,  5.14,  5.73    up 0+03:53:08  19:31:05
82 processes:  6 running, 76 sleeping
CPU: 60.6% user,  0.0% nice,  2.1% system,  0.0% interrupt, 37.3% idle
Mem: 4598M Active, 2854M Inact, 11G Laundry, 6409M Wired, 6375M Free
ARC: 3850M Total, 1721M MFU, 2090M MRU, 665K Anon, 19M Header, 19M Other
     3406M Compressed, 3942M Uncompressed, 1.16:1 Ratio
Swap: 18G Total, 18G Used, 396K Free, 99% Inuse, 68K In

So: Swap is full, approx. 6 GB memory is reported as free.

This is surprising. Can I somehow tune this in any way, so that
the memory available is used for the build ? Or is the problem somewhere
else ?

Running similar builds on 12.0 without patches reported
swap_pager_getswapspace(24): failed
messages.

-- 
pi@opsec.eu            +49 171 3101372                    One year to go !

From owner-freebsd-current@freebsd.org  Sat Sep 14 18:05:13 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id CDB8BFAECA
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 18:05:13 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic307-9.consmr.mail.ne1.yahoo.com
 (sonic307-9.consmr.mail.ne1.yahoo.com [66.163.190.32])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46W0lr3q1Bz4G8H
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 18:05:12 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: U7vAjOkVM1nitrfSD_zwoorP53o4qBG7qg2dCG3c0SY5xRf_GW66FMnL0QgKPJ2
 pP4OPj2QlpnuZrFfE_2aETCw22J_ekkSfgqF8dY6Y2SY_DZD1eq708adQERxawvMdATryDYL6aUQ
 ZDM8K21WCYzffPjdqRUdwEd9OvYo0f5gT0pWEIiF7T7wBzvTUYfzJpSnGWyYvYypa3QGwNe_ax3j
 P3cZnhpFc3k2p3.ip0xAbXqksMw6sBdsxWYDw5Cp4QMBUj.iNYFtdQbiF4cxumhzsx7YvYSce3N8
 s.Bw4pWBdTtduIAqBhFYD9YyWSwjI9ctXLT0WlmtcmXN8KP_UTUl1wwsWT9iuX.0vpi_8CU7ceK8
 78haZdNOfQFyVx83Rod7YVShCYHfZ0SU5eOVIaUbviOVDABNHnBAajXmKZhim70N3moW4ASMp998
 RTcy6nad2dtG1fK4V2dTEkBqXIGIvMEw.C5B6f2Zfql1eLAb5OPqKImtQ5yP6.sXliJShf232tBy
 dI2mqgwEatnHtqQfy3nNdlBpMNstdsZNPEddXx7Jw3jXoi0eEXK1aD3H7lvvq00udlPyNEN15Zdg
 4hadZu7qog5haqFNeiG7RrL7sqk_dlcsSvoi5MXbSZVNbbkQwm33d0Yv5eGi6Hd3bRdC5XVw.aja
 ZokYlc2EtoNBR8WaEdW0Oan6tH4nu1jb0Lr7ixkEw8fKugZjaD9RmyfEemZfFi8Yf0j8Ni5rtYoI
 9KeKpcWlV2xcq3hAeEqM5gbut9nkVsbupNlR8lJLofEQmpCLw6jCz5UkoSSQuCvao5730b.RYtGx
 9MBcliC.qTe8TKwL8OI4UAUi5SqwyrdodnECPE6xgW8LdZIazfj.4zYvJfboCKhhDHGPAergKo5Y
 tkmhhb90IkEuNFWsOJtko1cxEl1x9t_On5vFfucOjWhjjzhTDiSdYZPlxpvm8xCVVAsUww68yGMm
 WFmyyqBJVafmubYz.Ir_HtCCCUeJXTQk5ConK9TCp1DFgVN2kFltCUo1lZDq53l.fZFmX44R5.PT
 L_TAbrrrvFGwuQ69nl_4dfPngGTDZ69RLH_JS2tGTyDx7_NMaKagPh1AMhetkUC.EQyWrZ_8bi1i
 l._yVjVRRdLbxYqsVPP99RNOKth4eSk2PbJGtw4Nvxx0m_Ilan0ZPbBNs7Ig2mTfAk0IqrBDfIh7
 HiJq.4NNiJXKrN_hQBF59oMvSG8A_suDdsY5uh9E.IXyV8y5lwrluNWLqIWrdgsBH15vhU..Iwms
 Wq4mR_nCRw397AvTNppkZPVDbO9Cmr6DGPcRIenN4NUu5B07I8L8PWaaV0g--
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic307.consmr.mail.ne1.yahoo.com with HTTP; Sat, 14 Sep 2019 18:05:10 +0000
Received: by smtp420.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID 13b11ce7938a56cd8c0fc711236cd56c; 
 Sat, 14 Sep 2019 18:05:10 +0000 (UTC)
From: Mark Millard <marklmi@yahoo.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: head -r352274 buildkernel targetting armv7 failure:
 am335x/am335x_dmtpps.c:304:3: error: implicit declaration of function
 'spinlock_enter' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
Message-Id: <0E884723-2826-4EDF-A16F-841E01E9D4EC@yahoo.com>
Date: Sat, 14 Sep 2019 11:05:08 -0700
To: FreeBSD Current <freebsd-current@freebsd.org>,
 freebsd-arm@freebsd.org
X-Mailer: Apple Mail (2.3445.104.11)
X-Rspamd-Queue-Id: 46W0lr3q1Bz4G8H
X-Spamd-Bar: --
X-Spamd-Result: default: False [-2.29 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-0.88)[-0.879,0];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[];
 TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MIME_GOOD(-0.10)[text/plain];
 MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.91)[-0.914,0];
 IP_SCORE_FREEMAIL(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+];
 RCPT_COUNT_TWO(0.00)[2];
 RCVD_IN_DNSWL_NONE(0.00)[32.190.163.66.list.dnswl.org : 127.0.5.0];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[];
 MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 IP_SCORE(0.00)[ip: (4.06), ipnet: 66.163.184.0/21(1.30), asn: 36646(1.04),
 country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 18:05:13 -0000

After updating my amd64 context to head -r352274,
attempting an amd64->armv7 cross buildworld buildkernel
ended up failing with:


--- am335x_dmtpps.o ---
/usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: implicit =
declaration of function 'spinlock_enter' is invalid in C99 =
[-Werror,-Wimplicit-function-declaration]
                mtx_lock_spin(&sc->pps_mtx);
                ^
/usr/src/sys/sys/mutex.h:383:26: note: expanded from macro =
'mtx_lock_spin'
#define mtx_lock_spin(m)        mtx_lock_spin_flags((m), 0)
                                ^
/usr/src/sys/sys/mutex.h:452:2: note: expanded from macro =
'mtx_lock_spin_flags'
        mtx_lock_spin_flags_((m), (opts), LOCK_FILE, LOCK_LINE)
        ^
/usr/src/sys/sys/mutex.h:429:2: note: expanded from macro =
'mtx_lock_spin_flags_'
        __mtx_lock_spin((m), curthread, (opts), (file), (line))
        ^
/usr/src/sys/sys/mutex.h:258:2: note: expanded from macro =
'__mtx_lock_spin'
        spinlock_enter();                                               =
\
        ^
/usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: this function =
declaration is not a prototype [-Werror,-Wstrict-prototypes]
/usr/src/sys/sys/mutex.h:383:26: note: expanded from macro =
'mtx_lock_spin'
#define mtx_lock_spin(m)        mtx_lock_spin_flags((m), 0)
                                ^
/usr/src/sys/sys/mutex.h:452:2: note: expanded from macro =
'mtx_lock_spin_flags'
        mtx_lock_spin_flags_((m), (opts), LOCK_FILE, LOCK_LINE)
        ^
/usr/src/sys/sys/mutex.h:429:2: note: expanded from macro =
'mtx_lock_spin_flags_'
        __mtx_lock_spin((m), curthread, (opts), (file), (line))
        ^
/usr/src/sys/sys/mutex.h:258:2: note: expanded from macro =
'__mtx_lock_spin'
        spinlock_enter();                                               =
\
        ^
. . .

(spinlock_enter was not the only example.)

=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)


From owner-freebsd-current@freebsd.org  Sat Sep 14 18:21:24 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id E7A18FB6F2
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 18:21:24 +0000 (UTC) (envelope-from ian@freebsd.org)
Received: from outbound3d.ore.mailhop.org (outbound3d.ore.mailhop.org
 [54.186.57.195])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46W16X4Ljkz4H5p
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 18:21:24 +0000 (UTC)
 (envelope-from ian@freebsd.org)
ARC-Seal: i=1; a=rsa-sha256; t=1568485283; cv=none;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 b=klFW3yvkPpzOyHUmvbCfgUzkP8BW6GdjNZLdF+Cnmlw6aSj0ZMBID9EjKLmIgPBzbyzaYYTCBUDSS
 tbdISjf0O3PkpmF++jEUxJpNKcbGeyIVa7t03ZS+j9MzSuWznU1zrKr1eY1N+cFkJBWMb5ZIjY9YxN
 wZ944662r4FF+tGUHQmLOZ6VikI+4TJUsUJfN4tvm2k7q4uuypEQScMiGWKgg8zQjcUr8D9wCiUTV/
 9JEpNnTmlkfmNxBuHZgJyJ9pP60xcFU6aa3LsjoL8U/EH0lhMicOfR5uQAEBpOZYNh+erqUbiFwQWv
 16sUqAI7VczLf5nOR1RrroR6NEVqBig==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=arc-outbound20181012;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:to:from:subject:message-id:dkim-signature:from;
 bh=WrGudQQQp/k6tQy/rIiXJ/OP84sFSGnd9aCvHMicsV0=;
 b=IV26p3+t6NoU1EGtppKkQhASR9yigUwqrQckijUDHCOR4rThfml62Tp9EtAXrrKG+eEJ4jcOsMvmr
 fnSF+i6RrspVSw/juAqVrosb/B445YIf1TAz3yrysQrWCMTmCRRc85+W6ybKrhGPWUhxttC6LuIpzV
 smPawJP8jQyKK+2VBzAuRis+ce5JNSnOlnAJGbd9XpnQ0DZFO+OXzkBgwLL/pu2EwcQiwHlVVsDCxK
 b91xovBajrC5LgbLtPB338pJ119BYYIbjnPvhhkUjfCc8lSdYTk0Ueb0jCWhMdqrpDbVjimu4BjjVF
 ZVqdRkI9Pe478pB+saV8VGSpatYZN5g==
ARC-Authentication-Results: i=1; outbound3.ore.mailhop.org;
 spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60;
 dmarc=none header.from=freebsd.org;
 arc=none header.oldest-pass=0;
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=outbound.mailhop.org; s=dkim-high;
 h=content-transfer-encoding:mime-version:content-type:references:in-reply-to:
 date:to:from:subject:message-id:from;
 bh=WrGudQQQp/k6tQy/rIiXJ/OP84sFSGnd9aCvHMicsV0=;
 b=UAenpQBp5VLn9MiUk04UsW9iUc3A1O+/f6Sbx88h7l9nob9BTFg+uA+ZiqEfimCRkg9rrEi6ang9I
 qmqG09c1eRnWhQRCx7cvKXg8n0julWCm1+T1Sd5BVrkR4w6xcrW81tSWZfS6IF4if1gT5wccUU09Cq
 T2SKD68xeNJEiQR3SrKh61mEWrQIYmNdct6/XJHOypcQKrOMCxmtrxEM+7Y3DetAjEhUFbX789LTqm
 BlMa1af8Bj7EqzMNYbmquuCXLvrplplcolA/LZV/yZladOc0AAawarV5S8T1i2Ja6ZZR0by79XxceE
 mcTDQPQVH6QbNBg8TywD9EmTwrWo4Pg==
X-MHO-RoutePath: aGlwcGll
X-MHO-User: 734be60b-d71c-11e9-b67d-cdd75d6ce7a8
X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information
X-Originating-IP: 67.177.211.60
X-Mail-Handler: DuoCircle Outbound SMTP
Received: from ilsoft.org (unknown [67.177.211.60])
 by outbound3.ore.mailhop.org (Halon) with ESMTPSA
 id 734be60b-d71c-11e9-b67d-cdd75d6ce7a8;
 Sat, 14 Sep 2019 18:21:21 +0000 (UTC)
Received: from rev (rev [172.22.42.240])
 by ilsoft.org (8.15.2/8.15.2) with ESMTP id x8EILK7m087044;
 Sat, 14 Sep 2019 12:21:20 -0600 (MDT) (envelope-from ian@freebsd.org)
Message-ID: <e675e9577469510f1c3145e1910b34745a9475fd.camel@freebsd.org>
Subject: Re: head -r352274 buildkernel targetting armv7 failure:
 am335x/am335x_dmtpps.c:304:3: error: implicit declaration of function
 'spinlock_enter' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
From: Ian Lepore <ian@freebsd.org>
To: Mark Millard <marklmi@yahoo.com>, FreeBSD Current
 <freebsd-current@freebsd.org>, freebsd-arm@freebsd.org
Date: Sat, 14 Sep 2019 12:21:20 -0600
In-Reply-To: <0E884723-2826-4EDF-A16F-841E01E9D4EC@yahoo.com>
References: <0E884723-2826-4EDF-A16F-841E01E9D4EC@yahoo.com>
Content-Type: text/plain; charset="UTF-8"
X-Mailer: Evolution 3.28.5 FreeBSD GNOME Team
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 46W16X4Ljkz4H5p
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-2.00 / 15.00];
 local_wl_from(0.00)[freebsd.org];
 NEURAL_HAM_MEDIUM(-1.00)[-0.999,0];
 ASN(0.00)[asn:16509, ipnet:54.186.0.0/15, country:US];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 18:21:25 -0000

On Sat, 2019-09-14 at 11:05 -0700, Mark Millard via freebsd-arm wrote:
> After updating my amd64 context to head -r352274,
> attempting an amd64->armv7 cross buildworld buildkernel
> ended up failing with:
> 
> 
> --- am335x_dmtpps.o ---
> /usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: implicit
> declaration of function 'spinlock_enter' is invalid in C99 [-Werror,-
> Wimplicit-function-declaration]
>                 mtx_lock_spin(&sc->pps_mtx);
>                 ^
> /usr/src/sys/sys/mutex.h:383:26: note: expanded from macro
> 'mtx_lock_spin'
> #define mtx_lock_spin(m)        mtx_lock_spin_flags((m), 0)
>                                 ^
> /usr/src/sys/sys/mutex.h:452:2: note: expanded from macro
> 'mtx_lock_spin_flags'
>         mtx_lock_spin_flags_((m), (opts), LOCK_FILE, LOCK_LINE)
>         ^
> /usr/src/sys/sys/mutex.h:429:2: note: expanded from macro
> 'mtx_lock_spin_flags_'
>         __mtx_lock_spin((m), curthread, (opts), (file), (line))
>         ^
> /usr/src/sys/sys/mutex.h:258:2: note: expanded from macro
> '__mtx_lock_spin'
>         spinlock_enter();                                            
>    \
>         ^
> /usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: this
> function declaration is not a prototype [-Werror,-Wstrict-prototypes]
> /usr/src/sys/sys/mutex.h:383:26: note: expanded from macro
> 'mtx_lock_spin'
> #define mtx_lock_spin(m)        mtx_lock_spin_flags((m), 0)
>                                 ^
> /usr/src/sys/sys/mutex.h:452:2: note: expanded from macro
> 'mtx_lock_spin_flags'
>         mtx_lock_spin_flags_((m), (opts), LOCK_FILE, LOCK_LINE)
>         ^
> /usr/src/sys/sys/mutex.h:429:2: note: expanded from macro
> 'mtx_lock_spin_flags_'
>         __mtx_lock_spin((m), curthread, (opts), (file), (line))
>         ^
> /usr/src/sys/sys/mutex.h:258:2: note: expanded from macro
> '__mtx_lock_spin'
>         spinlock_enter();                                            
>    \
>         ^
> . . .
> 
> (spinlock_enter was not the only example.)
> 
> 

My bad, I forgot to include <lock.h> when I switched the code to
spinlocks.  Should be fixed by r352333.

-- Ian


From owner-freebsd-current@freebsd.org  Sat Sep 14 18:29:33 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6A92FBA77
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 18:29:33 +0000 (UTC)
 (envelope-from jmg@gold.funkthat.com)
Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "gate2.funkthat.com",
 Issuer "Let's Encrypt Authority X3" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 46W1Hv5R0Hz4HTT
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 18:29:31 +0000 (UTC)
 (envelope-from jmg@gold.funkthat.com)
Received: from gold.funkthat.com (localhost [127.0.0.1])
 by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id x8EISvjM089596
 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
 Sat, 14 Sep 2019 11:28:57 -0700 (PDT)
 (envelope-from jmg@gold.funkthat.com)
Received: (from jmg@localhost)
 by gold.funkthat.com (8.15.2/8.15.2/Submit) id x8EISvu9089595;
 Sat, 14 Sep 2019 11:28:57 -0700 (PDT) (envelope-from jmg)
Date: Sat, 14 Sep 2019 11:28:57 -0700
From: John-Mark Gurney <jmg@funkthat.com>
To: Kurt Jaeger <lists@opsec.eu>
Cc: FreeBSD Current <freebsd-current@freebsd.org>
Subject: Re: poudriere, swap full and top says memory is free ?
Message-ID: <20190914182857.GM96402@funkthat.com>
Mail-Followup-To: Kurt Jaeger <lists@opsec.eu>,
 FreeBSD Current <freebsd-current@freebsd.org>
References: <20190914173805.GC2863@home.opsec.eu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190914173805.GC2863@home.opsec.eu>
X-Operating-System: FreeBSD 11.0-RELEASE-p7 amd64
X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7  ED9B D5FF 5A51 C0AC 3D65
X-Files: The truth is out there
X-URL: https://www.funkthat.com/
X-Resume: https://www.funkthat.com/~jmg/resume.html
X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE
X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger?
User-Agent: Mutt/1.6.1 (2016-04-27)
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3
 (gold.funkthat.com [127.0.0.1]); Sat, 14 Sep 2019 11:28:57 -0700 (PDT)
X-Rspamd-Queue-Id: 46W1Hv5R0Hz4HTT
X-Spamd-Bar: /
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy
 when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com
X-Spamd-Result: default: False [-0.42 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-0.95)[-0.954,0]; MID_RHS_MATCH_FROM(0.00)[];
 FROM_HAS_DN(0.00)[]; RCVD_TLS_ALL(0.00)[];
 MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com];
 AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-0.93)[-0.927,0];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
 IP_SCORE(-0.74)[ip: (-1.91), ipnet: 208.87.216.0/21(-0.95), asn: 32354(-0.76),
 country: US(-0.05)]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[];
 FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com];
 MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[];
 ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US];
 SUBJECT_ENDS_QUESTION(1.00)[];
 FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 18:29:34 -0000

Kurt Jaeger wrote this message on Sat, Sep 14, 2019 at 19:38 +0200:
> - a poudriere build
> - of a list of ports
> - on 12.0-RELEASE-p10
> - on a 4 core+4 hyperthreads CPU, an Intel(R) Xeon(R) CPU E3-1230 v6
>   @ 3.50GHz
> - with 32 GB RAM
> - zpool with 2x 500 GB SSDs as a mirror
> 
> and right now, this can be seen:
> 
> last pid: 90922;  load averages:  5.02,  5.14,  5.73    up 0+03:53:08  19:31:05
> 82 processes:  6 running, 76 sleeping
> CPU: 60.6% user,  0.0% nice,  2.1% system,  0.0% interrupt, 37.3% idle
> Mem: 4598M Active, 2854M Inact, 11G Laundry, 6409M Wired, 6375M Free
> ARC: 3850M Total, 1721M MFU, 2090M MRU, 665K Anon, 19M Header, 19M Other
>      3406M Compressed, 3942M Uncompressed, 1.16:1 Ratio
> Swap: 18G Total, 18G Used, 396K Free, 99% Inuse, 68K In
> 
> So: Swap is full, approx. 6 GB memory is reported as free.
> 
> This is surprising. Can I somehow tune this in any way, so that
> the memory available is used for the build ? Or is the problem somewhere
> else ?

Are you sure that this hasn't just recently completed a large link of
something like Chromium?  There are known to be compiles that can take
many GB's of memory and if they recently exited, there hasn't been time
to swap stuff back in...  or is this the steady state over the entire
compile?

-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."

From owner-freebsd-current@freebsd.org  Sat Sep 14 18:49:39 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id 56BA3FC18C
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 18:49:39 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic305-2.consmr.mail.bf2.yahoo.com
 (sonic305-2.consmr.mail.bf2.yahoo.com [74.6.133.41])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46W1l61yVHz4JRp
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 18:49:37 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: 5BsV46gVM1mzhZrBvKAkAQtJBUEFf7moNfs4uQSplvOEX2qjHmekKGLqJoEhh7X
 _CGj0.rznU4KFChxrVfulN_EszgqH7auJfdxZ483udQUu8Efma9VRV0PWInDaXz_tGVFAKjG.LmA
 eXZG3mhABgopWocV8t_iDOc_S46a2.o0v2vsVJvD1UUh69qKSHj_Rke07VRqPth5BQpjGRUjSHO2
 6se1AfFWyn9Js1LGtO1YMELiDzRL96GNV8ii3SQt7U.ZFIGIJZgz_7Y2ow8pqWjhkfEayps.0k3.
 5P8.3CLQ3D44L6tmh1aTq.5R_Pshpam.oXCktQ9PVA1KAx5sAymuxFVlubivhn_HCcGHozP6IWI.
 oumuHX6sxp7bAHfGTNnzDYXlzBGM2IJAZ3DiUPPP8_82lW7j.CCD8mV8cFTp0TR2fbslKZKOeA.J
 1BtRNuXQims1pvVRaLnw2DzH4M9oRFHltZ5sG7v0UgbKFckVO58S7AigJd1bs50HK4VbnFMvrBgq
 IcGug_aQtH6V8joiRi9nmVGHcy2LPDsRCgCKzVQLlGgrcW0h8otmyO11Un6RHfWBMvpKCBsiECes
 L2AkLlRWeROQSU5sim6Y3pluDbxVJhZFDssf83lPcH5olwNqcyxsvTcFMbLfnQ6O362jJEyS.zI.
 uS.B3z3VQeQaXdQWsoWuB9Mi1GJDPZ1NkNAWwbMFXbl.Nz3XXTeA7Mw6qBfJztIcfkYDUfnsaj2X
 O.S5Mlrt8g4W_ZxTNVv5LT.RLzAGUEh.3gXjvWiqfTvAvyyyrh1k2str69j7AVV9wJtYPyQ.bcZd
 50d2Li1zC1cxvTKoEIb6u25tcLt3L6rWEkyb7UtFI6O0MHhLc_DsdnNGgJlUEGvmhDojdtbUA.ot
 _noXTjk6mHgPpoSPcg.2ujj3CkogApCXKdFpNbtjl0WvSFs.bJ8PCqlxf_39Civ1b0RwDnr990L1
 21OBT5mL3_rxP06rt832hSu.jHek4ifhAI3ih2QF4eFXlOuRCAGPRWg60Cw0okxpUWSNwgR8mwDu
 Q.qdfTcJJBI4WCSsunX.lNYmIYIcd9nVeH0esMlXwfA1Q05WHXtokTkNNbSqNpX_gkFeZJ_S9reI
 .APNnmiF753XTdcU2IuFOVJx_JrJkpRvpaJb5YmfGuByE1bcOKzniQEyV2JZvKCL84nLfys8yHjD
 uKZ0DHzqyLv3tpkX8ntVH6H.WNDn7aA7Op83JjYgGRD4Nxc9_ZeSGApOZscYGsUNH6.GeVw2QFzz
 gpeUFtnRy8emVisrsTbV9bs1KtpZxTaZ5sR4SIPXnpQcZpm74T07tj42QLIXVo4rc
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic305.consmr.mail.bf2.yahoo.com with HTTP; Sat, 14 Sep 2019 18:49:35 +0000
Received: by smtp410.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID 4844f306538fab6cdf33a9de820ec6e0; 
 Sat, 14 Sep 2019 18:49:33 +0000 (UTC)
Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: head -r352274 buildkernel targetting armv7 failure:
 am335x/am335x_dmtpps.c:304:3: error: implicit declaration of function
 'spinlock_enter' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
From: Mark Millard <marklmi@yahoo.com>
In-Reply-To: <e675e9577469510f1c3145e1910b34745a9475fd.camel@freebsd.org>
Date: Sat, 14 Sep 2019 11:49:31 -0700
Cc: FreeBSD Current <freebsd-current@freebsd.org>,
 freebsd-arm@freebsd.org
Content-Transfer-Encoding: 7bit
Message-Id: <2F803AD5-900F-44F3-99E9-89082D604CA8@yahoo.com>
References: <0E884723-2826-4EDF-A16F-841E01E9D4EC@yahoo.com>
 <e675e9577469510f1c3145e1910b34745a9475fd.camel@freebsd.org>
To: Ian Lepore <ian@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
X-Rspamd-Queue-Id: 46W1l61yVHz4JRp
X-Spamd-Bar: --
X-Spamd-Result: default: False [-2.31 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-0.86)[-0.864,0];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[];
 RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MIME_GOOD(-0.10)[text/plain];
 MV_CASE(0.50)[]; NEURAL_HAM_LONG(-0.95)[-0.947,0];
 TO_DN_SOME(0.00)[]; IP_SCORE_FREEMAIL(0.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 RCVD_IN_DNSWL_NONE(0.00)[41.133.6.74.list.dnswl.org : 127.0.5.0];
 RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2];
 FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+];
 FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:26101, ipnet:74.6.128.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 IP_SCORE(0.00)[ip: (3.76), ipnet: 74.6.128.0/21(1.45), asn: 26101(1.16),
 country: US(-0.05)]; 
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 18:49:39 -0000



On 2019-Sep-14, at 11:21, Ian Lepore <ian@freebsd.org> wrote:

> On Sat, 2019-09-14 at 11:05 -0700, Mark Millard via freebsd-arm wrote:
>> After updating my amd64 context to head -r352274,
>> attempting an amd64->armv7 cross buildworld buildkernel
>> ended up failing with:
>> 
>> 
>> --- am335x_dmtpps.o ---
>> /usr/src/sys/arm/ti/am335x/am335x_dmtpps.c:304:3: error: implicit
>> declaration of function 'spinlock_enter' is invalid in C99 [-Werror,-
>> Wimplicit-function-declaration]
>>                mtx_lock_spin(&sc->pps_mtx);
>>                ^
>> (...shortened...)
>> . . .
>> 
>> (spinlock_enter was not the only example.)
>> 
>> 
> 
> My bad, I forgot to include <lock.h> when I switched the code to
> spinlocks.  Should be fixed by r352333.

Thanks.

It is interesting that:

https://ci.freebsd.org/job/FreeBSD-head-armv7-build/6042/

shows a successful build of -r352274 (the last before
-r352275 broke both arm and aarch64). Prior builds also
were successful.

I'm manually applied your update to -r352274 and am rebuilding
from scratch.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)


From owner-freebsd-current@freebsd.org  Sat Sep 14 19:41:22 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id C1370FD5E4
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 19:41:22 +0000 (UTC) (envelope-from lists@opsec.eu)
Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1])
 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
 server-signature RSA-PSS (4096 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46W2tn5Lpfz4MjZ
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 19:41:21 +0000 (UTC)
 (envelope-from lists@opsec.eu)
Received: from pi by home.opsec.eu with local (Exim 4.92.2 (FreeBSD))
 (envelope-from <lists@opsec.eu>) id 1i9Dv3-000NDk-Mp
 for freebsd-current@freebsd.org; Sat, 14 Sep 2019 21:41:17 +0200
Date: Sat, 14 Sep 2019 21:41:17 +0200
From: Kurt Jaeger <lists@opsec.eu>
To: FreeBSD Current <freebsd-current@freebsd.org>
Subject: Re: poudriere, swap full and top says memory is free ?
Message-ID: <20190914194117.GD2863@home.opsec.eu>
References: <20190914173805.GC2863@home.opsec.eu>
 <20190914182857.GM96402@funkthat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190914182857.GM96402@funkthat.com>
X-Rspamd-Queue-Id: 46W2tn5Lpfz4MjZ
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none;
 spf=pass (mx1.freebsd.org: domain of lists@opsec.eu designates
 2001:14f8:200::1 as permitted sender) smtp.mailfrom=lists@opsec.eu
X-Spamd-Result: default: False [-4.90 / 15.00]; ARC_NA(0.00)[];
 NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
 R_SPF_ALLOW(-0.20)[+mx:c]; TO_MATCH_ENVRCPT_ALL(0.00)[];
 MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[];
 DMARC_NA(0.00)[opsec.eu]; RCPT_COUNT_ONE(0.00)[1];
 NEURAL_HAM_LONG(-1.00)[-1.000,0];
 SUBJECT_ENDS_QUESTION(1.00)[]; TO_DN_ALL(0.00)[];
 IP_SCORE(-3.60)[ip: (-9.48), ipnet: 2001:14f8::/32(-4.73), asn: 12502(-3.79),
 country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[];
 R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:12502, ipnet:2001:14f8::/32, country:DE];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 19:41:22 -0000

Hi!

> > Mem: 4598M Active, 2854M Inact, 11G Laundry, 6409M Wired, 6375M Free
> > ARC: 3850M Total, 1721M MFU, 2090M MRU, 665K Anon, 19M Header, 19M Other
> >      3406M Compressed, 3942M Uncompressed, 1.16:1 Ratio
> > Swap: 18G Total, 18G Used, 396K Free, 99% Inuse, 68K In
> > 
> > So: Swap is full, approx. 6 GB memory is reported as free.

> > This is surprising. Can I somehow tune this in any way, so that
> > the memory available is used for the build ? Or is the problem somewhere
> > else ?
> 
> Are you sure that this hasn't just recently completed a large link of
> something like Chromium?

Yes, because I plot memory/swap/etc using nagios. It's not only
a spike.

> There are known to be compiles that can take
> many GB's of memory and if they recently exited, there hasn't been time
> to swap stuff back in...  or is this the steady state over the entire
> compile?

Building a few ports (firefox, libreoffice etc) takes some time,
so it has been stable during that phase.

-- 
pi@opsec.eu            +49 171 3101372                    One year to go !

From owner-freebsd-current@freebsd.org  Sat Sep 14 23:56:35 2019
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.nyi.freebsd.org (Postfix) with ESMTP id D0484D2F9C
 for <freebsd-current@mailman.nyi.freebsd.org>;
 Sat, 14 Sep 2019 23:56:35 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
Received: from sonic310-22.consmr.mail.gq1.yahoo.com
 (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 46W8YH0FL4z4d08
 for <freebsd-current@freebsd.org>; Sat, 14 Sep 2019 23:56:34 +0000 (UTC)
 (envelope-from marklmi@yahoo.com)
X-YMail-OSG: Pk7Lg44VM1mhl5nqPWTt24AQrCpWJbiabHpqarYh2tP6kwDS0Kbl7NPQ3F2g_83
 buQL674.ZLsE3aVxIkVpf3Hbh8thPwlB0crzaKK6Acq0FW9OKZku4PASJAzY5D7b9rW82rSfdp6Z
 r003hYSvJf0yTJvR3KtPZog5WvhvNDoZ8lPl3I1PiUNCNkUNbVhzSCEvpZxRAArDAc7GcdvyrE04
 HUzLpC6McaZ7RUxnR52XdUgbW4kpoMKZv2cipE7iigXXbA2PhmIx0LpAqNJGIusFLqc8SibRwaOj
 UKhXlTpa9fvE0vfEVf9EkTEgXgum_i3b_Ffne_tbNoyIyd0XL7dfxNPhHwFzKdHcf9HUsT3.znax
 bZB3f1ZpIg18nJ2PUuMwQP57OYVOwQUXiMF6VgTN8dRNEGONvaLfUxRSb6mKK9_rplvoicnYs.y0
 HEqJmQodpSQ2jaUmh5iG6RKQGBJtKZV7vnaSXe1YhTrkrhOW8MUNCILuF30g0Y2C0vaiDRervXm.
 j_co3UStpvRz7FCeI.VpagI5FGIEu13Qvc5A539C6Vz9JtWkHjR9QK5PsiqXJcTudG.6sRXS6fvf
 aeaLl5pFPQRPUQ7yuEl7f2SCrCpn2kkPcbUfMEjbB7Wz42YPkXc6dMd8jVDUxHeOlPWosLCvuwEZ
 9BaP9Hef_ZytbG64ZUWUfJUoX_SwGFaOcgoXATjAL37LwDiI13To3HT9cEvzeae4jlUZ3NSzVhId
 sBxffEIyJKH59B9Y.3EzfQd8wpjOqRgNQFGRWocSyGWeIVMeszXGvSE9d.14TFv0B0.gYRiDd2sI
 YNw5j3v8.YIZNL_M2cFNpjk33u5IfRXQsk6_Xx8S4Gskt4tFKkj_5kCYFHqcQg4Jc4U.3WaufBE4
 MoZFBjLwhkjpX0dfCCU8Vs93Cndqb_B0qQurQl5PE.ZziL7wKoy_TjgXvambwjhxSdjJwsYWoWF2
 _LZEqmbiNSRDZQl0cUetJReGXTTPT4yNLpawurAA571Xyq80sEfWEhlOs_SfBAoJxg8cRdWe6wZM
 UETvUqcwbGtKDgbLPcCAWpUXyJCMvpEaSkCHsz4tAduq1nehT0AL6oJHJLLGmr91V8_Gn2IqaHhW
 nOUr46onZQciFTvRfKSghDacJ0ivQ1OVsOnkupY0xbco0kcXAniFjpvv8Zz63.g7pALAIhk_4ewj
 ZZjk_PU3o_qoY0oD2dg3.tNV_q2j0fq0uEwgIizaiZj_yg.f_SysMKYRWUhtPt90kM5aYrRBd9VF
 TygXb70O0e.MlZjXBZ9tDHu8JSEGn5zbJQhto5UJHJjV_LgCk25CoD84XplnuRLEmU08myf4t._P
 viWtftmDZeRQA.w8XoYZuAfIwiP2AKBZM.9A5EQ--
Received: from sonic.gate.mail.ne1.yahoo.com by
 sonic310.consmr.mail.gq1.yahoo.com with HTTP; Sat, 14 Sep 2019 23:56:32 +0000
Received: by smtp401.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA
 ID b0136dac9959d9f52e5dda39ac632019; 
 Sat, 14 Sep 2019 23:56:28 +0000 (UTC)
From: Mark Millard <marklmi@yahoo.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Subject: Re: spurious out of swap kills
Message-Id: <8269B1A1-48DE-4997-B1DD-0E1F69EC9A52@yahoo.com>
Date: Sat, 14 Sep 2019 16:56:28 -0700
To: "truckman@freebsd.org" <truckman@FreeBSD.org>,
 FreeBSD Current <freebsd-current@freebsd.org>
X-Mailer: Apple Mail (2.3445.104.11)
References: <8269B1A1-48DE-4997-B1DD-0E1F69EC9A52.ref@yahoo.com>
X-Rspamd-Queue-Id: 46W8YH0FL4z4d08
X-Spamd-Bar: --
X-Spamd-Result: default: False [-2.47 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[];
 TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com];
 FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[];
 DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2];
 DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject];
 FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[];
 MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com];
 ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US];
 MID_RHS_MATCH_FROM(0.00)[];
 DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.979,0];
 R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[];
 NEURAL_HAM_LONG(-1.00)[-0.996,0]; MIME_GOOD(-0.10)[text/plain];
 IP_SCORE(0.00)[ip: (1.22), ipnet: 98.137.64.0/21(0.93), asn: 36647(0.74),
 country: US(-0.05)]; IP_SCORE_FREEMAIL(0.00)[];
 TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[148.69.137.98.list.dnswl.org : 127.0.5.0];
 RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 14 Sep 2019 23:56:35 -0000

Konstantin Belousov kostikbel at gmail.com wrote on
Fri Sep 13 05:53:41 UTC 2019 :

> Basically, page fault handler waits for vm.pfault_oom_wait *
> vm.pfault_oom_attempts for a page allocation before killing the =
process.
> Default is 30 secs, and if you cannot get a page for 30 secs, there is
> something very wrong with the machine.

The following was not for something like a Ryzen, but for
an armv7 board using a USB device for the file system and
swap/paging partition. Still it may be a suggestive
example of writing out a large amount of laundry.

There was an exchange I had with Warner L. that implied easily
having long waits in the queue when trying to write out the
laundry (or other such) in low end contexts. I extract some
of it below.

dT: 1.006s  w: 1.000s
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w    d/s   kBps   =
ms/d   %busy Name
   56    312      0      0    0.0    312  19985  142.6      0      0    =
0.0   99.6| da0

Note: L(q) could be a lot bigger than 56 but I work with the
example figures that I used at the time and that Warner commented on.
The 142.6 ms/w includes time waiting in the queue and was vastly
more stable than the L(q) figures.

Warner wrote, in part:

QUOTE
142.6ms/write is the average of the time that the operations that =
completed during the polling interval took to complete. There's no =
estimating here.

So, at 6 or 7 per second for the operation to complete, coupled with a =
parallel factor of 1 (typical for low end junk flash), we wind up with =
56 operations in the queue taking 8-10s to complete.
END QUOTE

Things went on from there but part of it was based on a
reporting patch that Mark Johnston had provided.

Me:
It appears to me that, compared to a observed capacity of
roughly around 20 MiBytes/sec for writes, large amounts of
bytes are being queued up to be written in a short time,
for which it just takes a while for the backlog to be
finished.


Warner:
Yes. That matches my expectation as well. In other devices, I've found =
that I needed to rate-limit things to more like 50-75% of the max value =
to keep variance in performance low. It's the whole reason I wrote the =
CAM I/O scheduler.

Me:
The following is from multiple such runs, several manually
stopped but some killed because of sustained low free
memory. I had left vm.pageout_oom_seq=3D12 in place for this,
making the kills easier to get than the 120 figure would. It
does not take very long generally for some sort of message to
show up.

(Added Note: 65s and 39s were at the large end of what I reported
at the time.)

. . .
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 164064, size: =
12288
waited 65s for async swap write
waited 65s for swap buffer
waited 65s for async swap write
waited 65s for async swap write
waited 65s for async swap write
v_free_count: 955, v_inactive_count: 1
Aug 20 06:11:49 pine64 kernel: pid 1047 (stress), uid 0, was killed: out =
of swap space
waited 5s for async swap write
waited 5s for swap buffer
waited 5s for async swap write
waited 5s for async swap write
waited 5s for async swap write
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314021, size: =
12288
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314084, size: =
32768
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314856, size: =
32768
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314638, size: =
131072
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 312518, size: 4096
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 312416, size: =
16384
waited 39s for async swap write
waited 39s for swap buffer
waited 39s for async swap write
waited 39s for async swap write
waited 39s for async swap write
swap_pager: indefinite wait buffer: bufobj: 0, blkno: 314802, size: =
24576
. . .

Warner:
These numbers are consistent with the theory that the swap device =
becomes overwhelmed, spiking latency and causing crappy down-stream =
effects. You can use the I/O scheduler to limit the write rates at the =
low end. You might also be able to schedule a lower write queue depth at =
the top end as well, but I've not seen good ways to do that.


=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)