From owner-freebsd-hackers@freebsd.org Tue Jan 22 07:38:44 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 623AE14B1793 for ; Tue, 22 Jan 2019 07:38:44 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from dedi548.your-server.de (dedi548.your-server.de [IPv6:2a01:4f8:d0a:2645::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE04A76C2E for ; Tue, 22 Jan 2019 07:38:42 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from [78.46.172.2] (helo=sslproxy05.your-server.de) by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89_1) (envelope-from ) id 1glqdk-0007iY-Hm for freebsd-hackers@freebsd.org; Tue, 22 Jan 2019 08:38:32 +0100 Received: from [82.135.62.35] (helo=mail.embedded-brains.de) by sslproxy05.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from ) id 1glqdk-000Dr1-CU for freebsd-hackers@freebsd.org; Tue, 22 Jan 2019 08:38:32 +0100 Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 4B0DA2A0043 for ; Tue, 22 Jan 2019 08:38:33 +0100 (CET) Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id kbezLiiDUHKM for ; Tue, 22 Jan 2019 08:38:33 +0100 (CET) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id F2D022A0928 for ; Tue, 22 Jan 2019 08:38:32 +0100 (CET) X-Virus-Scanned: amavisd-new at zimbra.eb.localhost Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id TM3B5X-y3qnX for ; Tue, 22 Jan 2019 08:38:32 +0100 (CET) Received: from linux-diu0.suse (unknown [192.168.96.161]) by mail.embedded-brains.de (Postfix) with ESMTP id D4DED2A0043 for ; Tue, 22 Jan 2019 08:38:32 +0100 (CET) From: Sebastian Huber To: freebsd-hackers@freebsd.org Subject: [PATCH] Remove unused *_sysinit_flags() declarations Date: Tue, 22 Jan 2019 08:38:31 +0100 Message-Id: <20190122073831.1735-1-sebastian.huber@embedded-brains.de> X-Mailer: git-send-email 2.16.4 X-Authenticated-Sender: smtp-embedded@poldinet.de X-Virus-Scanned: Clear (ClamAV 0.100.2/25320/Tue Jan 22 03:29:28 2019) X-Rspamd-Queue-Id: EE04A76C2E X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=softfail (mx1.freebsd.org: 2a01:4f8:d0a:2645::2 is neither permitted nor denied by domain of sebastian.huber@embedded-brains.de) smtp.mailfrom=sebastian.huber@embedded-brains.de X-Spamd-Result: default: False [-2.74 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[mail.embeddedbrains.de,mx.netmuc.net]; NEURAL_HAM_SHORT(-0.67)[-0.667,0]; MID_CONTAINS_FROM(1.00)[]; IP_SCORE(-0.97)[ipnet: 2a01:4f8::/29(-2.51), asn: 24940(-2.35), country: DE(-0.01)]; DMARC_NA(0.00)[embedded-brains.de]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; RCVD_COUNT_SEVEN(0.00)[8]; HAS_X_AS(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 07:38:44 -0000 Definitions were removed more than two years ago. --- sys/sys/rmlock.h | 1 - sys/sys/rwlock.h | 1 - 2 files changed, 2 deletions(-) diff --git a/sys/sys/rmlock.h b/sys/sys/rmlock.h index da5d7274e78..cdcb6edc3b9 100644 --- a/sys/sys/rmlock.h +++ b/sys/sys/rmlock.h @@ -54,7 +54,6 @@ void rm_init_flags(struct rmlock *rm, const char *name, int opts); void rm_destroy(struct rmlock *rm); int rm_wowned(const struct rmlock *rm); void rm_sysinit(void *arg); -void rm_sysinit_flags(void *arg); void _rm_wlock_debug(struct rmlock *rm, const char *file, int line); void _rm_wunlock_debug(struct rmlock *rm, const char *file, int line); diff --git a/sys/sys/rwlock.h b/sys/sys/rwlock.h index 43bbc68cf13..58d5881eb76 100644 --- a/sys/sys/rwlock.h +++ b/sys/sys/rwlock.h @@ -130,7 +130,6 @@ void _rw_init_flags(volatile uintptr_t *c, const char *name, int opts); void _rw_destroy(volatile uintptr_t *c); void rw_sysinit(void *arg); -void rw_sysinit_flags(void *arg); int _rw_wowned(const volatile uintptr_t *c); void _rw_wlock_cookie(volatile uintptr_t *c, const char *file, int line); int __rw_try_wlock_int(struct rwlock *rw LOCK_FILE_LINE_ARG_DEF); -- 2.16.4 From owner-freebsd-hackers@freebsd.org Tue Jan 22 09:32:22 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCCFE14B5C26; Tue, 22 Jan 2019 09:32:22 +0000 (UTC) (envelope-from shreyankfbsd@gmail.com) Received: from mail-yb1-xb36.google.com (mail-yb1-xb36.google.com [IPv6:2607:f8b0:4864:20::b36]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 94F7A8307E; Tue, 22 Jan 2019 09:32:21 +0000 (UTC) (envelope-from shreyankfbsd@gmail.com) Received: by mail-yb1-xb36.google.com with SMTP id y7so8249704ybg.13; Tue, 22 Jan 2019 01:32:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=50N+nEgDiRPbL/DP3FElDMi+xWaSrMEkL1zKIznrOqE=; b=pDzZphqLyZ2neIRxZOOvVoJTG8G+8D3YtIALlqUFFisrPli4O771sOlogA118MO7PH c9GNE+EPgX0VKc89tF1JchxzF3lkPLHRJwOPDJTlpB3yUY22OqfyV04hDyP7Y+3qppQW 9DkNybX501/kyPQJ2UzYhn/cdh9uGsVhpOSMTdeJ2H05M4vXXlMNBu5pyOArgN+Az1h6 hHTVzxVk1CNNTauHMLP3O0/Xj+uspxkZkc1YXIw+lx0mN02Lca0hBPLoO6876A4eQu94 BrN9aSGbsT9dG/akkx7r1Lja7u+Br5pFnpSBa4uWDB2kluqtu+MHAKaJINmHL8Kq1FJP V/BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=50N+nEgDiRPbL/DP3FElDMi+xWaSrMEkL1zKIznrOqE=; b=UxLACAaGQOzqBZIzkMy1XE5ZVRWBwy9dbPr46ZyTrchRT5y8SGnH+t+GJMDEouOLel Zy0mXXK3ACsKBUBXkfGCXSzCyoziyJU/OPf+jbDoYtnWguRjBXwbrf10CWxqNSqx4PWH OAoy3Lu9pxztpdxnRJDqlKD0g0iCEFQRB5luLJwq+LVlOiTWicVwL75f9EFlhic5dXWY tGB5JWhdY/cEpgIbByMV6y29LP0r7eN+FtftgAUctQxz6xkk5cM5PI7E9F3Eu9X3oUjK lNOxjBa7PYWodqWkm0MuXWG2aKP9vKkCWGiH6TeGKRfkD+OAAkw2V85B/SlZE+4vveXp 2DPA== X-Gm-Message-State: AJcUukdM9PxrcweARjmqeiayt0h7WQV2mbCUb4GrXAqqeFvivj7zpMl/ 6x8r3FbLyh5xbYAZGZdzOcZhLCggQxj9jbP5v5C1 X-Google-Smtp-Source: ALg8bN7yTeGKJ8bWFvJ4Gv1nhBnvYXurUTKD0DrdNzSJSOG6F4YsED+bs/wk6tRY1BwdCtWOqLs8kDCAOOG7ZrnP4lM= X-Received: by 2002:a25:8209:: with SMTP id q9mr20266506ybk.21.1548149540897; Tue, 22 Jan 2019 01:32:20 -0800 (PST) MIME-Version: 1.0 From: shreyank amartya Date: Tue, 22 Jan 2019 15:02:09 +0530 Message-ID: Subject: PCIe device performance To: freebsd-hackers@freebsd.org Cc: freebsd-drivers@freebsd.org X-Rspamd-Queue-Id: 94F7A8307E X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=pDzZphqL; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of shreyankfbsd@gmail.com designates 2607:f8b0:4864:20::b36 as permitted sender) smtp.mailfrom=shreyankfbsd@gmail.com X-Spamd-Result: default: False [-6.37 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.995,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[6.3.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.55)[-0.555,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.81)[ip: (-9.62), ipnet: 2607:f8b0::/32(-2.46), asn: 15169(-1.88), country: US(-0.08)]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 09:32:23 -0000 Hi, I have a PCIe device BAR mapped to a particular region in KVA. I use memcpy to write into that region and measure the transfer rate. Recently, I was able to increase the transfer rate by setting the memory attribute of the BAR mapped region to VM_MEMATTR_WRITE_COMBINING. However, the transfer rate fluctuates wildly and I cannot get a deterministic value. What could be the cause for this? Thanks Shreyank Amartya From owner-freebsd-hackers@freebsd.org Tue Jan 22 13:18:18 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C6E7148D282; Tue, 22 Jan 2019 13:18:18 +0000 (UTC) (envelope-from babupalit@gmail.com) Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6FDC28C506; Tue, 22 Jan 2019 13:18:17 +0000 (UTC) (envelope-from babupalit@gmail.com) Received: by mail-qk1-x735.google.com with SMTP id y78so14140362qka.12; Tue, 22 Jan 2019 05:18:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=kvBNyxPA/ARzaT4W5l8p7ZX+Lo7eVuyS3Fxsx4ec36E=; b=KV8gU4DxQ8MzCIouRN5Na6lBgSbSAVHMAhTCc2Hokw/hE+zyGnj0fCrmLcU6zTxOZJ eLAUOB3So4sbaH7MNf4j5Qi4/vSHLOd8ZPjymfe7Rfv5yFb+UpoT5WAeJIaoURbLnukc moKGnvhdTLEHb6ea1y+fmNgGjNxEZc6XftyIZIXpcCr0Pg/FTNkL8vUg+2mlnBvb4c+K vWqQiGHHUkDiuQpypSVfwEtbT0BVjQV2vnRKbx4FzFI68N8SQmjUur4ZXV1KyRs7BOm3 iom26YzIydDXZKRHTTLUEmv8kpIbCe2tvFK4BdoNLis5BNQ9/LObRyAoYLo4WQ4f2UHG R0sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=kvBNyxPA/ARzaT4W5l8p7ZX+Lo7eVuyS3Fxsx4ec36E=; b=WB4uRUAQWRc8iieDpXzd2YkgXt9crVgDf36nHF/Rw7AEZYpMUTdpLMFVoU0+msFSC1 1Bwt10MHthL5WeO9Aeb2C97ry6JXqdCbYPMSuLZrUPTAy0EY+VVAkJHfjlF0MrOcqhZr q8jKemq0NeUPRBFcFdX2vciJoBFJWYNnhXo2D4nqxrd1GTwU8PD/pQlXksduaNYU4ByG rOP9canfDvfkABJtkB+0F8Hv0zE7F30Xk9btzRWe+m4jxGCvN3bprClm2KFO6z6/wOfs 0DJdDIo+s26zEZncdEUikQR3lkQbrsQnvTRiPyBR6tqdoBhwK1fAXJskLVQZLD9pPu2L CJKg== X-Gm-Message-State: AJcUukevXm89tmBBm7bbPmQlVnE+iLYOrkwLlBQ4KZPvC+jtn2k7sWbe Ttg29OEeJNvwfXXglgKkpofJY4yi1LAb27WfzZj/g96FAQg= X-Google-Smtp-Source: ALg8bN7t8btdZlNTm9RKVcw1xwxLrjS3fOQvTKg5ZsTWvvrwggOBck1nlkvaIQoqGQcLjfNKKd/AZ0g0/9TcTe7eadU= X-Received: by 2002:a37:5257:: with SMTP id g84mr28251531qkb.76.1548163096819; Tue, 22 Jan 2019 05:18:16 -0800 (PST) MIME-Version: 1.0 From: Arpan Palit Date: Tue, 22 Jan 2019 18:48:05 +0530 Message-ID: Subject: "panic: softdep_deallocate_dependencies: dangling deps", while runing IO on mounted HDD and doing HDD hot-unplug. To: freebsd-hackers@freebsd.org, freebsd-drivers@freebsd.org X-Rspamd-Queue-Id: 6FDC28C506 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=KV8gU4Dx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of babupalit@gmail.com designates 2607:f8b0:4864:20::735 as permitted sender) smtp.mailfrom=babupalit@gmail.com X-Spamd-Result: default: False [-6.72 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[5.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.99)[-0.990,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.72)[ip: (-9.20), ipnet: 2607:f8b0::/32(-2.46), asn: 15169(-1.88), country: US(-0.08)]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-Mailman-Approved-At: Tue, 22 Jan 2019 14:00:40 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 13:18:18 -0000 Hi, I am working on HDD hot-unplug "dangling deps" panic issue. Provided the following steps to recreate and observations when the issue hit. STEP 1. Installed stable/11 freeBSD branch with below commit head. ---------------------------------------------------------------------------------------------------------- commit 7b249ab3e16a3d41d0a58a43d7d89137a1c9ec00 (HEAD -> stable/11, origin/stable/11) Author: delphij Date: Tue Jan 22 04:20:52 2019 +0000 MFC r342856: Added support for the SIOCGI2C ioctl. ---------------------------------------------------------------------------------------------------------- Step 2. Attached HDD and mounted to local file system. Step 3. While IO is running to the mounted device; hot-unplug the device and issue umount command for the mounted device. Observations: =========== -> observed panic as b_ioflags is not to BIO_ERROR and also observed b_error is set to zero. backtrace: -------------- fsync: giving up on dirty (error = 6) 0xfffff8029cbda938: tag devfs, type VCHR usecount 1, writecount 0, refcount 15 rdev 0xfffff80247038400 flags (VI_DOOMED|VI_ACTIVE) v_object 0xfffff80247d320f0 ref 0 pages 95 cleanbuf 10 dirtybuf 2 lock type devfs: EXCL by thread 0xfffff8029c84d620 (pid 774, df, tid 100720) dev ada1p1 panic: softdep_deallocate_dependencies: dangling deps cpuid = 57 KDB: stack backtrace: #0 0xffffffff80b20d27 at kdb_backtrace+0x67 #1 0xffffffff80add3c7 at vpanic+0x177 #2 0xffffffff80add453 at panic+0x43 #3 0xffffffff80d84376 at softdep_deallocate_dependencies+0x76 #4 0xffffffff80b7cd2c at brelse+0x16c #5 0xffffffff80b9bc5d at flushbuflist+0x15d #6 0xffffffff80b9b891 at bufobj_invalbuf+0x81 #7 0xffffffff80b9f06e at vgonel+0x18e #8 0xffffffff80b9f68f at vgone+0x2f #9 0xffffffff809aba42 at devfs_delete+0x1a2 #10 0xffffffff809ac1e2 at devfs_populate_loop+0x2b2 #11 0xffffffff809abf1a at devfs_populate+0x4a #12 0xffffffff809b0dcc at devfs_populate_vp+0x8c #13 0xffffffff809afa0f at devfs_getattr+0x1f #14 0xffffffff810dc8a7 at VOP_GETATTR_APV+0xf7 #15 0xffffffff80bad553 at vn_stat+0xa3 #16 0xffffffff80bab44f at vn_statfile+0x4f #17 0xffffffff80a8a8a9 at kern_fstat+0xa9 -> Also you can find similar issue in the following PR. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225423 As I am working on the above issue, Can anyone please provide some solution/suggestion to go forward. Thanks, Arpan From owner-freebsd-hackers@freebsd.org Tue Jan 22 18:29:52 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 628B214AAC40 for ; Tue, 22 Jan 2019 18:29:52 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E6F4297A5C for ; Tue, 22 Jan 2019 18:29:50 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by mail-ed1-x530.google.com with SMTP id d39so20078821edb.12 for ; Tue, 22 Jan 2019 10:29:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=longcount-org.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=uFbTPEkRsutpt8eR6Vq5NDd7Jhhr3i7zW4UbGSQpBnw=; b=bmIA+u1YcqdqsQaSuUGH9LhVPcO053ui2vIE5pZwtwIyr9gfdB50IWjIIVSQKaxfL/ HdfBP5xNkVIcG9gx7qApyuXZcKV9J7Vb0fsLTXD+Cnk5JI47qw/wW+nBh6fVnDPp0lyv lnKXFFJqfgApKK/oLni4kuz8Daa9TTxAC6+fDcGNwI+fdSSIbrxaC+2Ocueibc1VGujc uaSWKXPVD8HqijIotW4jwDLI9D8Tpmcjj4N0f0pNiHTPC5m24RanzATrppOV4QEwyf75 EoR0MZ6pmVdB38FtVC4e+aIDMEYcfVbaczuBX/s0a5YSkLGsP+5Rzs69KimUYHVAIg4i Vw7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=uFbTPEkRsutpt8eR6Vq5NDd7Jhhr3i7zW4UbGSQpBnw=; b=Ajl781WFGRE4t7hXTbOT5pOsbP0hppYnKFLS7THs2AdaPDtLefBg3oGAl8KvD5TSYS T8rMigOqqaYXyaYn7mT8jjAkObaGP/UIKcU/2KWVG94e+waIm2Qg1hzYx0aPxcE48RDr 2yO17w1iJiPZEu2+/m3bOdrZ40J4lHv6pOK6Jo9rLBzmMZ4qtbOrKSDA8HmUYuJezWsY QPGCAookI3haZH/3j2WdKau0n1cqBMq9VznBfl9UoDl/NF0djrZW4rXD9D1R7g0DDLMr D6dZEIPh0f9ylh3fzDsHYPUWw2RmvTrKtEd4t18mS3AHESxynLAcPUW0VLYkHzr5/Ypj l3Cg== X-Gm-Message-State: AJcUukfY21VbH19CI6QDHl+p3+1D75pfjYOIzfhW7J1TFXW7mouLYKVE Ep4kSOjQyRyzCsSw/w53V3pCFGDJYTAgnXdYsMdXFREJ X-Google-Smtp-Source: ALg8bN5YmzcNh6IE0MJua6+1yVBd40XaCYGa31srkmEGqJQ93k59ht9hwF0L7QhkzrkHCfVurB8sSEMawKNHBsAVQcY= X-Received: by 2002:a17:906:94c3:: with SMTP id d3-v6mr28821407ejy.104.1548181788902; Tue, 22 Jan 2019 10:29:48 -0800 (PST) MIME-Version: 1.0 From: Mark Saad Date: Tue, 22 Jan 2019 13:29:37 -0500 Message-ID: Subject: CPU Isolation To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: E6F4297A5C X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=longcount-org.20150623.gappssmtp.com header.s=20150623 header.b=bmIA+u1Y X-Spamd-Result: default: False [-5.72 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.990,0]; R_DKIM_ALLOW(-0.20)[longcount-org.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[longcount.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.63)[ip: (-8.98), ipnet: 2a00:1450::/32(-2.18), asn: 15169(-1.89), country: US(-0.08)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[longcount-org.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx5.googlemail.com,aspmx4.googlemail.com,aspmx3.googlemail.com,alt2.aspmx.l.google.com,aspmx2.googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[0.3.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.79)[-0.790,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 18:29:52 -0000 All I am looking to setup a 12-STABLE box in a way that is similar to a CentOS server I am running. That server is setup using the boot option isocpus, to remove cores/cpus from the kernel scheduler at boot. What I want to try on 12-STABLE is to set aside some cores to run process and some kernel threads that will only always run on a dedicated core with out the chance for being preempted / interrupted by something else on that core / cpu . What I do not see is a way to evict processes off a running cpu post boot or a way to isolate them preboot. Am I missing something? -- mark saad | nonesuch@longcount.org From owner-freebsd-hackers@freebsd.org Wed Jan 23 04:56:32 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 143E814BAECF for ; Wed, 23 Jan 2019 04:56:32 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from know-smtprelay-omc-10.server.virginmedia.net (know-smtprelay-omc-10.server.virginmedia.net [80.0.253.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 27C468A0D4 for ; Wed, 23 Jan 2019 04:56:29 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from [192.168.1.5] ([86.10.101.211]) by cmsmtp with ESMTPA id mAVYgtavOW7DqmAVYggDN0; Wed, 23 Jan 2019 04:51:24 +0000 X-Originating-IP: [86.10.101.211] X-Authenticated-User: J.deBoynePollard-newsgroups@NTLWorld.COM X-Spam: 0 X-Authority: v=2.3 cv=OO4JIxSB c=1 sm=1 tr=0 a=FQ5CjUvp3JFI4KFGyeqcZw==:117 a=FQ5CjUvp3JFI4KFGyeqcZw==:17 a=N659UExz7-8A:10 a=2rVjqWD_AAAA:8 a=6I5d2MoRAAAA:8 a=itly7gIdAAAA:8 a=Ye9q-bpsAAAA:8 a=Qz2FlVj3fCzIrE-oZ-MA:9 a=yvbpt9Z72HqDk5-P:21 a=cBQpUmNTzvWB-TmK:21 a=pILNOxqGKmIA:10 a=zJ3bSuzWvcgA:10 a=KP-NXTVtFukA:10 a=P3dZmzmzSbsA:10 a=ENIBttV5YU4A:10 a=-FEs8UIgK8oA:10 a=NWVoK91CQyQA:10 a=ULaUcM2Ibn9MdPUUwucP:22 a=IjZwj45LgO3ly-622nXo:22 a=1RpNR2E4bTkVPcsa2RFZ:22 a=pHzHmUro8NiASowvMSCR:22 a=nt3jZW36AmriUCFCBwmW:22 Subject: nosh version 1.39 To: FreeBSD Hackers , Debian users , Supervision References: <54430B41.3010301@NTLWorld.com> <76c00c13-4cc9-ed9c-f48f-81a3f050b80b@NTLWorld.com> <5e8454f7-8b8d-ae9f-5aca-3b1b5737f81a@NTLWorld.COM> From: Jonathan de Boyne Pollard Message-ID: <9b60cf3a-9db7-7d80-92d1-be2731ae6155@NTLWorld.COM> Date: Wed, 23 Jan 2019 04:51:24 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <5e8454f7-8b8d-ae9f-5aca-3b1b5737f81a@NTLWorld.COM> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4wfPffovW6T5V/grS6ZdhQoyM1l+DqJIJNdtfaRLKvz+sk5pWNbwIeoA/gIcejUPKwlWlA6A9J54/VVSqzs4owWgUKU6h53LIODw4S2qq9guU/8exYVbfg y9dOjYY3/P8rCq8dQgseAJBXmV0N4aQuuAaUazhni7iwbAJOpFtNsvzg+WAZe+tALa0V/YFMlv/McIYjBqwFwNZN3tJLahCZXt4avjTGwWJb5iZKW4yAgrmY cBZ1TEdeoSIg1/6RrdEpuXgZ8qXkWtUxxUrTCpjHgbbrdFWDi8G8yHMA7zxkFjCg X-Rspamd-Queue-Id: 27C468A0D4 X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.16 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[ntlworld.com:s=meg.feb2017]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:80.0.253.64/28]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.20)[asn: 5089(1.10), country: GB(-0.09)]; NEURAL_HAM_MEDIUM(-0.95)[-0.953,0]; NEURAL_SPAM_SHORT(0.61)[0.605,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[ntlworld.com:+]; MX_GOOD(-0.01)[mx.mnd.ukmail.iss.as9143.net,mx.tb.ukmail.iss.as9143.net]; RCVD_IN_DNSWL_NONE(0.00)[74.253.0.80.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[NTLWorld.COM,quarantine]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:5089, ipnet:80.0.0.0/16, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 04:56:32 -0000 The nosh package is now up to version 1.39 . * http://jdebp.eu./Softwares/nosh/ * https://www.freebsd.org/news/status/report-2017-07-2017-09.html#The-nosh-Project * http://jdebp.info./Softwares/nosh/ I missed announcing 1.38, so this announcement will cover both versions. These versions see a major addition to the user-space virtual terminal subsystem, various other changes in several areas, the completion of some items mentioned as placeholders in version 1.37, and some bug fixes. Completed placeholders ====================== make-read-only-fs is now fully implemented, and is no longer a placeholder. More service bundles ==================== There are several more additions to the set of service bundles supplied with the toolset: connman, ofono, dundee, cntlm, minidlna, powertop, alsa-state, alsa-restore, unattended-upgrade-shutdown, apt-daily-update, apt-daily-upgrade, LCDd, phpsessionclean, tinysshd, watchman, rngd, isnsd, isnsdd, usbmux, and VBoxBalloonCtrl. atd is now a Linux-only service, with the BSDs now having an atrun service. More packages ============= The new nosh-run-bcron, nosh-bcron-as-cron-shims, nosh-debian-crontab, and nosh-debian-crontab-anacron packages deal in running the services and providing the data files for various cron toolsets. The former two deal in bcron, running its services and providing the crontab command as an alias for bcrontab; and the latter two (only available for Linux operating systems) deal in Debian's /etc/crontab file. The new nosh-openrc-shims package contains shims for OpenRC's rc-service and rc-update commands. And the new nosh-run-via-open-rc package contains OpenRC scripts for running the service manager. The new nosh-linux-shims package contains shims for commands to be found in the non-portable util-linux toolset, such as setterm (more on which later). The Debian desktop and server base -run packages no longer preset ntpd and openntpd, on the grounds that a range of such services exist and these are not necessarily the installed softwares. More tools ========== New commands include getuidgid, userenv-fromenv, setgid-fromenv, envgid, printenv, setlogin, console-decode-ecma48, console-control-sequence, console-flat-table-viewer, console-input-method, and local-stream-socket-connect. The userenv command is now a combination of two of these new commands, getuidgid and userenv-fromenv. It has also gained options for not setting SHELL and USER/LOGNAME. setlogin sets the login account that is associated with a kernel session, as returned by the logname command. printenv is roughly equivalent to the conventional tool of the same name, except that it is a nosh/exec built-in command and that it supports several forms of output (including properly quoted rc.conf form, NUL-terminated form, and envdir form) in addition to the conventional human-readable form. This built-in command makes a common idiom easier. When combining clearenv, read-conf/envdir, and printenv to read a configuration setting, before the advent of the built-in command one had to employ `command -v printenv` (because clearenv unsets PATH). Now one can invoke it as simply printenv. One common use of this idiom is by the toolset's own build system and by the external configuration import subsystem, to read things like the amalgamated /etc/system-control/convert/rc.conf and an os_version file. Further to this, the amalgamated rc.conf now has an os_version setting on Linux operating systems, consolidating the code for obtaining that in one place. console-flat-table-viewer is a full-screen TUI viewer for various sorts of common flat database tables. It decodes the vis(3) encoding that is employed in various FreeBSD system tables. It also handles tables that use the standard ASCII US, RS, GS, and FS characters. File separators permit a form of continual update and redisplay if used in combination with pipes. local-stream-socket-connect is the AF_LOCAL socket equivalent of tcp-socket-connect. Improvements to existing tools and bug fixes ============================================ The Z shell completions now function better, and now cover a lot more of the commands in the toolset. systemd service unit conversion has been modified to make use of the new environment commands. The conversion tool in particular makes use of these when converting per-user Desktop Bus services. The EnvironmentUser extension has been replaced by an EnviromnentUserOnly extension, so that User and Group are consistently the sources of the user account and primary group. Additional settings now supported by convert-systemd-units include RuntimeDirectoryGroup, RuntimeDirectoryPreserve, WantsMountsFor, AfterMountsFor, and RequiresMountsFor. convert-systemd-units now also supports %T, %V, and %E expansions and snippets files. By analogy to \S, the login-banner command now also recognizes the \N sequence. The external configuration import subsystem now generates per-user service bundles that import user-wide environment variables from a ${HOME}/config/service-bundles/common pseudo-bundle, allowing users to maintain a single environment directory that affects all per-user services. One can thus use rcctl or system-control set-service-env against this common pseudo-bundle to set environment variables in all per-user services. Other improvements to external configuration import include the automatic generation of service bundles for dbus services, at both the system-wide and per-user levels. The N and P actions of console-multiplexor-control when applied to kernel virtual terminals now work properly. The external configuration input subsystem now imports defaultrouter properly from the amalgamated rc.conf. The cleanX service no longer generates incorrect symbolic links in /tmp. ttylogin services on user-space virtual terminals were sometimes (depending from exactly what order things ran in) causing a loop because of vhangup(). This has been fixed. And a spelling error in the names of the UNIXREMOTEEUID and UNIXREMOTEEGID UCSPI-UNIX environment variables has been fixed. Input methods ============= * http://jdebp.eu./Softwares/nosh/japanese-input-methods.html Providing a "front-end processor" for running "input methods" has been a to-do item on the roadmap for a long time. It has always been a goal to provide more than just U.S.-centric mechanisms in user-space virtual terminals, and this already ranges from allowing one to use fonts with large Unicode glyph repertoires through to providing the full ISO 9995-3 common secondary keyboard group. The problems with input method support were that all of the existing systems that I could tie into required X11 servers, X11 libraries, or direct low-level access to the framebuffer. Then I discovered OpenVanilla. The .cin file mechanism is table-based, does not involve plug-ins needing direct drawing access to the framebuffer, and is common across that and at least 8 different other tools. It is now common to user-space virtual terminals, too. Just as one can take a SCO Unix or FreeBSD kbdmap file and (after conversion) use it with user-space virtual terminals, one can take other people's existing .cin files and use them. Other people have written quite a lot of them, moreover. The blurb page on Japanese input methods hyperlinked earlier focuses on Japanese, but there is a wide range of .cin files available from plenty of sources, from Hangeul Jamo through Array40 to Esperanto. A new console-input-method command is at the heart of new input method services, which plumb in to user-space virtual terminals in between the terminal emulator(s) and the realizer(s). The pre-supplied "head0" user-space virtual terminal now has an input method service plumbed in. The nosh Guide documents input method front-end processors. And a new console-input-method-control command can be used to control them in a few basic ways. In line with the philosophy of adopting and adapting existing ideas, visible from the aforementioned keyboard map and .cin files as well as the whole building upon the architecture of daemontools notion, the input method front-end processor provides various control key chords and function keys that are roughly compatible with other systems, such as Microsoft's Japanese IME in Windows and the OSF/1 IMLIB. One can, of course, drive it with the extra keys that are dedicated to the purpose on a JIS 106/109-key keyboard or a Korean 103/106-key keyboard. There should be not too much change required to existing typing habits. Other terminal improvements =========================== In addition to the aforementioned documentation of front-end processors, the Guide also now documents how "head0" works (although this is just one way in which one can set up virtual terminals) and has a new chapter on some of the choices of fonts, keyboard maps, and input methods that are available from various sources for user-space virtual terminals. (In a separate project, I have been working on fixes to FreeBSD's vtfontcvt that permit it to convert Ubuntu Monospace, which as the Guide documents it currently cannot handle.) * http://jdebp.eu./Proposals/linux-kvt-manual-pages.html There are new manual entries for TERM(7), linux-vt(4), linux-console(4), and TerminalCapabilities(3). No utility other than console-ncurses-realizer uses NCurses and terminfo any more. They have flaws in how they model real terminals and terminal emulators. All other TUI tools are built around a different library, that decodes terminal input using a proper full UTF-8 ECMA-48 state machine rather than by limited and faulty pattern matching, and that is geared towards primarily ECMA-48 and ITU T.416 output (with various common DEC private extensions). This enables better handling of cursor, editing, and calculator keypad keys with modifiers; full recognition of "application" and "normal" modes on the cursor and calculator keypads; full use of 24-bit colour where the terminal supports it; and use of the DEC Locator or XTerm mouse. One such is a new realizer, console-termios-realizer, another realize-a-terminal-on-a-terminal realizer that is more capable than console-ncurses-realizer. console-fb-realizer now has better fallback behaviour in the event of not being supplied a keyboard map, which has been a fairly common configuration error. It now falls back to the U.S. English International keyboard map, rather than to an empty one where no keys do anything. Other improvements lie in extended keys, on the cursor and calculator keypads. console-terminal-emulator now fully supports individually switching these two keypads between "application" and "normal" modes, and fully supports separate control sequences sent by keys for each mode. To align with this, the "fkey49" to "fkey61" actions in SCO Unix/FreeBSD keyboard maps have changed in console-convert-kbdmap to distinguish the cursor keypad from the calculator keypad, and console-fb-realizer collaborates in ensuring that the terminal emulator(s) receive as much modifier state information accompanying such keys as is appropriate. The keyboard map files themselves now follow a new naming convention. The external configuration import subsystem will not delete any old keyboard map files that you might still be using in a running realizer service, so it is left to you to clean them up. The new naming convention allows for, say, different "Japanese" maps to be used with a 101/104-key keyboard and a 106/109-key keyboard. The user-space virtual terminal subsystem now also has the notion of accelerator key input messages, i.e. alphanumeric key input with the ALT key modifier. These are recognized by the realizers and the terminal emulator generates escape sequences for them. Note that these escape sequences, which are neither ECMA-48 nor DEC VT/SCO standard, conflict with ECMA-48 7-bit control aliases. console-decode-ecma48 is a debugging aid for text sent to/from terminals. It decodes a sequence of ECMA-48 characters, control characters, escape sequences, and control sequences into a human-readable representation. It can handle both ECMA-48 output (as sent to terminals) and ECMA-48 input (as received from terminals), and recognizes as an extension various common additional control sequences, such as the control sequences for function keys generated by the SCO Unix console, and a bunch of DEC VT private control sequences. * https://unix.stackexchange.com/a/491883/5132 console-control-sequence provides a human-readable way of emitting common ECMA-48, DEC VT, AIXterm, and XTerm control sequences. It is aliased as setterm and is to a large extent a workalike for the setterm from the util-linux package, except that it is portable where util-linux is not. It is both portable to other operating systems, and portable to other terminals. It does not implement the few things that util-linux setterm does that are specific to Linux and to Linux's built-in teminal emulator, but conversely it implements quite a number of standard ECMA-48 and DEC VT control sequences that the util-linux setterm does not. These include turning the calculator keypad to/from application mode; switching to/from the XTerm alternate screen buffer; turning DEC Locator reports on/off; turning XTerm mouse reports on/off; setting the cursor shape; changing the mappings of the Delete and Backspace keys; using the strikethrough, italic, and invisible attributes; turning background colour erase on/off; setting the underline type; indirect 8-bit colour; direct 24-bit colour; and DEC VT soft reset. vc-reset-tty and console-resize are now implemented by invoking console-control-sequence, removing the hardwiring of control sequences from both. console-terminal-emulator now recognizes a few additional control sequences, such as DECST8C (used by console-control-sequence to set regular tabstops if the interval length is 8). It also has tabstops at 8 column intervals in its reset state. Both the FreeBSD and Linux kernel virtual terminals do this, even though real DEC VTs do not, so user-space virtual terminals do too. The --keep-term option to clearenv now retains a few more terminal-related environment variables. =========== This way up And finally, a feature that you will not want. * https://unix.stackexchange.com/a/465166/5132 Over many years, people have regularly asked for a terminal emulator where the line progression goes from bottom to top. The idea that they have is that they will find this easier than top to bottom. In response to a recent question in the same vein on Stack Exchange, I have given console-fb-realizer, console-ncurses-realizer, and console-termio-realizer the ability to realize displays with the line progression reversed. Having used this, it seems to me that people only want this because they have not tried it. Trying it makes a convincing case for not using it. So I have retained this feature in order that, at long last, there is an actual terminal system with this feature that people can try, and come to the same realization. (-: From owner-freebsd-hackers@freebsd.org Wed Jan 23 05:38:59 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B31714BBD32 for ; Wed, 23 Jan 2019 05:38:59 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1FFE88B2A7 for ; Wed, 23 Jan 2019 05:38:59 +0000 (UTC) (envelope-from peter@rulingia.com) Received: by mailman.ysv.freebsd.org (Postfix) id D499B14BBD31; Wed, 23 Jan 2019 05:38:58 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C12EC14BBD30 for ; Wed, 23 Jan 2019 05:38:58 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vtr.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D23908B2A6 for ; Wed, 23 Jan 2019 05:38:57 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vtr.rulingia.com (8.15.2/8.15.2) with ESMTPS id x0N5cjIB029984 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 23 Jan 2019 16:38:51 +1100 (AEDT) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id x0N5cdBe017002 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 23 Jan 2019 16:38:39 +1100 (AEDT) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id x0N5cd1R017001 for hackers@freebsd.org; Wed, 23 Jan 2019 16:38:39 +1100 (AEDT) (envelope-from peter) Date: Wed, 23 Jan 2019 16:38:39 +1100 From: Peter Jeremy To: hackers@freebsd.org Subject: Cross-building CPU variants Message-ID: <20190123053839.GC75994@server.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="gBBFr7Ir9EOA20Yy" Content-Disposition: inline X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.11.1 (2018-12-01) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 05:38:59 -0000 --gBBFr7Ir9EOA20Yy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable A have an AMD GX-412TC (Jaguar) and would like to cross-build an optimised version (ie CPUTYPE?=3Dbtver2) of FreeBSD (and ports) on my Intel IvyBridge system. Unfortunately, this fails because CPUTYPE?=3Dbtver2 causes Clang to generate instructions that aren't supported on IvyBridge. Does anyone have any suggestions on how to make this work? --=20 Peter Jeremy --gBBFr7Ir9EOA20Yy Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAlxH/d9fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzRPbQ/8Dn2Z/G4/s+Z73HBKXeoK3MCpCvUvjhhIcrxsWsPbmnskf1cCwWnisUbq W/4E7bnfTymYLw7Ozv79fDE9eImf0Om9Bh9AjCcn1q0EC/D74/apqcdz1dXc3Sn8 mHG4V9vf9xcXhugYORFyU2QZ78zFqAGw4CeWDrvU5UNRLMrsyrulfxbOWhOqNj4w yAD3D7isQx8XZy/JyGmbnIl5eyd4WprJ54XoYBtgaWZsh15H3Gxh9b/5+Oi9eOch qPoCC5yUjqaOLBSqFU+8c/xJ/9WROx5Tjo45w28gu6Cu9JkZezciBrUTbjgRSShK UuGjWG9RcRxGXoLntdMoJqmC7h+qLYWyi+nz32eez8BQ0AKJ0qTr9RbdrN/7blux AHNfcbcJtWQ8SsRz+5QuMrgAn9Yi6CMwDout3tJHOQBJH+P4F3fKWpNoeI8xejID R/RxaCQ7zAeIWe5rK/OwpMuVwvfNd5TRxcbLU4oqJOdjikYR2sTlqVJw6ESqEEKw EiswNjWZe07wD7CIdwWPfPgWJnhzfvjQqC8ocdQQ4zZy4UhaXyezYoOs061xUXHW O/Q1A6gC9+eUc6eYwXs1FXznOg/2A3lCInSa5znycNy7Qnu9UJRgOduNFTo0Qez/ 5SXAXmF5SyLzSlJWZmepVZxpvQmwonPDyOxZXDskDdzin5KEiiM= =iSfY -----END PGP SIGNATURE----- --gBBFr7Ir9EOA20Yy-- From owner-freebsd-hackers@freebsd.org Wed Jan 23 07:52:55 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 908DB14BE83D for ; Wed, 23 Jan 2019 07:52:55 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3BD918F062 for ; Wed, 23 Jan 2019 07:52:53 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id x0N7dkgB059920 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 23 Jan 2019 08:39:46 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id x0N7dfk9059905; Wed, 23 Jan 2019 08:39:41 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Wed, 23 Jan 2019 08:39:41 +0100 (CET) From: Wojciech Puchar To: Mark Saad cc: FreeBSD Hackers Subject: Re: CPU Isolation In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 3BD918F062 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-6.87 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[puchar.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[puchar.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; IP_SCORE(-3.58)[ip: (-9.44), ipnet: 194.1.144.0/24(-4.72), asn: 43476(-3.78), country: PL(0.03)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 07:52:55 -0000 man cpuset On Tue, 22 Jan 2019, Mark Saad wrote: > All > I am looking to setup a 12-STABLE box in a way that is similar to a > CentOS server I am running. > That server is setup using the boot option isocpus, to remove > cores/cpus from the kernel scheduler at boot. What I want to try on > 12-STABLE is to set aside some cores to run process and some kernel > threads that will only always run on a dedicated core with out the > chance for being preempted / interrupted by something else on that > core / cpu . What I do not see is a way to evict processes off a > running cpu post boot or a way to isolate them preboot. Am I missing > something? > > -- > mark saad | nonesuch@longcount.org > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@freebsd.org Thu Jan 24 04:58:00 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EE0B14C28B8 for ; Thu, 24 Jan 2019 04:58:00 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8EC028118F for ; Thu, 24 Jan 2019 04:57:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 4CE8F14C28B7; Thu, 24 Jan 2019 04:57:59 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A40E14C28B6 for ; Thu, 24 Jan 2019 04:57:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com [IPv6:2607:f8b0:4864:20::734]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B1E778118E for ; Thu, 24 Jan 2019 04:57:58 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x734.google.com with SMTP id 68so2489407qke.9 for ; Wed, 23 Jan 2019 20:57:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=vgIoAEa0avXhFalWz0HGtuR8ljJj7G74a0oP+OvaG9c=; b=dv54/8eQK6wxkzIXzUVtEQehnIPZNBwdUyt/p0Nhyd+7G5VjJ6zwSE9cdcErkJ9pdR P2jZAimXshaJlQCDoGvrdVwyU6ltPCLMorPbICx6o/e1wuBDhp5Wh/xcKX0WYbh7RScK 8L01rKLLpPRlnJL+m47C/MIKXsJGMmVzYomcQersx947S/ryHrXGp5apTDfCcM2NJkHd OM2g9CwYQ73k08FXaHGnHj0r4VmIW28ICuIlgcF0v72Z5q961MgY6vAg6JIZLsAqn6vU 7T3ZPmpR5BiqED38I2MHUYsLaBRRFKa9vJ/llIxlPnGpxqvx2rfjXv6gqWHq3jHCoJ/8 JpXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vgIoAEa0avXhFalWz0HGtuR8ljJj7G74a0oP+OvaG9c=; b=LITHNZTTNgVG4vWITXFYD+qf9PEna4KA1mfUlc7bzsFe8jZuUreNNen8Zff01jMR1l lWxkNOdjZjjUUUcnenmi7vwJPAYPxhvkLIOn0HNLWyuWn5LDaLIJk8o5vQb64aKQ/a37 0hBtzAgUyTJZLvPxzrSMBWJCCIwJCgDcXrW62e1hM1JxK4kMI5oQhewgtiHPibuRRJbu JUMY6QV6kcZ6suMXDb5EfGIFOn6x2Fb5+uMe7PaTybXnIfx3u2DtjvIXEXUHwNdj1oWF n1k36OEpFJ+1HIZTKdT3LQtEts5dFfd5bUCNp3GkKtRtn3jvE02aU2+4CLNcOq+h2jxi 3NoA== X-Gm-Message-State: AJcUukeu5VdZ52UryALoBiSFFhDsd3sEp/Pf5WYCHo79KL6dEobjkL0g ADXW6+3cSmKm9ML70qgPxaZZ7HuVwTjo+nRd/pc+HA== X-Google-Smtp-Source: ALg8bN67ZHWG5wMMW1bbqY7L0pv/0knxZvU/6AJU2qT+IY+mKIJyLccFVjyRpdAvbL1jzHTzmoNYsz4IosEU3sHv1ds= X-Received: by 2002:a37:9604:: with SMTP id y4mr4512383qkd.279.1548305877786; Wed, 23 Jan 2019 20:57:57 -0800 (PST) MIME-Version: 1.0 References: <20190123053839.GC75994@server.rulingia.com> In-Reply-To: <20190123053839.GC75994@server.rulingia.com> From: Warner Losh Date: Wed, 23 Jan 2019 21:57:46 -0700 Message-ID: Subject: Re: Cross-building CPU variants To: Peter Jeremy Cc: "freebsd-hackers@freebsd.org" X-Rspamd-Queue-Id: B1E778118E X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.95 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.95)[-0.947,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 04:58:00 -0000 On Tue, Jan 22, 2019 at 10:41 PM Peter Jeremy wrote: > A have an AMD GX-412TC (Jaguar) and would like to cross-build an optimised > version (ie CPUTYPE?=btver2) of FreeBSD (and ports) on my Intel IvyBridge > system. Unfortunately, this fails because CPUTYPE?=btver2 causes Clang to > generate instructions that aren't supported on IvyBridge. > > Does anyone have any suggestions on how to make this work? > I think TARGET_CPUTYPE=byver2 is the magic you are looking for. Warner From owner-freebsd-hackers@freebsd.org Thu Jan 24 16:33:12 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4EACA14B4B70 for ; Thu, 24 Jan 2019 16:33:12 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A85C71876 for ; Thu, 24 Jan 2019 16:33:11 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-io1-xd2b.google.com with SMTP id s8so5190687iob.13 for ; Thu, 24 Jan 2019 08:33:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=6n/YT1cMd3SBt1DmBIq/5qF1CPjhFPbhPxw9XHfSiZk=; b=j5hRJcj2oCSRrxWDd/dasf+6qZsnVkEO7U7QFhgcZi4ts3kj2LyhFXbe85XVWzJpcS U0jVCG/8rmP1I9WCjiqI7UIHxW6HzKieoKX/Oc4MkBHtkZUSNH0Gt91locoYT/NXkV/7 o387s/sRjlqaCMaQUFarVE9PJOU605NWsIG6uCjmSw5UPzeuCfe+LLJcoY2mmWE2hGkM 2sg7Snd/WYHFfjuqwLvFv5oiIDdrVdGVEPaCCkASJ/9+9Lo5RYcgP+cz/4gqdMICLcSf zxIf5XTzuf25lJgrqjEyXi5jhUyVJLDgmctY7/FAMiq5YgWw/FqLFuVcSZk46PNS/GlD a+cA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=6n/YT1cMd3SBt1DmBIq/5qF1CPjhFPbhPxw9XHfSiZk=; b=jOJeABYkA78PvWj5fI/rG/DhH242o4OfLz2ZRjbjNqzIMfgO5ny1L+Coy0O2unCS/V X7rMNr57cjunqk2I2aa52UM6pvSN8FWq0zVVZJWnSqpXAzytkdvLHM3zz6Cl4VK6sdTx UlULcawLZbLLahrh42AcDbt0WxoCQKn4H8k81GDWusdtdJlKXXjygsAHpdONUqtj4sRV nRCjfdEVCHziB6U9lBYF1cwjtzuhnu5M/SS92/CacwM7sL/gMM6u+ZSwBhqvg9qNpK3a lL+ToqFwCNyWj0Vg3vDLhknxjelSIJcnAMx+RAmtMRYNu7F1EBJDE3SZKVATzZAItoOu 6SKA== X-Gm-Message-State: AHQUAuaodC+clD6QlcgVYE7tc1xY7b1lBnkrE/IrlnzdHmw0Iz/t2dXX YF2DG7rWVYGEF5l/YW9QhdZ5vfvARx03+vyP9BPyo78= X-Google-Smtp-Source: ALg8bN4Sbe5aouHNBmgMOPjzNosNr5N/4/7ZozJ3Rmkwh+ZEGtq6EKfaHuitf+T0DjV3+kq+w7AXaCPD0bpX6kA+QaI= X-Received: by 2002:a5e:c802:: with SMTP id y2mr4172408iol.198.1548347590074; Thu, 24 Jan 2019 08:33:10 -0800 (PST) MIME-Version: 1.0 From: Zaphod Beeblebrox Date: Thu, 24 Jan 2019 11:32:59 -0500 Message-ID: Subject: Scheduled ideas: Two threads with one "stone" To: FreeBSD Hackers X-Rspamd-Queue-Id: 1A85C71876 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=j5hRJcj2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of zbeeble@gmail.com designates 2607:f8b0:4864:20::d2b as permitted sender) smtp.mailfrom=zbeeble@gmail.com X-Spamd-Result: default: False [-6.59 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[b.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(-2.69)[ip: (-8.98), ipnet: 2607:f8b0::/32(-2.48), asn: 15169(-1.91), country: US(-0.08)]; NEURAL_HAM_SHORT(-0.89)[-0.894,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 16:33:12 -0000 W.R.T. hyperthreading, it's come to my attention that FreeBSD has had some recommendations for awhile --- namely hyperthreading is bad for secure systems. That-is-to-say this is not about recent vulnerabilities of recent Intel flavoured CPUs, but rather some thoughts on a different solution to a different problem. I was building world on my laptop when I noticed how sucky it had become. Now... I well know this is due to the fact that even a "nice -19 make -j32 buildworld" will slow the normal priority userland down by about 1/2 because the thread that is "me" on the CPU is competing with one of the threads that is saturating the CPU. My normal priority thread will get scheduled, but it competes equally for resources with threads at much lower priority. It then struck me that the fix for this problem would also address the hyperthreading side channel attack. It struck me that two simple rules would apply to scheduling hyperthreads (rather than the global "turn them off for security" ... To be scheduled on a paired hyperthread, the candidate thread should: 1. Belong to the same user 2. Be the same (or the same within a margin) effective priority My argument here for (1) is that it makes the hyperthreading side channel attack moot. The same user has many ways to access the other process... the hyperthreading attack would just be a waste of time. My argument for (2) is that it would restore the experience of responsiveness the local machine that we have on non-hyperthreading processors. From owner-freebsd-hackers@freebsd.org Thu Jan 24 17:34:34 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9469C14B6E19 for ; Thu, 24 Jan 2019 17:34:34 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from mail.rlwinm.de (mail.rlwinm.de [IPv6:2a01:4f8:171:f902::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A6D9274ACE for ; Thu, 24 Jan 2019 17:34:33 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from crest.bultmann.eu (unknown [IPv6:2a00:c380:c0d5:1:1db3:cc65:1157:29d8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.rlwinm.de (Postfix) with ESMTPSA id 1CEA449E2 for ; Thu, 24 Jan 2019 17:34:21 +0000 (UTC) Subject: Re: Scheduled ideas: Two threads with one "stone" To: freebsd-hackers@freebsd.org References: From: Jan Bramkamp Message-ID: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> Date: Thu, 24 Jan 2019 18:34:21 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: A6D9274ACE X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of crest@rlwinm.de designates 2a01:4f8:171:f902::5 as permitted sender) smtp.mailfrom=crest@rlwinm.de X-Spamd-Result: default: False [-3.79 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DMARC_NA(0.00)[rlwinm.de]; MX_GOOD(-0.01)[mail.rlwinm.de]; NEURAL_HAM_SHORT(-0.52)[-0.515,0]; IP_SCORE(-0.97)[ipnet: 2a01:4f8::/29(-2.49), asn: 24940(-2.33), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 17:34:34 -0000 On 24.01.19 17:32, Zaphod Beeblebrox wrote: > W.R.T. hyperthreading, it's come to my attention that FreeBSD has had some > recommendations for awhile --- namely hyperthreading is bad for secure > systems. That-is-to-say this is not about recent vulnerabilities of recent > Intel flavoured CPUs, but rather some thoughts on a different solution to a > different problem. > > I was building world on my laptop when I noticed how sucky it had become. > Now... I well know this is due to the fact that even a "nice -19 make -j32 > buildworld" will slow the normal priority userland down by about 1/2 > because the thread that is "me" on the CPU is competing with one of the > threads that is saturating the CPU. My normal priority thread will get > scheduled, but it competes equally for resources with threads at much lower > priority. You can use `idprio 10 ...` to minimize the impact of CPU bound tasks. If you want to go even further you could apply hierarchical resource limits on the make process. The idprio chainloader puts make and all its descendants into the idle scheduling class which allows normal time sharing threads to preempt them. From owner-freebsd-hackers@freebsd.org Thu Jan 24 17:58:45 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C85C14B7A7B for ; Thu, 24 Jan 2019 17:58:45 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0981675FAF for ; Thu, 24 Jan 2019 17:58:44 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wm1-x331.google.com with SMTP id f188so4004575wmf.5 for ; Thu, 24 Jan 2019 09:58:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=8JMEP0iElhHxSwxjbm/d1y+xqmuofgl7xu/hz1l3qKQ=; b=qPf0X/I8z7RNESDkUE9phXx+uNbkiHM7RcZJHJkqt1Aa4YdUHqohW4hGONM472l5UW w0k8aPKkGiZPuRaQfrRxLKBxu9K5tqs2u4kioDlDDEO/s1cFco6S7V3rnK6vVwYiVcxj Agd41V945LKpi60llo4PYL7yrMaJETT0jHR40rN4mobHNxyqxhH4hnUtntX7YOOJk4wG bW9n/dwlHGGrZIe4IIHROuT2k8yaHmHSqzj6PicLAlSocs73Ox+IojRY/lqYYmOuwnGh dRauE+b4Eqf6TQs0CBmjjVyyHwpZldZJQja95SAUaue4nsNiDVP8tYBjAaUzCEe2DVmG uvTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=8JMEP0iElhHxSwxjbm/d1y+xqmuofgl7xu/hz1l3qKQ=; b=eXdvePl6wjWmffhWEP1xGyVDOTVruQ4Rl7kCK4P/QlVAmosOax3vEriMC6+1q0gHIx h2ZpKZ3xZ42Ts4I6bhrPU5fyix6P/BZH0nI4DUH/5LAjkFyFDZRpAg1N7p41rYL4/+f8 AdPsBXtRgVPBibaXabCTJN/UK+PNz7dv3D2KVpjEgJugKhWuBPDFcGdpfVzW8Xox4cbn /ckipAk0JdQ9fdwarU1SyCz4s8VeU5WMls58B3wkiqfyVeJ5Ig9/8A6btFNl5It03OtJ mJVdyfeg8N2Vjc0gi8E/OJBjimx0pkM46NHHJH1yaopgS5ZEfHdRQh6pDggK1WIWylut xrwg== X-Gm-Message-State: AJcUukf8CdrBdOPcKcRF2jFPugkVPK/UcmwWl4wljLd+S8eR3W/Zi/vL VccM0YNVDaJFsQtPSl8r3n44mFIf X-Google-Smtp-Source: ALg8bN4/UE74gkDS+L9/Ons5iimg8ThA8iRQENXg+2aCoLalB6Et4VEN+22aRcP2/qM6THUJM96qZA== X-Received: by 2002:a1c:8d12:: with SMTP id p18mr3920290wmd.31.1548352722373; Thu, 24 Jan 2019 09:58:42 -0800 (PST) Received: from ernst.home (p5B0239C2.dip0.t-ipconnect.de. [91.2.57.194]) by smtp.gmail.com with ESMTPSA id s3sm45123078wmj.23.2019.01.24.09.58.41 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 24 Jan 2019 09:58:41 -0800 (PST) Date: Thu, 24 Jan 2019 18:58:40 +0100 From: Gary Jennejohn To: freebsd-hackers@freebsd.org Subject: Re: Scheduled ideas: Two threads with one "stone" Message-ID: <20190124185840.0973ea7f@ernst.home> In-Reply-To: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> References: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.1 (GTK+ 2.24.32; amd64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 0981675FAF X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=qPf0X/I8; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gljennjohn@gmail.com designates 2a00:1450:4864:20::331 as permitted sender) smtp.mailfrom=gljennjohn@gmail.com X-Spamd-Result: default: False [-6.64 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[gljennjohn@gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.90)[-0.899,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FREEMAIL_REPLYTO(0.00)[gmail.com]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.73)[ip: (-9.47), ipnet: 2a00:1450::/32(-2.21), asn: 15169(-1.91), country: US(-0.08)]; RCVD_IN_DNSWL_NONE(0.00)[1.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 17:58:45 -0000 On Thu, 24 Jan 2019 18:34:21 +0100 Jan Bramkamp wrote: > On 24.01.19 17:32, Zaphod Beeblebrox wrote: > > W.R.T. hyperthreading, it's come to my attention that FreeBSD has had some > > recommendations for awhile --- namely hyperthreading is bad for secure > > systems. That-is-to-say this is not about recent vulnerabilities of recent > > Intel flavoured CPUs, but rather some thoughts on a different solution to a > > different problem. > > > > I was building world on my laptop when I noticed how sucky it had become. > > Now... I well know this is due to the fact that even a "nice -19 make -j32 > > buildworld" will slow the normal priority userland down by about 1/2 > > because the thread that is "me" on the CPU is competing with one of the > > threads that is saturating the CPU. My normal priority thread will get > > scheduled, but it competes equally for resources with threads at much lower > > priority. > > You can use `idprio 10 ...` to minimize the impact of CPU bound tasks. > If you want to go even further you could apply hierarchical resource > limits on the make process. The idprio chainloader puts make and all its > descendants into the idle scheduling class which allows normal time > sharing threads to preempt them. > Or switch to SCHED_4BSD. I ran your command line and was still able to watch a HD movie in full screen mode without a single glitch. My Ryzen has 6 physical cores and 6 SMTs. -- Gary Jennejohn From owner-freebsd-hackers@freebsd.org Thu Jan 24 18:11:12 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 368B514B86A0 for ; Thu, 24 Jan 2019 18:11:12 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-it1-x132.google.com (mail-it1-x132.google.com [IPv6:2607:f8b0:4864:20::132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C05476C3D for ; Thu, 24 Jan 2019 18:11:11 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-it1-x132.google.com with SMTP id p197so5868464itp.0 for ; Thu, 24 Jan 2019 10:11:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=wXUvV+8GT3e/ClD9IlLfUFgJ55fyudjqugOqJU0qpbE=; b=uFIByv+dlR0iUigwbZ81bXj70I9Kpl3hMnddHI97exfVcjekzrIuH+3Dn7x3harJ+0 Ii+LMQjqwtwpto4mAgoOO1M1jnz+tZD/OTI7PY742qeR3XP48sazFcFdQ0FsOiMwgTkt sWKmo6nQfG6mv9Ay4RuBLpr478Ni8kycu2BqadvVIxxWpRrymlrpS9qw7z1BZU7LrdU1 98kHvzeaLfTzIDN0II12tNbGgWJfo8X3Sncl4QYHq5acVKGO6aoxFE1FRA8bgHxYkTUN yFX62avznxnScXtTsQkL/Qt1rze4rCoZ9mfeUJ15DL9yPzdE6UKuJAK/IxaolAuTVhL5 b1zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wXUvV+8GT3e/ClD9IlLfUFgJ55fyudjqugOqJU0qpbE=; b=nvVEN5Xcztc6wSfCLGIwkLoL8Rh5JyYEPus3Gb/gbaU2OsNUEVDGRW4Twf9tHLg8x9 Tp4gIGV3FONCvxahz7p1iQTG9ihgF8wdnfs9F7GTKaRHOlczbWFZNAwpYPpJeo96zejM y6ohYOQCziu8FCaItCSD97wsW6hxbCBIrw7NHIUatUm3mXkwYrs/ZPmv5UJewCZjOXrm CnqLf8t5gLoUYUfGA18aUEHhfpPMiZc4u+MgKp9yhXkIw/CVjli7q8RzZFTrlF1ghLkl eNDSAhW/moz5faqxmJdqP9C+DQfCpHLXUQdsWnMHAb4Z/8H5U1SVkKXy7cbi9qiRNy6L dgZQ== X-Gm-Message-State: AJcUukfcY7knf2Zor/C0P4qg0btTQGT6RaxOzI/0GwlgQJAYiIkblbIA YzlVCsr3h83HHX6kTl4PctyXborX19MIDYBbcZj0YtY= X-Google-Smtp-Source: ALg8bN4glLt5uEj3BBq7Z6puJvuiBhkiD3B3v8jlXyqr0DUQpyxEWM49IJ/2gImph3VM/wNRes/F+VupZcXx3D9uhO0= X-Received: by 2002:a24:214a:: with SMTP id e71mr2200856ita.60.1548353470379; Thu, 24 Jan 2019 10:11:10 -0800 (PST) MIME-Version: 1.0 References: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> In-Reply-To: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> From: Zaphod Beeblebrox Date: Thu, 24 Jan 2019 13:10:59 -0500 Message-ID: Subject: Re: Scheduled ideas: Two threads with one "stone" To: Jan Bramkamp Cc: FreeBSD Hackers X-Rspamd-Queue-Id: 5C05476C3D X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uFIByv+d; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of zbeeble@gmail.com designates 2607:f8b0:4864:20::132 as permitted sender) smtp.mailfrom=zbeeble@gmail.com X-Spamd-Result: default: False [-6.48 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.90)[-0.900,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; IP_SCORE(-2.57)[ip: (-8.37), ipnet: 2607:f8b0::/32(-2.48), asn: 15169(-1.91), country: US(-0.08)]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 18:11:12 -0000 On Thu, Jan 24, 2019 at 12:36 PM Jan Bramkamp wrote: > On 24.01.19 17:32, Zaphod Beeblebrox wrote: > > W.R.T. hyperthreading, it's come to my attention that FreeBSD has had > some > > recommendations for awhile --- namely hyperthreading is bad for secure > > systems. That-is-to-say this is not about recent vulnerabilities of > recent > > Intel flavoured CPUs, but rather some thoughts on a different solution > to a > > different problem. > > > > I was building world on my laptop when I noticed how sucky it had become. > > Now... I well know this is due to the fact that even a "nice -19 make > -j32 > > buildworld" will slow the normal priority userland down by about 1/2 > > because the thread that is "me" on the CPU is competing with one of the > > threads that is saturating the CPU. My normal priority thread will get > > scheduled, but it competes equally for resources with threads at much > lower > > priority. > > You can use `idprio 10 ...` to minimize the impact of CPU bound tasks. > If you want to go even further you could apply hierarchical resource > limits on the make process. The idprio chainloader puts make and all its > descendants into the idle scheduling class which allows normal time > sharing threads to preempt them. > No. This is insufficient to the task. If your goal is that the "idle" or "low priority" task does not effect the high priority tasks, this is insufficient w.r.t. hyperthreading. In hyperthreading (to discuss just one variant of many), two CPU cores share all of their functional bits. This is useful because, in general, CPU functional bits are fairly idle. Even if both threads on hyperthreaded pair are "runable" right now, they can be fetching from memory or, in the exreme, one could be issuing floating point and one could be issuing integer, IIRC. The problem I'm proposing to solve is when the two processes are of sufficiently differing effective priorities (however they get to that point) that they not be scheduled on the same hyperthreaded pair. The reason for this is that beyond scheduling the processes to the cores, we have no control over the resource contention of the cores --- and by that I claim that the lower priority process can force the higher priority process to wait as it consumes resources shared by the two cores. In an ideal world, the hyperthread interface on the cpu would allow some combination of prioritizing cores and their resource use, but that is not how Intel's chips currently work. To be clear, and this is easy to experience, find a FreeBSD desktop with a i7 (or similar) running X and (say) kde. Not something lightweight. Make sure it has enough RAM (16 or 32 big is probably good for this experiment). When idle, take a "feel" on the interface. No lag, things are spiffy. Now, go to /usr/src. Make sure it's there. Then run "nice -19 make -j32 buildworld" there. For extra credit, you can even use idprio if you like. Now... duplicate your feel test. The interface will be laggy. Things will be slower. If you look at the process table, it's not that your effective priorities are coming anywhere close to those from -19 ... or even if you use idprio... still laggy. What is happening here? Think about your processor. FreeBSD will ensure that your normal priority processes are scheduled first on a few of the 8 cores that show with an i7. But chances are very high that your processes are landing opposite low priority processes on the hyperthreaded pairs. This means that your effective processsor use is being cut in half (-ish) by the hyperthreaded processes grabbing the shared resources roughly half the time. You can game out having two processes ready and how often they fall on the same pair vs. falling on non-paired. You can even do the thought experiment of higher priority processes being placed on cores with pairs deliberately not used until all non-pared cores are in use --- urm... this is a big diversion... then we have the opportunity cost of the then partially unused resources ... etc. I should stop there. There are simple things that make a big difference before that digression produces many small things with even smaller effects. From owner-freebsd-hackers@freebsd.org Thu Jan 24 18:13:02 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D15D14B8BCD for ; Thu, 24 Jan 2019 18:13:02 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-it1-x12d.google.com (mail-it1-x12d.google.com [IPv6:2607:f8b0:4864:20::12d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B191E76F57 for ; Thu, 24 Jan 2019 18:13:01 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-it1-x12d.google.com with SMTP id a6so5822983itl.4 for ; Thu, 24 Jan 2019 10:13:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+L1QVQp2wxuCUfaDZOTWspLkjSSgPwtm9F18auNS6KY=; b=QtK1O344JewsHhNXp7olaDCh1+uoV8TnjCXWqFWQPbq0h/Zes1NGwX6ieWhLtG1z4W V1DcLD2e47gLcQmolbvp+IY5C+rooQb9gLOj/ZLhW+B+bMviU5NV+LGgT4Mjr/x2j/dS OMqkFNeGQVn4DyvtsRe4K0vkcEgsN4NrQN1x5hdV6zuc87hO2fhFLTCBEERj1Yg8V4C+ zBeJ+FZmJkrl/xoLQvxlheY9ZWTN+bMWnBvK4IglWGgcpV2WAh+sY+cul1qgtjOnuf9P h1ttOl0EfYRIYGe7/GOX27MTFU3x/lcUrY+AdyPHK13U6hwX+4B56rvHw4tglzLuysYv A14w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+L1QVQp2wxuCUfaDZOTWspLkjSSgPwtm9F18auNS6KY=; b=XJoahNNijxZn0jBqIHUHTyzq0kVDsACwYq8GMNvobyhKbVRQKC49Nb/D49zX9C/7af zjm5Cs1rnxiN6x/aKAHbU/4gu6jE3KiOIe7/i/LNY2dZyv9mEqyEsq/Y7J2B5GT3Jcio DzdiKpEadGfJmy+Dzli/2TimSqZH90iay6rH4OdNNfqrdX56u9D1mCnNW6FTlTHA2Li0 /hKm1EUVE4QHpfOQlJqC6HzukAhQwW05o2f4/f/l0S5ROoTEKE5HLQQpA/0wWHh3uclz wI4+aqxKYcr6Cync78y4TJ21oT8txGbmrXqTiM7dJAn/w7nUmFsFoMshV8emkWs7xaQJ WLYg== X-Gm-Message-State: AJcUukdmcViNyqjcDTpcky/Hca+vqs3BhS96rpigfb8N/JsbHZNnZG6j KebPFPHdA7EkCCP34mMs3NtFM22wH0qh4tGPVAgimqM= X-Google-Smtp-Source: ALg8bN4bwiDkk3LbSc5hMUx6t767r+xNgTYv9uXZGL8ZNrAAWsrErK3A19INwLyocslOBTOgV/TX8W7+d8ayyq4Xp1c= X-Received: by 2002:a24:d4c2:: with SMTP id x185mr2240621itg.34.1548353581052; Thu, 24 Jan 2019 10:13:01 -0800 (PST) MIME-Version: 1.0 References: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> <20190124185840.0973ea7f@ernst.home> In-Reply-To: <20190124185840.0973ea7f@ernst.home> From: Zaphod Beeblebrox Date: Thu, 24 Jan 2019 13:12:50 -0500 Message-ID: Subject: Re: Scheduled ideas: Two threads with one "stone" To: gljennjohn@gmail.com Cc: FreeBSD Hackers X-Rspamd-Queue-Id: B191E76F57 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=QtK1O344; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of zbeeble@gmail.com designates 2607:f8b0:4864:20::12d as permitted sender) smtp.mailfrom=zbeeble@gmail.com X-Spamd-Result: default: False [-6.46 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-0.90)[-0.896,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[d.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.56)[ip: (-8.30), ipnet: 2607:f8b0::/32(-2.49), asn: 15169(-1.91), country: US(-0.08)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 18:13:02 -0000 On Thu, Jan 24, 2019 at 1:01 PM Gary Jennejohn wrote: > > > Or switch to SCHED_4BSD. I ran your command line and was still > able to watch a HD movie in full screen mode without a single > glitch. My Ryzen has 6 physical cores and 6 SMTs. > This really isn't about "can you do X while building world" it's an argument about how unfairly we allocate CPU these days w.r.t. hyperthreading and related technologies. It's not "does this minimally work" it's about "can we do better" .... From owner-freebsd-hackers@freebsd.org Thu Jan 24 21:30:10 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C57714BF19E; Thu, 24 Jan 2019 21:30:10 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53CB987E77; Thu, 24 Jan 2019 21:30:06 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: by mail-qt1-f178.google.com with SMTP id l11so8518453qtp.0; Thu, 24 Jan 2019 13:30:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=hUKdz919+mEPW5W6Aoll9C+ddUwkBKh8hPpHgAbLvL0=; b=tsbprUIzj71qcw84AixpNJvyK5QDJmey3kQGKmJlWHHKiHspzxlvQTtm6GiQh1BmJF RdhnF8P+FsnhmtNV0KTWYXo3lbg58rJt+x8y3f4IrFG+8CC2Gg84SPU9EvOqYMM+gEPn Q0hFHb358V+dfR1RnRXAXzSd8z8PHAImyuIthHVGtQppk9xZOZzHLAZ7CQKsLiT7VdMM zBn1VhXjP/45WisDelh/0s13uHv+Io6Fapot0ILs+cs34dVhrrmjy5Loid4E/cL6o0pj IYjBJPnH+z36V8WBqb0EfZbbZhaOPaJ6wN+7jb0gh31tYocSkxusFemzIMdcDvbJTc0n uB9w== X-Gm-Message-State: AJcUukciPcpjBsuLrdVzkpRaCA3yHHiP0XiENHijzIxQH5KBlxrnuS7f OtpcaEO7vIOrVC0j4aIFqUpt5GUz8Kg= X-Google-Smtp-Source: ALg8bN7hbrhgf5eDSiqQw66KaDSLNLz0lmL3UNygnP7rENiWfupKVZDt7OQvznLE/cUBVAJP9/xiQA== X-Received: by 2002:ac8:33fc:: with SMTP id d57mr8181946qtb.82.1548365400557; Thu, 24 Jan 2019 13:30:00 -0800 (PST) Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com. [209.85.160.177]) by smtp.gmail.com with ESMTPSA id s46sm75865689qtc.63.2019.01.24.13.29.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 Jan 2019 13:30:00 -0800 (PST) Received: by mail-qt1-f177.google.com with SMTP id l11so8518357qtp.0; Thu, 24 Jan 2019 13:29:59 -0800 (PST) X-Received: by 2002:a0c:c404:: with SMTP id r4mr7824550qvi.131.1548365399524; Thu, 24 Jan 2019 13:29:59 -0800 (PST) MIME-Version: 1.0 From: Mateusz Piotrowski <0mp@freebsd.org> Date: Thu, 24 Jan 2019 22:29:46 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Call for 2018Q4 quarterly status reports (deadline: 2019-01-31) To: FreeBSD Hackers , FreeBSD Current Cc: monthly@freebsd.org, Edward Tomasz Napierala X-Rspamd-Queue-Id: 53CB987E77 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of mpp302@gmail.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=mpp302@gmail.com X-Spamd-Result: default: False [-4.07 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.93)[-0.925,0]; RCVD_IN_DNSWL_NONE(0.00)[178.160.85.209.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[0mp@freebsd.org,mpp302@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[0mp@freebsd.org,mpp302@gmail.com]; IP_SCORE(-1.14)[ipnet: 209.85.128.0/17(-3.71), asn: 15169(-1.91), country: US(-0.08)] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2019 21:30:10 -0000 Dear FreeBSD Community, The deadline for the next FreeBSD Quarterly Status update is January 31, 2019, for work done since the last round of Quarterly Reports [1]. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and provide a great way to inform FreeBSD users and developers about work that is underway and completed. Submission of reports is not restricted to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The preferred method is to follow the guidelines at the Quarterly GitHub repository: https://github.com/freebsd/freebsd-quarterly Alternatively you can fetch the Markdown template, fill it in, and email it to quarterly@FreeBSD.org. The template can be found at: https://raw.githubusercontent.com/freebsd/freebsd-quarterly/master/report-sample.md The old XML generator and templates are no longer used. We look forward to seeing your 2018Q4 reports! Cheers, Mateusz (on behalf of quarterly@) [1]: https://www.freebsd.org/news/status/report-2018-01-2018-09.html