From owner-freebsd-stable@freebsd.org Sun Apr 8 14:19:12 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06DF3F84ED0 for ; Sun, 8 Apr 2018 14:19:12 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9BD16820EB for ; Sun, 8 Apr 2018 14:19:11 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (115-166-20-68.dyn.iinet.net.au [115.166.20.68]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id w38EJ7qp079540 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 8 Apr 2018 07:19:10 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: kern.sched.quantum: Creepy, sadistic scheduler To: freebsd-stable@freebsd.org References: <9FDC510B-49D0-4722-B695-6CD38CA20D4A@gmail.com> <8cfdb8a3-86a0-17ba-1e41-ff1912a30ee9@m5p.com> <2d3dfb30-fb34-55e4-0572-df17ceb5f66e@freebsd.org> From: Julian Elischer Message-ID: <88870e92-04a0-abab-3db2-4696dd968489@freebsd.org> Date: Sun, 8 Apr 2018 22:19:01 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Apr 2018 14:19:12 -0000 On 7/4/18 10:21 pm, Peter wrote: > Julian Elischer wrote: >> for a single CPU you really should compile a kernel with SMP turned >> off >> and 4BSD scheduler. >> >> ULE is just trying too hard to do stuff you don't need. > > Julian, > > if we agree on this, I am fine. > (This implies that SCHED_4BSD will *not* be retired for an > indefinite time!) There is no reason to retire it. We implemented a scheduler interface that both schedulers stick to. > > I tested yesterday, and SCHED_4BSD doesn't show the annoying behaviour. > SMP seems to be no problem (and I need that), but PREEMPTION is > definitely related to the problem (see my other message sent now). > > P. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to > "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Sun Apr 8 18:41:46 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 477E8F989D3 for ; Sun, 8 Apr 2018 18:41:46 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0C5E7592C for ; Sun, 8 Apr 2018 18:41:45 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:470:1f07:15ff::1f] (haymarket.m5p.com [IPv6:2001:470:1f07:15ff::1f]) (authenticated bits=0) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTPSA id w38IfFZK053936 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 8 Apr 2018 14:41:25 -0400 (EDT) (envelope-from george+freebsd@m5p.com) Subject: Re: more data: SCHED_ULE+PREEMPTION is the problem To: freebsd-stable@freebsd.org References: From: George Mitchell Openpgp: preference=signencrypt Autocrypt: addr=george+freebsd@m5p.com; prefer-encrypt=mutual; keydata= xsFNBFgnLnwBEADAJDiBKQX77LFRz9wZW8mz3KvaQol2nIremcws0F1mz/zgFlk6uhQVtwnL wb4XL5LdFwcNE1+QZzPLcbYWoWQlz0lBw1bMuKAgr0S6V2e0+I0DqhKeslVFctcTwtvT6pnK VLZXO/7ZGAaLzG4K5vSPzgoevU+YI/pxNsVCH2UO/c3jQW63uEt25mIZbCF1Pu4jgp4RhIgF ujn877r/j6OwBwjzRUu3E6ADp+U825d+5YCuQMEH0wIPnn9GTpXvfdKdbwOIl2akqXqs4cnk iATWfK3r6D4mvDEj1OPHlTvJYcfic7aOIiAwmx1C1v78GjXOdOOA0SGffNix3C2/8oZUO1+V Aet4MKpUKkduWSvULhIkHNZ5Nu8SIJOqge8pmtHxuNXAMfMrAjMdjPwwBFLsYg3Xa2E2oJwg ehTauwd/EDJFcVCyDCyCAYOi/BH/+XQyxzgDlY9N9qj9tHqhVPI6XK7t8UVffGiZUq4rHp5J RdOToqiTNC6eCJBczhMIW+DuFvWU9e6W708T1dz0Accn6Lrgk4eRIn3GFPBG+TxnpjAqHsbW 607dcnD3YKAqY4e+khczL4EObhe7dC1v2fmZiAC6Ds3WHR11IfqoUgCkIwJ590Ej+ElygJFF XxI82wtEz9hkeLLvItpyEJNVjppViRW+Dgl/U7ypHB3qDgYjgwARAQABzSBHZW9yZ2UgTWl0 Y2hlbGwgPGdlb3JnZUBtNXAuY29tPsLBlwQTAQgAQQIbIwUJCWYBgAULCQgHAgYVCAkKCwIE FgIDAQIeAQIXgBYhBDXTOGR5LbCVuZCmV8EREt5vqeH5BQJYWXFRAhkBAAoJEMEREt5vqeH5 SRsQAIb/crcxXyAyeokAuTjN+YXkEFdVv/JrOgNXKdCukXt/UGd3nZTcAzpllytDIvIlPTNI 2/nZ5sm6ymeyVwmvkrM/r3sRUib5ftakJpbv0wn2j+eCGAca8IA2frBUg9bEXKcHRJRQCztG cpousHzOziTCRQ/1NfPcIFBNbQMPUVoQ96cJPvM/XfFGOISKKsSI78skHm6Oazh1hLCQTKvy hNNgVpNP/PHCHMbla1+SNgyn35CUelTh153lnkOhw1XyX6IxY4o5Bhcf3YrxAVcoeHq3FEH0 T3ygAdI14VrZGXXitBAMI78QLB0FiWMoPQ3Oddnoh6tlT4djnsc9IW0Tzk6ozvQL7sKdYgO8 ZlIpkBqQQfpSHzwPu9EkOFggPWB9WtP3IajQ0lt1LovqMug4C8APRC2/1cvi+GUIRwjVsop0 ukhlLTTJd3/S4Muh3s87M4Rdek1xpOiMKjYOVaxmhQQ91oz971zJuJJWpX7uUQiXx9oEwCDW TvI5yEuqYLsMUwx3d2iFTr+HbtlBJmF+Vguyrn/a3vFK8P/TH9fMvNeTdln9SuOOa1SAMMcy czOpBYg9RpzNLshUJVrhKzugT59Rl2wsNQsQCUkzgF3f8cZHJyl+8x6t0nuM9LTkMv13YIXS Mde3UOD2EaBhmeIqvC/adQHxpNudvxM1viFJDnTnzsFNBFgnLnwBEAC7kzsZqjBRPonnr/63 C98FSa3LikvqQWygmPSCC9DsFX/fB0CSXIHTrHQ4a7lXdfZyTZcGdxXN+MC8O8thjvVq6WYm CpyOJ/bq4SxOa9cnQSJ5SP9VCmVoDN/3T4ybXFzLAt756kfQ5jsVuP0m6iQ4z918zhZXk+Mo qdwGjYTxsBD02a7m1aeYafyaI2mZ+vdEy0cDhV4PDXI6ThLNAavTPji6ZrBdH5a4LMg30u8v kkNe0eCKvU+0cWb2VQIeddMhhiGSBE2Dv+A4eNe1VZvoGpLDlYdnoHraVFL2GHNFGymj/uRf 1hja5kW9Rekisqby5SpGABwrFFEs7ABpfYG0IRBKbjjG1Cqdfe/R6ETJFvvNOOpCKPAWeqYf Isxv4OHWTmhKihhIanYWCAe1MDLRRbj7UrOTZeia2WJ4v58xbU5rVQoHI/Tzaq86rXzRWITc 5w+kresMad0zpvQ900BdHc8ATNY2aW/Fr5it5OqMvIW6Y4gc8Z95MkQPVc8vj3WzfxuWtZNK 6Wbv4r6Qbiwg1RpY1JoEIoF4OsZJOVMQaB6ezD7xvaa2eY6nRGtq9SoJxo2qvlSbLlKq8NdH VYHhtTbpQ1NfrEJfv5sLX5W5IpoYww48bFYH67+7r1f/W3voBptSgE7qnYAm6Em9bEAKOQEX 8BXwoa54fn03z6TyhwARAQABwsFkBBgBCAAPBQJYJy58AhsMBQkJZgGAAAoJEMEREt5vqeH5 6PcP+JvrMM7ZM8UlnbrY4Er9psPj3ayllRhQFA9h6GNUKYuSzSxOrPaT96s8KUGMCr4jrn1S WFmeeNLLOgSJmQRicMh6LmnKq6WY6UaOfn7Y9O62NUjXfEI3Bw4ID36YCdQ+CJd14r0YOf4M 5F50bvHV3lbzD9TXZPxecHKC2ZUMBbT37tsckWCLL3lzKMsqQLwBUmgBl1NIUc7gyXxiNyxb 6SPVF1NguDDM438mcg9jSRAyjgAk6POUEM8YIXkw0Gg6HF1tNMJJ1xTMBCLYl6fHTtsxJpf9 yo+Hnw346hqYzXn4ytHJ49Ngcre8uhqM1l8iMpa17tEjfalkc1FWR9/qvoowOKtxpvblsy3a YzeEFgIomhLzISz6IafQ3S7Mt5AFlqwN/qQHx0k2V66GzDG0ngZBPROP1sXSpdJzO0zbJQFn MZE3f+y8vXMcE/MBXR7kAdYYApiEMQzVxy9TdQDU3lGLptcPZ1IOntTNFFrvp5NwsKi+6C9i mXtd5kJ1PwhcJYW3/ov/490l60C5SFUL/RZ/NOW8SHFaPcqlGcqIlexFKbzrMQwmYXo95jWB eZ0Qn+raxCUFZNGiwtusyQGBMcpHVJUanOCNd1z4ZbfmhUjDJKC/7YWDunvaDRSukGiRCl6J s8caqXHiVZjx+s76iWzm6AHRP5jg9D6EtTOrGiE= Message-ID: <3323e648-65a9-6270-3b0f-59064e4203de@m5p.com> Date: Sun, 8 Apr 2018 14:41:09 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="kW0WoDviQ3YGLXY5AaB2ZSmzxu30KmbcX" X-Spam-Status: No, score=0.2 required=10.0 tests=HELO_MISC_IP, RP_MATCHES_RCVD autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mattapan.m5p.com X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mailhost.m5p.com [IPv6:2001:470:1f07:15ff::f7]); Sun, 08 Apr 2018 14:41:26 -0400 (EDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Apr 2018 18:41:46 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --kW0WoDviQ3YGLXY5AaB2ZSmzxu30KmbcX Content-Type: multipart/mixed; boundary="krxBEe0HaWVxSHcE02CbHjZa7hRDjJr8G"; protected-headers="v1" From: George Mitchell To: freebsd-stable@freebsd.org Message-ID: <3323e648-65a9-6270-3b0f-59064e4203de@m5p.com> Subject: Re: more data: SCHED_ULE+PREEMPTION is the problem References: In-Reply-To: --krxBEe0HaWVxSHcE02CbHjZa7hRDjJr8G Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 04/07/18 10:18, Peter wrote: > [...] > B. Findings: > ------------ > 1. Filesystem >=20 > I could never reproduce this when reading from plain UFS. Only when > reading from ZFS (direct or via l2arc). > [...] My consistent way of reproducing the problem was to run ports/misc/dnetc while trying to buildworld/buildkernel. But I was always building on a UFS partition. I don't know why your results would be different. -- George --krxBEe0HaWVxSHcE02CbHjZa7hRDjJr8G-- --kW0WoDviQ3YGLXY5AaB2ZSmzxu30KmbcX Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAlrKYksACgkQwRES3m+p 4fnfXg/+OFaYPZTEYieTUik19UoS5vYa6MJvfKtS7GbtH8ziwcX8hIKMgiQ/6ftB 8ACinBN7KFUFh2frSJAA54QPLqKwCmUAgyoQ+LqGoQl9lqPlOPTJp88k8S3YbU7W mgAuPz/86RHtA4mahkvKleTTIX6o9rSm5pW9RrvCNXjy28ACKYRFvMz7bIbIYaxn c3RUmRaUyfXhIhbtsMjw/XtBmuShxNhM9q19NLogtHb2pXl/KUbUQ1r0kkgw1oCq MlMzL/85Ul7bvGs1p+90kWssx6RgEKhPr1P9vkMCHKK9xnqoS/jzpEbe4uTnJb0J S0o4jqnmSJJVEgvWTAx888muAniNExTeS6mDZoF+f1UcDZqyjMpFN+0HQ7FP9J3a o534CLFF9rINylXUAAqTN1Qy4NF1lGIcj9c4CyL7Llx3ya4++OTpraXCuxjsObWt qoEsx18GpoMk5/bqoKqua5cqnFscbSrXEqNuBhL4uEgikQwj23Ay4IwXHdp0gMyJ JVF/9XLmg1rjmxYK3LMxXiwn8s9Bl8cv03ZtbDn3kOUqK1VV8OpgznzS8GWYrVlQ vBl8+gx1rSHaBIPwNheZmDyghv9BgfwMll5kQHjnbbZ6O4n1vf+5+FTflrJflanI k/gvi2fD6mJvPV1UQlYa5Wm6BSgus3WwT/nL6X+MsHplTqxy290= =NPwZ -----END PGP SIGNATURE----- --kW0WoDviQ3YGLXY5AaB2ZSmzxu30KmbcX-- From owner-freebsd-stable@freebsd.org Mon Apr 9 10:07:52 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B75AF95C55 for ; Mon, 9 Apr 2018 10:07:52 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailout07.t-online.de (mailout07.t-online.de [194.25.134.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 958196C035 for ; Mon, 9 Apr 2018 10:07:51 +0000 (UTC) (envelope-from se@freebsd.org) Received: from fwd06.aul.t-online.de (fwd06.aul.t-online.de [172.20.26.150]) by mailout07.t-online.de (Postfix) with SMTP id 3FE33420CFFE; Mon, 9 Apr 2018 12:07:49 +0200 (CEST) Received: from Stefans-MBP-7.fritz.box (bHeiVQZc8hgkglkBRjzC9xU0QHIZZH4ijVCQNKQItTi0cyeWFcDdPUONjj+4Z3-Z+Q@[84.154.99.226]) by fwd06.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1f5TiF-1lhPfs0; Mon, 9 Apr 2018 12:07:47 +0200 Subject: Re: more data: SCHED_ULE+PREEMPTION is the problem To: freebsd-stable@freebsd.org References: From: Stefan Esser Openpgp: preference=signencrypt Autocrypt: addr=se@freebsd.org; prefer-encrypt=mutual; keydata= xsBNBFVxiRIBCADOLNOZBsqlplHUQ3tG782FNtVT33rQli9EjNt2fhFERHIo4NxHlWBpHLnU b0s4L/eItx7au0i7Gegv01A9LUMwOnAc9EFAm4EW3Wmoa6MYrcP7xDClohg/Y69f7SNpEs3x YATBy+L6NzWZbJjZXD4vqPgZSDuMcLU7BEdJf0f+6h1BJPnGuwHpsSdnnMrZeIM8xQ8PPUVQ L0GZkVojHgNUngJH6e21qDrud0BkdiBcij0M3TCP4GQrJ/YMdurfc8mhueLpwGR2U1W8TYB7 4UY+NLw0McThOCLCxXflIeF/Y7jSB0zxzvb/H3LWkodUTkV57yX9IbUAGA5RKRg9zsUtABEB AAHNLlN0ZWZhbiBFw59lciAoVC1PbmxpbmUpIDxzdC5lc3NlckB0LW9ubGluZS5kZT7CwH8E EwEIACkFAlhtTvQCGwMFCQWjmoAHCwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRBH67Xv Wv31RAn0B/9skuajrZxjtCiaOFeJw9l8qEOSNF6PKMN2i/wosqNK57yRQ9AS18x4+mJKXQtc mwyejjQTO9wasBcniKMYyUiie3p7iGuFR4kSqi4xG7dXKjMkYvArWH5DxeWBrVf94yPDexEV FnEG9t1sIXjL17iFR8ng5Kkya5yGWWmikmPdtZChj9OUq4NKHKR7/HGM2dxP3I7BheOwY9PF 4mhqVN2Hu1ZpbzzJo68N8GGBmpQNmahnTsLQ97lsirbnPWyMviWcbzfBCocI9IlepwTCqzlN FMctBpLYjpgBwHZVGXKucU+eQ/FAm+6NWatcs7fpGr7dN99S8gVxnCFX1Lzp/T1YzsBNBFVx iRIBCACxI/aglzGVbnI6XHd0MTP05VK/fJub4hHdc+LQpz1MkVnCAhFbY9oecTB/togdKtfi loavjbFrb0nJhJnx57K+3SdSuu+znaQ4SlWiZOtXnkbpRWNUeMm+gtTDMSvloGAfr76RtFHs kdDOLgXsHD70bKuMhlBxUCrSwGzHaD00q8iQPhJZ5itb3WPqz3B4IjiDAWTO2obD1wtAvSuH uUj/XJRsiKDKW3x13cfavkad81bZW4cpNwUv8XHLv/vaZPSAly+hkY7NrDZydMMXVNQ7AJQu fWuTJ0q7sImRcEZ5EIa98esJPey4O7C0vY405wjeyxpVZkpqThDMurqtQFn1ABEBAAHCwGUE GAEKAA8FAlVxiRICGwwFCQWjmoAACgkQR+u171r99UQEHAf/ZxNbMxwX1v/hXc2ytE6yCAil piZzOffT1VtS3ET66iQRe5VVKL1RXHoIkDRXP7ihm3WF7ZKy9yA9BafMmFxsbXR3+2f+oND6 nRFqQHpiVB/QsVFiRssXeJ2f0WuPYqhpJMFpKTTW/wUWhsDbytFAKXLLfesKdUlpcrwpPnJo KqtVbWAtQ2/o3y+icYOUYzUig+CHl/0pEPr7cUhdDWqZfVdRGVIk6oy00zNYYUmlkkVoU7MB V5D7ZwcBPtjs254P3ecG42szSiEo2cvY9vnMTCIL37tX0M5fE/rHub/uKfG2+JdYSlPJUlva RS1+ODuLoy1pzRd907hl8a7eaVLQWA== Cc: Jeff Roberson Message-ID: <07279919-3b8f-3415-559f-6e7e66cb51c9@freebsd.org> Date: Mon, 9 Apr 2018 12:07:46 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 8bit X-ID: bHeiVQZc8hgkglkBRjzC9xU0QHIZZH4ijVCQNKQItTi0cyeWFcDdPUONjj+4Z3-Z+Q X-TOI-MSGID: 844b33f7-960a-488b-a313-5a452df14973 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Apr 2018 10:07:52 -0000 Am 07.04.18 um 16:18 schrieb Peter: > 3. kern.sched.preempt_thresh > > I could make the problem disappear by changing kern.sched.preempt_thresh  from > the default 80 to either 11 (i5-3570T) or 7 (p3) or smaller. This seems to > correspond to the disk interrupt threads, which run at intr:12 (i5-3570T) or > intr:8 (p3). [CC added to include Jeff as the author of the ULE scheduler ...] Since I had somewhat similar problems on my systems (with 4 Quad-Core with SMT enabled, i.e. 8 threads of execution) with compute bound processes keeping I/O intensive processes from running (load average of 12 with 8 "CPUs"), and these problems where affected by preempt_thresh, I checked how this variable is used in the scheduler. The code is in /sys/kern/sched_ule.c. It controls, whether a thread that has become runnable (e.g., after waiting for disk I/O to complete) will preempt the thread currently running on "this" CPU (i.e. the one executing this test in the kernel). IMHO, sched_preempt should default to a much higher number than 80 (e.g. 190), but maybe I misunderstand some of the details ... static inline int sched_shouldpreempt(int pri, int cpri, int remote) { The parameters are: pri: the priority if the now runnable thread cpri: the priority of the thread that currently runs on "this" CPU remote: whether to consider preempting a thread on another CPU The priority values are those displayed by top or ps -l as "PRI", but with an offset of 100 applied (i.e. pri=120 is displayed as PRI=20 in top). If this thread has less priority than the currently executing one (cpri), the currently running thread will not be preempted: /* * If the new priority is not better than the current priority there is * nothing to do. */ if (pri >= cpri) return (0); If the current thread is the idle thread, it will always be preempted by the now runnable thread: /* * Always preempt idle. */ if (cpri >= PRI_MIN_IDLE) return (1); A value of preempt_thresh=0 (e.g. if "options PREEMPTION" is missing in the kernel config) lets the previously running thread continue (except if was the idle thread, which has been dealt with above). The compute bound thread may continue until its quantum has expired. /* * If preemption is disabled don't preempt others. */ if (preempt_thresh == 0) return (0); For any other value of preempt_thresh, the new priority of the thread that just has become runnable will be compared to preempt_thresh and if this new priority is higher (lower numeric value) or equal to preempt_thresh, the thread for which (e.g.) disk I/O finished will preempt the current thread: /* * Preempt if we exceed the threshold. */ if (pri <= preempt_thresh) return (1); ===> This is the only condition that depends on preempt_thresh > 0 <=== The flag "remote" controls whether this thread will be scheduled to run, if its priority is higher or equal to PRI_MAX_INTERACT (less than or equal to 151) and if the opposite is true for the currently running thread (cpri). The value of remote will always be 0 on kernels built without "options SMP". /* * If we're interactive or better and there is non-interactive * or worse running preempt only remote processors. */ if (remote && pri <= PRI_MAX_INTERACT && cpri > PRI_MAX_INTERACT) return (1); The critical use of preempt_thresh is marked above. If it is 0, no preemption will occur. On a single processor system, this should allow the CPU bound thread to run for as long its quantum lasts. A value of 120 (corresponding to PRI=20 in top) will allow the I/O bound thread to preempt any other thread with lower priority (cpri > pri). But in case of a high priority kernel thread being active during this test (with a low numeric cpri value), the I/O bound process will not preempt that higher priority thread (i.e. some high priority kernel thread). Whether the I/O bound thread will run (instead of the compute bound) after the higher priority thread has given up the CPU, will depend on the scheduler decision which thread to select. And for "timeshare" threads, this will often not be the higher priority (I/O bound) thread, but the compute bound thread, which then may execute until next being interrupted by the I/O bound thread (which will not happen, if no new I/O has been requested). This might explain, why setting preempt_thresh to a very low value (in the range of real-time kernel threads) enforces preemption of the CPU bound thread, while any higher (numeric) value of preempt_thresh prevents this and makes tdq_choose() often select the low priority CPU bound over the higher priority I/O bound thread. BUT the first test in sched_shouldpreempt() should prevent any user process from ever preempting a real-time thread "if (pri >= cpri) return 0;". For preemption to occur, pri must be numerically lower than cpri, and pri must be numerically lower than or equal to preempt_thresh. > a. with kern.sched.preempt_thresh=80 > > $ lz4 DATABASE_TEST_FILE /dev/null & while true; >   do ps -o pid,pri,"%cpu",command -p 2119,$! >   sleep 3 > done > [1] 6073 [...] >  PID PRI %CPU COMMAND > 6073  52  6.5 lz4 DATABASE_TEST_FILE /dev/null > 2119  99 91.5 -bash (bash) The I/O bound thread does not preempt the compute bound thread, when becoming runnable (data arrived from disk). With the value of preempt_thresh=80 (corresponding to PRI=-20) only real-time threads may cause preemption, the I/O bound thread can not (PRI=52 / pri=152). A value of preempt_thresh in the range of 190 (corresponding to PRI=90) should allow the lz4 process to preempt the CPU bound process (with higher pri/PRI). > b. with kern.sched.preempt_thresh=11 > >  PID PRI %CPU COMMAND > 4920  21  0.0 lz4 DATABASE_TEST_FILE /dev/null > 2119 101 93.5 -bash (bash) [...] >  PID PRI %CPU COMMAND > 4920  85 43.0 lz4 DATABASE_TEST_FILE /dev/null > 2119  85 45.5 -bash (bash) Such a low preempt_thresh does not allow any user process to preempt any other one (except when running with temporarily increased priority in the kernel). Only a kernel thread (soft interrupt?) at might cause preemption, and if the interrupt is due to a read issued by the I/O bound thread completing, then the I/O bound process is not the one being preempted. This will make the timeshare scheduler select the process with higher priority (lower PRI) that did not recently run (i.e. the I/O bound process, if both have the same PRI), when the kernel thread goes to sleep. But (if my analysis is correct) this indicates, that preempt_thresh set to an extremely low value just helps by accident. The kernel thread interrupts the CPU bound thread, and the I/O bound thread is selected as the next runnable thread in the time-share run queue, either because of its lower PRI value or because it did not run last before the preemption occurred (with equal PRI for both). But, in fact, the same scheduler selection should have occured in test (a), too, if e.g. a soft interrupt preempts the compute bound thread. Not sure, why this does not happen ... (And this may be an indication, that I do not fully understand what's going on ;-) ...) > From this we can see that in case b. both processes balance out nicely and > meet at equal CPU shares. > Whereas in case a., after about 10 Seconds (the first 3 records) they move to > opposite ends of the scale and stay there. > > From this I might suppose that here is some kind of mis-calculation or > mis-adjustment of the task priorities happening. I'd be interested in your results with preempt_thresh set to a value of e.g. 190. The PRI=85 values in your test case (b) correspond to pri=185, and with preempt_thresh slightly higher that that, the lz4 process should still get a 50% share of the CPU. (If its PRI grows over that of the CPU bound process, it will not be able to preempt it, so its PRI should match the one of the CPU bound process). Regards, STefan From owner-freebsd-stable@freebsd.org Mon Apr 9 19:27:47 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55B21F9C7C6 for ; Mon, 9 Apr 2018 19:27:47 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id ECA9D6A065 for ; Mon, 9 Apr 2018 19:27:46 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w39JRktK008285 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 9 Apr 2018 15:27:46 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w39JRiQo055842 for ; Mon, 9 Apr 2018 15:27:44 -0400 (EDT) (envelope-from mike@sentex.net) To: FreeBSD-STABLE Mailing List From: Mike Tancsa Subject: devd out of swap space ? (zfs arc related ?) Openpgp: preference=signencrypt Organization: Sentex Communications Message-ID: Date: Mon, 9 Apr 2018 15:27:45 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Apr 2018 19:27:47 -0000 On one of my internal nfs test boxes, I have noticed that kernels from March 28th and April 6th r332100 ended up with devd running out of swap space and being killed at some point. First time I thought perhaps a fluke due to some stress testing I was doing. But I left the box running over the weekend and Sat AM it died at 3:51am (perhaps when periodic runs?) I have lots of free memory, however, ARC is chewing up 27G of the 32G. CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 800K Active, 15M Inact, 9964K Laundry, 30G Wired, 516M Free ARC: 28G Total, 4996M MFU, 23G MRU, 5440K Anon, 78M Header, 593M Other 27G Compressed, 28G Uncompressed, 1.04:1 Ratio Swap: 20G Total, 21M Used, 20G Free Anyone else seen anything like this on a recent RELENG11 STABLE ? ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada From owner-freebsd-stable@freebsd.org Tue Apr 10 05:49:03 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7253FA321E for ; Tue, 10 Apr 2018 05:49:02 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from umail2.aei.mpg.de (umail2.aei.mpg.de [194.94.224.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A95783473 for ; Tue, 10 Apr 2018 05:49:01 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail2.aei.mpg.de (Postfix) with ESMTP id 6E9191BABAC5 for ; Tue, 10 Apr 2018 07:48:54 +0200 (CEST) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 52862406ADE for ; Tue, 10 Apr 2018 07:48:54 +0200 (CEST) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id 3067A406ADB for ; Tue, 10 Apr 2018 07:48:54 +0200 (CEST) Received: from arc.aei.uni-hannover.de ([130.75.117.1]) by intranet.aei.uni-hannover.de (IBM Domino Release 9.0.1FP8) with ESMTP id 2018041007485333-327682 ; Tue, 10 Apr 2018 07:48:53 +0200 Date: Tue, 10 Apr 2018 07:48:53 +0200 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: freebsd-stable@freebsd.org Subject: Re: devd out of swap space ? (zfs arc related ?) Message-Id: <20180410074853.e471483ad792eb9e400d06dc@aei.mpg.de> In-Reply-To: References: Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.6.0 (GTK+ 2.24.31; amd64-portbld-freebsd11.1) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 9.0.1FP8|February 23, 2017) at 10/04/2018 07:48:53, Serialize by Router on intranet/aei-hannover(Release 9.0.1FP8|February 23, 2017) at 10/04/2018 07:48:53, Serialize complete at 10/04/2018 07:48:53 X-TNEFEvaluated: 1 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2018.4.10.53915 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1000_LESS 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, BODY_SIZE_800_899 0, IN_REP_TO 0, LEGITIMATE_SIGNS 0, MSG_THREAD 0, NO_URI_HTTPS 0, REFERENCES 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __FRAUD_MONEY_BIG_COIN 0, __FRAUD_MONEY_BIG_COIN_DIG 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_TEXT_P 0, __MIME_TEXT_P1 0, __MIME_VERSION 0, __NO_HTML_TAG_RAW 0, __REFERENCES 0, __SANE_MSGID 0, __STOCK_PHRASE_7 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __TO_NO_NAME 0, __URI_NO_WWW 0, __URI_NS ' X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2018 05:49:03 -0000 On Mon, 9 Apr 2018 15:27:45 -0400 Mike Tancsa wrote about devd out of swap space ? (zfs arc related ?): > Anyone else seen anything like this on a recent RELENG11 STABLE ? I think I have seen something similar last week with a -stable from somewhen in March. Lots of processes crashed over night due to "out of swap space" although there appeared to be plenty of both swap and RAM. Somehow it looked arc-related to me, but I havn't been able to reproduce it so far (however, I did not try too hard, either ;-). This is what top shows on the machine right now: CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 2776K Active, 7231M Inact, 31M Laundry, 24G Wired, 934M Buf, 288M Free ARC: 20G Total, 6678M MFU, 13G MRU, 1060K Anon, 182M Header, 146M Other 19G Compressed, 82G Uncompressed, 4.24:1 Ratio cu Gerrit From owner-freebsd-stable@freebsd.org Tue Apr 10 11:41:20 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BEEAF94F5D for ; Tue, 10 Apr 2018 11:41:20 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E1F770A03 for ; Tue, 10 Apr 2018 11:41:19 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.15.2) with ESMTPS id w3ABfDgo090913 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 10 Apr 2018 13:41:14 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.15.2/Submit) with UUCP id w3ABfDFJ090912 for freebsd-stable@FreeBSD.ORG; Tue, 10 Apr 2018 13:41:13 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.15.2/8.15.2) with ESMTP id w3ABZWE5050561 for ; Tue, 10 Apr 2018 13:35:32 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.15.2/8.15.2) with ESMTP id w3ABY4Gd050356 for ; Tue, 10 Apr 2018 13:34:04 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.15.2/8.15.2/Submit) id w3ABY4Wv050354 for freebsd-stable@FreeBSD.ORG; Tue, 10 Apr 2018 13:34:04 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: Appendices - more data: SCHED_ULE+PREEMPTION is the problem Date: Tue, 10 Apr 2018 13:26:33 +0200 Organization: even some more stinky socks Message-ID: <389733b6-0fec-78f1-8969-83bd95cd9fdd@citylink.dinoex.sub.org> References: <07279919-3b8f-3415-559f-6e7e66cb51c9@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: oper.dinoex.de; logging-data="49188"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-Reply-To: Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (uucp.dinoex.sub.de [194.45.71.2]); Tue, 10 Apr 2018 13:41:15 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2018 11:41:20 -0000 I forgot to attach the commands used to create the logs - they are ugly anyway: [1] dtrace -q -n '::sched_choose:return { @[((struct thread *)arg1)->td_proc->p_pid, stringof(((struct thread *)arg1)->td_proc->p_comm), timestamp] = count(); } tick-1s { exit(0); }' | sort -nk 3 | awk '$1 > 27 {$3 = ($3/1000000)*1.0/1000; printf "%6d %20s %3.3f\n", $1, $2, $3 }' [2] dtrace -q -n '::runq_choose_from:entry /arg1 == 0||arg1 == 32/ { @[arg1, timestamp] = count(); }' | sort -nk2 From owner-freebsd-stable@freebsd.org Tue Apr 10 11:41:48 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE43BF94FC8 for ; Tue, 10 Apr 2018 11:41:47 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CE8470E48 for ; Tue, 10 Apr 2018 11:41:47 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.15.2) with ESMTPS id w3ABfF6p090927 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 10 Apr 2018 13:41:16 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.15.2/Submit) with UUCP id w3ABfFm9090926 for freebsd-stable@FreeBSD.ORG; Tue, 10 Apr 2018 13:41:15 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.15.2/8.15.2) with ESMTP id w3ABZWE7050561 for ; Tue, 10 Apr 2018 13:35:32 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.15.2/8.15.2) with ESMTP id w3ABY3tC050347 for ; Tue, 10 Apr 2018 13:34:03 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.15.2/8.15.2/Submit) id w3ABY3Ia050342 for freebsd-stable@FreeBSD.ORG; Tue, 10 Apr 2018 13:34:03 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: Re: more data: SCHED_ULE+PREEMPTION is the problem Date: Tue, 10 Apr 2018 13:22:07 +0200 Organization: even some more stinky socks Message-ID: References: <07279919-3b8f-3415-559f-6e7e66cb51c9@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: oper.dinoex.de; logging-data="48596"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-Reply-To: <07279919-3b8f-3415-559f-6e7e66cb51c9@freebsd.org> Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (uucp.dinoex.sub.de [194.45.71.2]); Tue, 10 Apr 2018 13:41:17 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2018 11:41:48 -0000 Hi Stefan, I'm glad to see You're thinking along similar paths as I did. But let me fist answer Your question straight away, and sort out the remainder afterwards. > I'd be interested in your results with preempt_thresh set to a value > of e.g.190. There is no difference. Any value above 7 shows the problem identically. I think this value (or preemtion as a whole) is not the actual cause for the problem; it just changes some conditions that make the problem visible. So, trying to adjust preempt_thresh in order to fix the problem seems to be a dead end. Stefan Esser wrote: > The critical use of preempt_thresh is marked above. If it is 0, no preemption > will occur. On a single processor system, this should allow the CPU bound > thread to run for as long its quantum lasts. I would like to contradict here. From what I understand, preemption is *not* the base of task switching. AFAIK preemption is an additional feature that allows to switch threads while they execute in kernel mode. While executing in user mode, a thread can be interrupted and switched at any time, and that is how the traditional time-sharing systems did it. Traditionally a thread would execute in kernel mode only during interrupts and syscalls, and those last no longer than a few ms, and for long that was not an issue. Only when we got the fast interfaces (10Gbps etc.) and got big monsters executing in kernel space (traffic-shaper, ZFS, etc.), that scheme became problematic and preemption was invented. According to McKusicks book, the scheduler is two-fold: an outer logic runs few times per second and calculates priorities. And an inner logic runs very often (at every interrupt?) and chooses the next runnable thread simply by priority. The meaning of the quantum is then: when it is used up, the thread is moved to the end of it's queue, so that it may take a while until it runs again. This is for implementing round-robin behaviour within a single queue (= a single priority). It should not prevent task-switching as such. Lets have a look. sched_choose() seems to be that low-level scheduler function that decides which thread to run next. Lets create a log of its decisions.[1] With preempt_thresh >= 12 (kernel threads left out): PID COMMAND TIMESTAMP 18196 bash 1192.549 18196 bash 1192.554 18196 bash 1192.559 66683 lz4 1192.560 18196 bash 1192.560 18196 bash 1192.562 18196 bash 1192.563 18196 bash 1192.564 79496 ntpd 1192.569 18196 bash 1192.569 18196 bash 1192.574 18196 bash 1192.579 18196 bash 1192.584 18196 bash 1192.588 18196 bash 1192.589 18196 bash 1192.594 18196 bash 1192.599 18196 bash 1192.604 18196 bash 1192.609 18196 bash 1192.613 18196 bash 1192.614 18196 bash 1192.619 18196 bash 1192.624 18196 bash 1192.629 18196 bash 1192.634 18196 bash 1192.638 18196 bash 1192.639 18196 bash 1192.644 18196 bash 1192.649 18196 bash 1192.654 66683 lz4 1192.654 18196 bash 1192.655 18196 bash 1192.655 18196 bash 1192.659 The worker is indeed called only after 95ms. And with preempt_thresh < 8: PID COMMAND TIMESTAMP 18196 bash 1268.955 66683 lz4 1268.956 18196 bash 1268.956 66683 lz4 1268.956 18196 bash 1268.957 66683 lz4 1268.957 18196 bash 1268.957 66683 lz4 1268.958 18196 bash 1268.958 66683 lz4 1268.959 18196 bash 1268.959 66683 lz4 1268.959 18196 bash 1268.960 66683 lz4 1268.960 18196 bash 1268.961 66683 lz4 1268.961 18196 bash 1268.961 66683 lz4 1268.962 18196 bash 1268.962 Here we have 3 Csw per millisecond. (The fact that the decisions are over-all more frequent is easily explained: when lz4 gets to run, it will do disk I/O, which quickly returns and triggers new decisions.) In the second record, things are clear: while lz4 does disk I/O, the scheduler MUST run bash, because nothing else is there. But when data arrives, it runs again lz4. But in the first record - why does the scheduler choose bash, although lz4 has already much higher prio (52 versus 97, usually)? > A value of 120 (corresponding to PRI=20 in top) will allow the I/O bound > thread to preempt any other thread with lower priority (cpri > pri). But in > case of a high priority kernel thread being active during this test (with a > low numeric cpri value), the I/O bound process will not preempt that higher > priority thread (i.e. some high priority kernel thread). > > Whether the I/O bound thread will run (instead of the compute bound) after > the higher priority thread has given up the CPU, will depend on the scheduler > decision which thread to select. And for "timeshare" threads, this will often > not be the higher priority (I/O bound) thread, but the compute bound thread, > which then may execute until next being interrupted by the I/O bound thread > (which will not happen, if no new I/O has been requested). Exactly. But why does this happen? Following the trail onwards from sched_choose() we find tdq_choose(), and that is quite straightforward looking for something runnable. The realtime and idle queues are simple and should grab the thread with highest priority, but the timeshare queue (which is the interesting piece here) has a special gimmick. It calls kern_sched.c:runq_choose_from() with an extra parameter tdq_ridx ("Current removal index"). From the design papers I get the clue that the heads of the timeshare runqueues are arranged in some kind of circular buffer - this is a specialty of SCHED_ULE; SCHED_4BSD does not have this feature, and does not call runq_choose_from(). And the tdq_ridx is the current index onto that circle. This mechanism (which I do not really understand currently) is the AFAIK only "element of distortion", which may cause that *not* the highest prio thread is selected to run. > For preemption to occur, pri must be numerically lower than cpri, and > pri must be numerically lower than or equal to preempt_thresh. Maybe. But as I wrote above, preemption is only concerning threads running in kernel mode. And what I want, is simply a normal task switching between the two processes running in user mode. > But, in fact, the same scheduler selection should have occured in test (a), > too, if e.g. a soft interrupt preempts the compute bound thread. Not sure, > why this does not happen ... (And this may be an indication, that I do not > fully understand what's going on ;-) ...) I think You do understand it - or we both don't understand it ;) - and there is still something else going on here which we do not yet see. > The PRI=85 values in your test case (b) correspond to pri=185, and with > preempt_thresh slightly higher that that, the lz4 process should still get a > 50% share of the CPU. (If its PRI grows over that of the CPU bound process, > it will not be able to preempt it, so its PRI should match the one of the CPU > bound process). As mentioned above, this does not work in practice. What I did instead was take a look at this circular index tdq_ridx. And what I found there is very strange: That index indeed moves from 0 to 63 (we have only one runq every 4 priorities) and repeats, and it takes 1/2 second to get around. Except when the problem appears - then it suddenly takes more than six seconds to get around! tdq_ridx is defined within sched_ule.c, and it is modified only here: > * Advance the insert index once for each tick to ensure that all > * threads get a chance to run. > */ > if (tdq->tdq_idx == tdq->tdq_ridx) { > tdq->tdq_idx = (tdq->tdq_idx + 1) % RQ_NQS; > if (TAILQ_EMPTY(&tdq->tdq_timeshare.rq_queues[tdq->tdq_ridx])) > tdq->tdq_ridx = tdq->tdq_idx; The comment suggests that tdq_ridx should increment with timer ticks, i.e. with constant speed! And, if the clock is stathz, that does figure: 64 / stathz = 0.5 seconds Here are the logs [2] showing that indeed the speed changes greatly, but only in the situation when the problem appears: preempt_thresh = 0: tdq_ridx timestamp 0 32.948718044 0 33.453705474 0 33.998990897 0 34.503690368 0 35.003946192 0 35.508674863 0 36.016583301 0 36.518663899 preempt_thresh = 80: 0 69.056754055 0 69.561710522 0 70.066654974 0 70.571848226 0 71.076660214 0 71.576621653 0 72.081640256 0 72.586606139 piglet running: (and adding half-round 32 for proof) 0 32.100767726 32 32.369727157 0 32.619581572 32 32.965831183 0 33.220777200 32 33.480751369 0 33.730745086 32 33.980736953 0 34.235749995 32 34.494315832 0 34.744162342 32 35.050767367 0 35.315405397 32 35.560715408 0 35.820902945 32 36.068976384 piglet and worker running: now clock is slow! 32 67.164163127 0 70.368819114 32 73.581467022 0 76.817670952 32 77.195794807 0 77.465528087 32 77.778554940 0 78.644335854 32 82.045835390 0 85.258399007 32 88.478827844 0 91.746418960 32 95.226730336 preempt_thresh back to 0: 32 47.988086885 0 48.255774908 32 48.507861276 0 48.765327585 32 49.011779231 0 49.268056274 32 49.515563800 0 49.815179616 32 50.067504569 0 50.322943146 32 50.572696557 0 50.823302445 32 51.074755232 I do currently not see how (or if) this would lead to the suboptimal selection of a runnable thread - but it might indeed do so. And, if there is really a problem with the clock, this may also have further effects. But then I have no clue what to make of this, or where to track it further. It might be that there is no problem with the scheduler, but instead the happening preemption does something bad with the clock. Or whatever... P. From owner-freebsd-stable@freebsd.org Tue Apr 10 19:13:22 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BCCFF913F3 for ; Tue, 10 Apr 2018 19:13:22 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B0C9382C5B for ; Tue, 10 Apr 2018 19:13:21 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.15.2) with ESMTPS id w3AJD6AQ042855 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 10 Apr 2018 21:13:06 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.15.2/Submit) with UUCP id w3AJD6EX042854 for freebsd-stable@FreeBSD.ORG; Tue, 10 Apr 2018 21:13:06 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.15.2/8.15.2) with ESMTP id w3AIiXLY004690 for ; Tue, 10 Apr 2018 20:44:33 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.15.2/8.15.2) with ESMTP id w3AIi32O004601 for ; Tue, 10 Apr 2018 20:44:03 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.15.2/8.15.2/Submit) id w3AIi3E2004595 for freebsd-stable@FreeBSD.ORG; Tue, 10 Apr 2018 20:44:03 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: Found the issue! - SCHED_ULE+PREEMPTION is the problem Date: Tue, 10 Apr 2018 20:30:35 +0200 Organization: even some more stinky socks Message-ID: References: <07279919-3b8f-3415-559f-6e7e66cb51c9@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: oper.dinoex.de; logging-data="2839"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 X-Mozilla-News-Host: news://localhost In-Reply-To: Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (uucp.dinoex.sub.de [194.45.71.2]); Tue, 10 Apr 2018 21:13:07 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2018 19:13:22 -0000 Results: 1. The tdq_ridx pointer The perceived slow advance (of the tdq_ridx pointer into the circular array) is correct behaviour. McKusick writes: >The pointer is advanced once per system tick, although it may not >advance on a tick until the currently selected queue is empty. Since >each thread is given a maximum time slice and no threads may be added >to the current position, the queue will drain in a bounded amount of >time. Therefore, it is also normal that the process (the piglet in this case) does run until it's time slice (aka quantum) is used up. 2. The influence of preempt_thresh This can be found in tdq_runq_add(). A simplified description of the logic there is as follows: td_priority < 152 ? -> add to realtime-queue td_priority <= 223 ? -> add to timeshare-queue if preempted circular-index = tdq_ridx else circular_index = tdq_idx + td_priority else -> add to idle-queue If the thread had been preempted, it is reinserted at the current working position of the circular array, otherwise the position is calculated from thread priority. 3. The quorum Most of the task switches come from device interrupts. Those are running at priority intr:8 or intr:12. So, as soon as preempt_thresh is 12 or bigger, the piglet is almost always reinserted in the runqueue due to preemption. And, as we see, in that case we do not have a scheduling, we have a simple resume! A real scheduling happens only after the quorum is exhausted. Therefore, reducing the quorum helps. 4. History In r171713 was this behaviour deliberately introduced. In r220198 it was fixed, with a focus on CPU-hogs and single-CPU. In r239157 the fix was undone due to performance considerations, with the focus on rescheduling only at end of the time-slice. 5. Conclusion The current defaults seem not very well suited for certain CPU-intense tasks. Possible solutions are one of: * not use SCHED_ULE * not use preemption * change kern.sched.quorum to minimal value. P. From owner-freebsd-stable@freebsd.org Wed Apr 11 01:48:44 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FA19F88C2D for ; Wed, 11 Apr 2018 01:48:44 +0000 (UTC) (envelope-from curtis@orleans.occnc.com) Received: from mta1-em1.orleans.occnc.com (mta1-em1.orleans.occnc.com [IPv6:2001:470:88e6:1::141]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mta1.orleans.occnc.com", Issuer "OCCNC secondary CA (ca1a2a)" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B28B67FC6E for ; Wed, 11 Apr 2018 01:48:43 +0000 (UTC) (envelope-from curtis@orleans.occnc.com) Received: from harbor1.v6only.occnc.com (harbor1-em2.v6only.occnc.com [IPv6:2001:470:88e6:2::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: curtis@occnc.com) by mta1-em1.orleans.occnc.com (Postfix) with ESMTPSA id 1E41A12505; Tue, 10 Apr 2018 21:48:42 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=orleans.occnc.com; s=curtis-orleans-20180228-204217; t=1523411322; bh=VRA9kKauBtSa02XysMULTq0YkRBYNXHJNx3oHVVt4PM=; h=To:cc:Reply-To:From:Subject:In-reply-to:Date; b=zOxv1q4nm7J3MqGkK7smRW3P4lIvsj1WPo8apHcO3Ze0gqrO5Rx1outwm5vWPV88W gG1dXa5CF4BKUNjL7jAgfwXjAYu+MApXEAY2+IPz/Qmlta6fjwK++6pLma0ihTzPjS 0b5GeoxX9f2yvIm8nXb1HLHFJ/1yVwSYxaYbiUKccZM46iAtb96/z5QYCrXMqCMxgV Ck6FkccewScwHjVYLEBb/FD7Ft0KtM7lTFNe2Vkh5T8mP1dP1qXN7obxVYMj0GXLKJ DLbqT5sJJZGyda5OPQU1Sek6QCdb6vXbYcp81fy+uiUj2YCml4+Twr6TmR4zxjYoRb ESC/yQGMDZLUw== To: Curtis Villamizar cc: freebsd-stable@freebsd.org Reply-To: Curtis Villamizar From: Curtis Villamizar Subject: Re: kern.maxswzone causing serious problems In-reply-to: Your message of "Thu, 29 Mar 2018 21:23:08 -0400." <387a65b7-d221-0a10-b801-1dd573054e10@orleans.occnc.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <13019.1523406353.1@harbor1-em2.v6only.occnc.com> Date: Tue, 10 Apr 2018 20:25:53 -0400 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2018 01:48:44 -0000 Replying to myself... again. Bug report https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227436 has been submitted. Hopefully this can get picked up in 11.2 and maybe patched into 11.1. If not at least go into 12. Curtis In message <387a65b7-d221-0a10-b801-1dd573054e10@orleans.occnc.com> Curtis Villamizar writes: > [...] > > Will check back later after regression testing. Apparently the best way > to get this to get some attention is to file a bug so once the changes > are fully verified in the regression testing I'll do that. Just in case > there is some further interaction with using more swap that recommended > by the current code. > > Curtis > [...] From owner-freebsd-stable@freebsd.org Wed Apr 11 21:06:58 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5ED2BF91986 for ; Wed, 11 Apr 2018 21:06:58 +0000 (UTC) (envelope-from debra.brandon@gsuitebox.com) Received: from mail-io0-x24e.google.com (mail-io0-x24e.google.com [IPv6:2607:f8b0:4001:c06::24e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0089580149 for ; Wed, 11 Apr 2018 21:06:57 +0000 (UTC) (envelope-from debra.brandon@gsuitebox.com) Received: by mail-io0-x24e.google.com with SMTP id y4so2910941iod.5 for ; Wed, 11 Apr 2018 14:06:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gsuitebox.com; s=google; h=mime-version:message-id:date:subject:from:to; bh=HThQU/lJr8LdJbRWN/K1IQ6FwNH5uBnMG/ehGhuywnk=; b=XazUcT55Ip5voL+xi202cn/UOp/WXP3U98TWb6rjmihW1p19SjSlCxW0eRygeLxQ6Z YjoefJBvXaHjGSzZgmEeUNh2sRshhXiEIFnx6wrXVV4d1zAPloFJmno7ZXFLekSCVG/F FqpzqAjxK3gNVS3CrDKlIBpCXKMRMLxx0/tqbMMozv9hBYEZs3i97ZL3cXKE3g4trqBr T8nvL1OWPcudpWYEP/f09PNOp48fJd6FWoZqdW3KfqZXYnToVLiwZqm6A3VGZsZZdwm/ 5c2T3UAVeav/4+B9vK+SpDE3oCCYsKRdPar9nal8XxMoGCX0Pmd0zt9wmHsa7fsbfMyP NJmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:message-id:date:subject:from:to; bh=HThQU/lJr8LdJbRWN/K1IQ6FwNH5uBnMG/ehGhuywnk=; b=VohPiCp3vxXAKSyo1r8KgWf0S7UP9PQuSzpjykIoY6ZlqU3hKkbbIevRCKotUL6iP3 gZuEaxtUL9dAbS18ZlGQ4ja3uHU7Cy+Uj7yWuoqkjl4jBxkgz9gYsAFrYHkTFx0Km5QE 9Rc2imu8f7XsmtwFWUqbMem5JhvV9OFi+XMMj8VjMO0dDLVy3Q7bny8z95eOBe7YhjVb Rnc75hhm+HYD498sYkSn+h5RZe1El+sBJbe0pZmYBz093BztJQprBb6kOkDmqSALq6xy NvmvRr7+qIhVRLjaL6w1rULdQiNyB7KB5eywdtKGZnt/1T9GaWIO4fv3ejK15vPA5EtB q48A== X-Gm-Message-State: ALQs6tDHsSQujWyhYxNeD0/ytUoGsdS/Sf/Hp/afhyVJoWXvoB9lv/ji EQvPTwiQ09XifOWVkApoom8V+IY+3LR0 X-Google-Smtp-Source: AIpwx48hD7hWPAu/3XfICjDRJ9dGy7Cx+jnwO2AoqNQtam84gzzKp7QOtXHRMLO2q7Cyt3Gi1ZedNempZA== MIME-Version: 1.0 X-Received: by 2002:a24:1982:: with SMTP id b124-v6mr2680259itb.50.1523480817523; Wed, 11 Apr 2018 14:06:57 -0700 (PDT) Message-ID: <000000000000f362b205699904de@google.com> Date: Wed, 11 Apr 2018 21:06:57 +0000 Subject: AWS/ Azure Global Accounts. From: debra.brandon@gsuitebox.com To: freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8"; format=flowed; delsp=yes Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2018 21:06:58 -0000 PGRpdiBkaXI9Imx0ciI+PHAgY2xhc3M9ImdtYWlsLU1zb05vU3BhY2luZyI+SGksPC9wPg0KDQo8 cCBjbGFzcz0iZ21haWwtTXNvTm9TcGFjaW5nIj5Ib3cgYWJvdXQgdGFyZ2V0aW5nIENsb3VkIFVz ZXJzIGFjcm9zcyB0aGUgIA0KR2xvYmUgZm9yIHlvdXINCmNsb3VkIG1hcmtldCBFeHBhbnNpb24u PC9wPg0KDQo8cCBjbGFzcz0iZ21haWwtTXNvTm9TcGFjaW5nIj48Yj5XZSBoYXZlIDEwMCUNCk9w dC1JbiBEYXRhIEludGVsbGlnZW5jZSBvZjwvYj46PC9wPg0KDQo8cCBjbGFzcz0iZ21haWwtTXNv Tm9TcGFjaW5nIiBzdHlsZT0ibWFyZ2luLWxlZnQ6MC41aW4iPjxzcGFuICANCnN0eWxlPSJmb250 LWZhbWlseTpXaW5nZGluZ3MiPsO8PHNwYW4gIA0Kc3R5bGU9ImZvbnQtdmFyaWFudC1udW1lcmlj Om5vcm1hbDtmb250LXZhcmlhbnQtZWFzdC1hc2lhbjpub3JtYWw7Zm9udC1zdHJldGNoOm5vcm1h bDtmb250LXNpemU6N3B0O2xpbmUtaGVpZ2h0Om5vcm1hbDtmb250LWZhbWlseTomcXVvdDtUaW1l cyAgDQpOZXcgUm9tYW4mcXVvdDsiPsKgDQo8L3NwYW4+PC9zcGFuPkFtYXpvbiBXZWIgU2Vydmlj ZXMgKEFXUykgVXNlcnM8c3Bhbj48L3NwYW4+PC9wPg0KDQo8cCBjbGFzcz0iZ21haWwtTXNvTm9T cGFjaW5nIiBzdHlsZT0ibWFyZ2luLWxlZnQ6MC41aW4iPjxzcGFuICANCnN0eWxlPSJmb250LWZh bWlseTpXaW5nZGluZ3MiPsO8PHNwYW4gIA0Kc3R5bGU9ImZvbnQtdmFyaWFudC1udW1lcmljOm5v cm1hbDtmb250LXZhcmlhbnQtZWFzdC1hc2lhbjpub3JtYWw7Zm9udC1zdHJldGNoOm5vcm1hbDtm b250LXNpemU6N3B0O2xpbmUtaGVpZ2h0Om5vcm1hbDtmb250LWZhbWlseTomcXVvdDtUaW1lcyAg DQpOZXcgUm9tYW4mcXVvdDsiPsKgDQo8L3NwYW4+PC9zcGFuPk1pY3Jvc29mdCBBenVyZSBVc2Vy czxzcGFuPjwvc3Bhbj48L3A+DQoNCjxwIGNsYXNzPSJnbWFpbC1Nc29Ob1NwYWNpbmciIHN0eWxl PSJtYXJnaW4tbGVmdDowLjVpbiI+PHNwYW4gIA0Kc3R5bGU9ImZvbnQtZmFtaWx5OldpbmdkaW5n cyI+w7w8c3BhbiAgDQpzdHlsZT0iZm9udC12YXJpYW50LW51bWVyaWM6bm9ybWFsO2ZvbnQtdmFy aWFudC1lYXN0LWFzaWFuOm5vcm1hbDtmb250LXN0cmV0Y2g6bm9ybWFsO2ZvbnQtc2l6ZTo3cHQ7 bGluZS1oZWlnaHQ6bm9ybWFsO2ZvbnQtZmFtaWx5OiZxdW90O1RpbWVzICANCk5ldyBSb21hbiZx dW90OyI+wqANCjwvc3Bhbj48L3NwYW4+R29vZ2xlIENsb3VkIFBsYXRmb3JtIFVzZXJzPHNwYW4+ PC9zcGFuPjwvcD4NCg0KPHAgY2xhc3M9ImdtYWlsLU1zb05vU3BhY2luZyIgc3R5bGU9Im1hcmdp bi1sZWZ0OjAuNWluIj48c3BhbiAgDQpzdHlsZT0iZm9udC1mYW1pbHk6V2luZ2RpbmdzIj7DvDxz cGFuICANCnN0eWxlPSJmb250LXZhcmlhbnQtbnVtZXJpYzpub3JtYWw7Zm9udC12YXJpYW50LWVh c3QtYXNpYW46bm9ybWFsO2ZvbnQtc3RyZXRjaDpub3JtYWw7Zm9udC1zaXplOjdwdDtsaW5lLWhl aWdodDpub3JtYWw7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgIA0KTmV3IFJvbWFuJnF1b3Q7Ij7C oA0KPC9zcGFuPjwvc3Bhbj5JQk0gQ2xvdWQgVXNlcnM8c3Bhbj48L3NwYW4+PC9wPg0KDQo8cCBj bGFzcz0iZ21haWwtTXNvTm9TcGFjaW5nIiBzdHlsZT0ibWFyZ2luLWxlZnQ6MC41aW4iPjxzcGFu ICANCnN0eWxlPSJmb250LWZhbWlseTpXaW5nZGluZ3MiPsO8PHNwYW4gIA0Kc3R5bGU9ImZvbnQt dmFyaWFudC1udW1lcmljOm5vcm1hbDtmb250LXZhcmlhbnQtZWFzdC1hc2lhbjpub3JtYWw7Zm9u dC1zdHJldGNoOm5vcm1hbDtmb250LXNpemU6N3B0O2xpbmUtaGVpZ2h0Om5vcm1hbDtmb250LWZh bWlseTomcXVvdDtUaW1lcyAgDQpOZXcgUm9tYW4mcXVvdDsiPsKgDQo8L3NwYW4+PC9zcGFuPk9y YWNsZSBDbG91ZCBhbmQgbWFueSBtb3JlLjxzcGFuPjwvc3Bhbj48L3A+DQoNCjxwIGNsYXNzPSJn bWFpbC1Nc29Ob1NwYWNpbmciPktpbmRseSBsZXQgbWUga25vdyB5b3VyIGludGVyZXN0IHRvIHBy b3ZpZGUgIA0KeW91IHdpdGgNCmRldGFpbGVkIGluZm9ybWF0aW9uIGZvciB0aGUgc2FtZS48L3A+ DQoNCjxwIGNsYXNzPSJnbWFpbC1Nc29Ob1NwYWNpbmciPlRoYW5rcyw8c3Bhbj48L3NwYW4+PC9w Pg0KDQo8cCBjbGFzcz0iZ21haWwtTXNvTm9TcGFjaW5nIj5EZWJyYSBCcmFuZG9uPHNwYW4+PC9z cGFuPjwvcD4NCg0KPHAgY2xhc3M9ImdtYWlsLU1zb05vU3BhY2luZyI+U3IgRGF0YSBTcGVjaWFs aXN0PC9wPg0KDQo8cCBjbGFzcz0iZ21haWwtTXNvTm9TcGFjaW5nIj5UbyByZW1vdmUgZnJvbSB0 aGlzIG1haWxpbmc6IHJlcGx5IHdpdGggIA0Kc3ViamVjdCBsaW5lIGFzDQomcXVvdDtsZWF2ZSBv dXQuJnF1b3Q7PHNwYW4+PC9zcGFuPjwvcD48L2Rpdj4NCjxwPiZuYnNwOzwvcD48YSBzdHlsZT0n ZGlzcGxheTogYmxvY2s7IG1hcmdpbjogMzJweCAwIDQwcHggMDsgcGFkZGluZzogIA0KMTBweDsg Zm9udC1zaXplOiAxZW07IHRleHQtYWxpZ246IGNlbnRlcjsgYm9yZGVyOiAwOyBib3JkZXItdG9w OiAxcHggc29saWQgIA0KZ3JheTsgJyBocmVmPSdodHRwczovL2dvby5nbC8ya3NkUnYnPnBvd2Vy ZWQgYnkgR1NNLiBGcmVlIG1haWwgbWVyZ2UgYW5kICANCmVtYWlsIG1hcmtldGluZyBzb2Z0d2Fy ZSBmb3IgR21haWwuPC9hPg0K From owner-freebsd-stable@freebsd.org Thu Apr 12 10:30:29 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 046FCF86ED6; Thu, 12 Apr 2018 10:30:29 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 480E178D58; Thu, 12 Apr 2018 10:30:27 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from bsdrookie.norma.com. (asterisk.enaza.ru [91.237.76.254]) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id w3CAUMZZ088475 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 12 Apr 2018 15:30:23 +0500 (YEKT) (envelope-from eugene@zhegan.in) To: freebsd-stable@freebsd.org, freebsd-fs@freebsd.org From: "Eugene M. Zheganin" Subject: HAST, cyclic singal 6, and inability to start Message-ID: Date: Thu, 12 Apr 2018 15:30:22 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 Content-Language: en-US X-Spamd-Result: default: False [-998.10 / 25.00] WHITELISTED_IPS(-999.00)[91.237.76.254] AUTH_NA(1.00)[] RCVD_COUNT_TWO(0.00)[2] MID_RHS_MATCH_FROM(0.00)[] DMARC_NA(0.00)[zhegan.in] FROM_EQ_ENVFROM(0.00)[] ARC_NA(0.00)[] ASN(0.00)[asn:57973, ipnet:91.237.76.0/24, country:RU] FROM_HAS_DN(0.00)[] R_SPF_NA(0.00)[] TO_MATCH_ENVRCPT_ALL(0.00)[] TO_DN_NONE(0.00)[] MIME_GOOD(-0.10)[multipart/alternative, text/plain] RCVD_TLS_ALL(0.00)[] RCPT_COUNT_TWO(0.00)[2] R_DKIM_NA(0.00)[] IP_SCORE(0.00)[ip: (-9.89), ipnet: 91.237.76.0/24(-7.82), asn: 57973(-4.89), country: RU(0.12)] X-Rspamd-Server: localhost X-Rspamd-Scan-Time: 0.58 X-Rspamd-Queue-ID: w3CAUMZZ088475 Content-Type: text/plain; charset=koi8-r; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Apr 2018 10:30:29 -0000 Hi. About a month ago I was experimenting with HAST on my servers, and, though I did have a complications with signal 6 on init phase, I was able to start it and it was working in test mode for a couple of weeks. After that I had to reboo both of them and now it doesn't start al all - both node hast is crashing on signal 6, and I'm unable to launch it as primary in either one. As soon as I switch from init or secondary to primary on either node - bad things are starting to happen - cyclic signal 6 for hastd and hangups for hastctl. Both nodes are running FreeBSD 11.1-RELEASE-pX (p1 and p6). Here's an extempt from the PR https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227461 I just created: ===Cut=== Node A (11.1-RELEASE-p1): ======== [root@gw0:/var/log]# service hastd start Starting hastd. [root@gw0:/var/log]# hastctl status Name Status Role Components hasta - init /dev/gpt/hasta tcp4://192.168.0.247 hastb - init /dev/gpt/hastb tcp4://192.168.0.247 [root@gw0:/var/log]# hastctl role secondary hasta [root@gw0:/var/log]# hastctl role secondary hastb [root@gw0:/var/log]# hastctl status Name Status Role Components hasta - secondary /dev/gpt/hasta tcp4://192.168.0.247 hastb - secondary /dev/gpt/hastb tcp4://192.168.0.247 Node B (11.1-RELEASE-p6): ======== [root@gw1:/var/log]# service hastd start Starting hastd. [root@gw1:/var/log]# hastctl status Name Status Role Components hasta - init /dev/gpt/hasta tcp4://192.168.0.248 hastb - init /dev/gpt/hastb tcp4://192.168.0.248 [root@gw1:/var/log]# hastctl role promary hasta usage: hastctl create [-d] [-c config] [-e extentsize] [-k keepdirty] [-m mediasize] name ... hastctl role [-d] [-c config] all | name ... hastctl list [-d] [-c config] [all | name ...] hastctl status [-d] [-c config] [all | name ...] hastctl dump [-d] [-c config] [all | name ...] [root@gw1:/var/log]# hastctl role primary hasta [root@gw1:/var/log]# hastctl role primary hastb [root@gw1:/var/log]# hastctl status (hangs) Node B dmesg: pid 26813 (hastd), uid 0: exited on signal 6 (core dumped) pid 26814 (hastd), uid 0: exited on signal 6 (core dumped) pid 26815 (hastd), uid 0: exited on signal 6 (core dumped) pid 26816 (hastd), uid 0: exited on signal 6 (core dumped) pid 26817 (hastd), uid 0: exited on signal 6 (core dumped) pid 26822 (hastd), uid 0: exited on signal 6 (core dumped) pid 26825 (hastd), uid 0: exited on signal 6 (core dumped) pid 26828 (hastd), uid 0: exited on signal 6 (core dumped) pid 26829 (hastd), uid 0: exited on signal 6 (core dumped) pid 26830 (hastd), uid 0: exited on signal 6 (core dumped) pid 26831 (hastd), uid 0: exited on signal 6 (core dumped) pid 26833 (hastd), uid 0: exited on signal 6 (core dumped) pid 26836 (hastd), uid 0: exited on signal 6 (core dumped) pid 26837 (hastd), uid 0: exited on signal 6 (core dumped) Node B messages: Apr 12 15:02:49 gw1 kernel: pid 26891 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:02:50 gw1 hastd[26679]: [hastb] (primary) Worker process killed (pid=26891, signal=6). Apr 12 15:02:50 gw1 hastd[26893]: [hasta] (primary) Descriptor 7 is open (pipe or FIFO), but should be closed. Apr 12 15:02:50 gw1 hastd[26893]: [hasta] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/hastd.c, line 303. Apr 12 15:02:50 gw1 kernel: pid 26893 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:02:51 gw1 hastd[26679]: [hasta] (primary) Worker process killed (pid=26893, signal=6). Apr 12 15:02:51 gw1 hastd[26896]: [hastb] (primary) Descriptor 7 is open (pipe or FIFO), but should be closed. Apr 12 15:02:51 gw1 hastd[26896]: [hastb] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/hastd.c, line 303. Apr 12 15:02:52 gw1 kernel: pid 26896 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:02:52 gw1 hastd[26679]: [hastb] (primary) Worker process killed (pid=26896, signal=6). Apr 12 15:02:52 gw1 hastd[26900]: [hasta] (primary) Descriptor 7 is open (pipe or FIFO), but should be closed. Apr 12 15:02:52 gw1 hastd[26900]: [hasta] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/hastd.c, line 303. Apr 12 15:02:53 gw1 kernel: pid 26900 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:02:54 gw1 hastd[26679]: [hasta] (primary) Worker process killed (pid=26900, signal=6). Apr 12 15:02:54 gw1 hastd[26904]: [hastb] (primary) Descriptor 7 is open (pipe or FIFO), but should be closed. Apr 12 15:02:54 gw1 hastd[26904]: [hastb] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/hastd.c, line 303. Apr 12 15:02:54 gw1 kernel: pid 26904 (hastd), uid 0: exited on signal 6 (core dumped) Now when I'm trying to switch A to primary: [root@gw0:/var/log]# hastctl role primary hastb [root@gw0:/var/log]# hastctl role primary hasta [root@gw0:/var/log]# hastctl status (hangs) Node A dmesg: pid 72301 (hastd), uid 0: exited on signal 6 (core dumped) pid 72328 (hastd), uid 0: exited on signal 6 (core dumped) pid 72355 (hastd), uid 0: exited on signal 6 (core dumped) pid 72389 (hastd), uid 0: exited on signal 6 (core dumped) pid 72412 (hastd), uid 0: exited on signal 6 (core dumped) pid 72436 (hastd), uid 0: exited on signal 6 (core dumped) pid 72467 (hastd), uid 0: exited on signal 6 (core dumped) pid 72496 (hastd), uid 0: exited on signal 6 (core dumped) pid 72514 (hastd), uid 0: exited on signal 6 (core dumped) pid 72530 (hastd), uid 0: exited on signal 6 (core dumped) pid 72554 (hastd), uid 0: exited on signal 6 (core dumped) pid 72584 (hastd), uid 0: exited on signal 6 (core dumped) pid 72620 (hastd), uid 0: exited on signal 6 (core dumped) pid 72656 (hastd), uid 0: exited on signal 6 (core dumped) pid 72708 (hastd), uid 0: exited on signal 6 (core dumped) pid 72759 (hastd), uid 0: exited on signal 6 (core dumped) pid 72799 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:04:40 gw0 kernel: pid 72530 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:04:41 gw0 hastd[63097]: [hasta] (primary) Worker process killed (pid=72530, signal=6). Apr 12 15:04:41 gw0 hastd[72554]: [hastb] (primary) Descriptor 8 is open (pipe or FIFO), but should be closed. Apr 12 15:04:41 gw0 hastd[72554]: [hastb] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/hastd.c, line 303. Apr 12 15:04:41 gw0 kernel: pid 72554 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:04:42 gw0 hastd[63097]: [hastb] (primary) Worker process killed (pid=72554, signal=6). Apr 12 15:04:42 gw0 hastd[72584]: [hasta] (primary) Descriptor 8 is open (pipe or FIFO), but should be closed. Apr 12 15:04:42 gw0 hastd[72584]: [hasta] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/hastd.c, line 303. Apr 12 15:04:42 gw0 kernel: pid 72584 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:04:43 gw0 hastd[63097]: [hasta] (primary) Worker process killed (pid=72584, signal=6). Apr 12 15:04:43 gw0 hastd[72620]: [hastb] (primary) Descriptor 8 is open (pipe or FIFO), but should be closed. Apr 12 15:04:43 gw0 hastd[72620]: [hastb] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/hastd.c, line 303. Apr 12 15:04:43 gw0 kernel: pid 72620 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:04:44 gw0 hastd[63097]: [hastb] (primary) Worker process killed (pid=72620, signal=6). Apr 12 15:04:44 gw0 hastd[72656]: [hasta] (primary) Descriptor 8 is open (pipe or FIFO), but should be closed. Apr 12 15:04:44 gw0 hastd[72656]: [hasta] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/hastd.c, line 303. Apr 12 15:04:44 gw0 kernel: pid 72656 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:04:45 gw0 hastd[63097]: [hasta] (primary) Worker process killed (pid=72656, signal=6). Apr 12 15:04:45 gw0 hastd[72708]: [hastb] (primary) Descriptor 8 is open (pipe or FIFO), but should be closed. Apr 12 15:04:45 gw0 hastd[72708]: [hastb] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/has td.c, line 303. Apr 12 15:04:45 gw0 kernel: pid 72708 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:04:46 gw0 hastd[63097]: [hastb] (primary) Worker process killed (pid=72708, signal=6). Apr 12 15:04:46 gw0 hastd[72759]: [hasta] (primary) Descriptor 8 is open (pipe or FIFO), but should be closed. Apr 12 15:04:46 gw0 hastd[72759]: [hasta] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/has td.c, line 303. Apr 12 15:04:46 gw0 kernel: pid 72759 (hastd), uid 0: exited on signal 6 (core dumped) Apr 12 15:04:47 gw0 hastd[63097]: [hasta] (primary) Worker process killed (pid=72759, signal=6). Apr 12 15:04:47 gw0 hastd[72799]: [hastb] (primary) Descriptor 8 is open (pipe or FIFO), but should be closed. Apr 12 15:04:47 gw0 hastd[72799]: [hastb] (primary) Aborted at function descriptors_assert, file /usr/src/sbin/hastd/has td.c, line 303. Apr 12 15:04:47 gw0 kernel: pid 72799 (hastd), uid 0: exited on signal 6 (core dumped) Node A config: ============== resource hasta { local /dev/gpt/hasta on gw0 { remote tcp4://192.168.0.247 source tcp4://192.168.0.248 } on gw1 { remote tcp4://192.168.0.248 source tcp4://192.168.0.247 } } resource hastb { local /dev/gpt/hastb on gw0 { remote tcp4://192.168.0.247 source tcp4://192.168.0.248 } on gw1 { remote tcp4://192.168.0.248 source tcp4://192.168.0.247 } } Node B config: ============== resource hasta { local /dev/gpt/hasta on gw0 { remote tcp4://192.168.0.247 source tcp4://192.168.0.248 } on gw1 { remote tcp4://192.168.0.248 source tcp4://192.168.0.247 } } resource hastb { local /dev/gpt/hastb on gw0 { remote tcp4://192.168.0.247 source tcp4://192.168.0.248 } on gw1 { remote tcp4://192.168.0.248 source tcp4://192.168.0.247 } } Backtrace: (gdb) bt #0 0x000000080155a84a in thr_kill () from /lib/libc.so.7 #1 0x000000080155a814 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52 #2 0x000000080155a789 in abort () at /usr/src/lib/libc/stdlib/abort.c:65 #3 0x0000000000414579 in pjdlog_abort (func=0x420e3f "descriptors_assert", file=0x420aeb "/usr/src/sbin/hastd/hastd.c", line=303, failedexpr=0x0, fmt=) at /usr/src/sbin/hastd/pjdlog.c:613 #4 0x0000000000408267 in descriptors_assert (res=0x80204b400, pjdlogmode=) at /usr/src/sbin/hastd/hastd.c:303 #5 0x00000000004146eb in hastd_primary (res=0x80204b400) at /usr/src/sbin/hastd/primary.c:1030 #6 0x000000000040a55a in check_signals () at /usr/src/sbin/hastd/hastd.c:359 #7 0x0000000000408852 in main (argc=, argv=) at /usr/src/sbin/hastd/hastd.c:1138 #8 0x0000000000403b0f in _start () #9 0x000000080064f000 in ?? () #10 0x0000000000000000 in ?? () (gdb) ===Cut=== If somebody has any idea how do I bring it up - please let me know. Thanks. Eugene. From owner-freebsd-stable@freebsd.org Sat Apr 14 11:31:31 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE870F9714C for ; Sat, 14 Apr 2018 11:31:30 +0000 (UTC) (envelope-from bmr@ringman.ch) Received: from mail-yw0-x22c.google.com (mail-yw0-x22c.google.com [IPv6:2607:f8b0:4002:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56BC380D41 for ; Sat, 14 Apr 2018 11:31:30 +0000 (UTC) (envelope-from bmr@ringman.ch) Received: by mail-yw0-x22c.google.com with SMTP id g9so4018940ywb.12 for ; Sat, 14 Apr 2018 04:31:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ringman-ch.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=rVE5xTGKHQ6ThJSUHSBmxAqnCyHp7YtzXYQ/Frd2gWc=; b=iy6cvgQOcLa12gJHMyNmDTKXzWOpHs/qCWaffHPRtEFPfgTkQXZKMOSJZHGYDeUn3q uDoQV0EkzZUqtCMGK58ayOdzB1sNSDfhpWXNvB1r3fKb6pioaw0xfTGIYxlzXK6D5jp/ nnbsXdjeyZdZNPB/6v8yA8+RfoaclqSt2uZaDcQ//inxw3Ikb5W6u0/vFgGOWgI5xcwg nTV4vYpJWmtDpFmorels48x3C/T2DBIWQUfbCZOBj8BbQ/2oF4Sx8MNfh2xdI7+BZ8sD 0IVApt+oI4XL7ZV5JBzVA2aSv3WGqFOQ8mUEqA3hHFzqEFWBbMR336ebIzwYcr7jwUw+ fOIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=rVE5xTGKHQ6ThJSUHSBmxAqnCyHp7YtzXYQ/Frd2gWc=; b=qumxB4hw2h1ywMi0Lcjwc2sqGrqVLaCuKetvyfxKVLrU4DfA+V4ledazITd8b1IiWC NJYRUPEFp8GejsLAmeWeAUOZrNR3NmKHXvtoG0uNchvryCQILCnGaUfLWIH0zvQV7oSE XcOisgig5VZACtPbDMrD1rVc2lqik54gySeoPL01YoDFhFSiiUV1F3YLrGvFqF/rc+pz F/CKdUsTpHY+48boYIxTH8Idw0PxkloYUW03lX/Xl9SsBa+abVnrgajMoDSOqvN5rNS1 rdW/W/vocXmsIsyaO8kRB9FmOWcxl1zTDUSXvBK26nBjaw5ozOj2iVg1fiwL2/NeesSZ QmHQ== X-Gm-Message-State: ALQs6tAIIjipn3ii9CxY/yjtT3VDlw3LnBbn8th+YlT4nlH7NQYETkgm PVJWhrVl5kEF4+yHqfZDGcmWQSgQr2Z3AvfZfO7NPXeNAzDr5g== X-Google-Smtp-Source: AIpwx4/RMtENk43wCV86liDfiiB4UHwM2a8/KXocq75tdJ8i8qLQDFb9OCSWi6kZ9oX3LA6jrupwz1okg5weoGVDOkM= X-Received: by 10.129.51.196 with SMTP id z187mr561409ywz.512.1523705489355; Sat, 14 Apr 2018 04:31:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.128.133 with HTTP; Sat, 14 Apr 2018 04:31:28 -0700 (PDT) In-Reply-To: <201804132232.w3DMWSmI019256@repo.freebsd.org> References: <201804132232.w3DMWSmI019256@repo.freebsd.org> From: Magnus Ringman Date: Sat, 14 Apr 2018 13:31:28 +0200 Message-ID: Subject: Re: svn commit: r332493 - stable/11/sys/net To: Brooks Davis Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org, freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Apr 2018 11:31:31 -0000 Hi Brooks, this MFC missed your r331077 (https://reviews.freebsd.org/D14706) thus stable buildkernel currently breaks on missing those two macros. (_IOC_NEWLEN and _IOC_NEWTYPE for searchability) Sk=C3=A5l, Magnus On Sat, Apr 14, 2018 at 12:32 AM, Brooks Davis wrote: > Author: brooks > Date: Fri Apr 13 22:32:28 2018 > New Revision: 332493 > URL: https://svnweb.freebsd.org/changeset/base/332493 > > Log: > MFC r332088: > > Add 32-bit compat for ioctls that take struct ifgroupreq. > > Use an accessor to access ifgr_group and ifgr_groups. > > Use an macro CASE_IOC_IFGROUPREQ(cmd) in place of case statements such > as "case SIOCAIFGROUP:". This avoids poluting the switch statements > with large numbers of #ifdefs. > > Reviewed by: kib > Obtained from: CheriBSD > Sponsored by: DARPA, AFRL > Differential Revision: https://reviews.freebsd.org/D14960 > > Modified: > stable/11/sys/net/if.c > stable/11/sys/net/if.h > Directory Properties: > stable/11/ (props changed) > > Modified: stable/11/sys/net/if.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- stable/11/sys/net/if.c Fri Apr 13 21:19:06 2018 (r332492) > +++ stable/11/sys/net/if.c Fri Apr 13 22:32:28 2018 (r332493) > @@ -133,8 +133,25 @@ struct ifreq32 { > CTASSERT(sizeof(struct ifreq) =3D=3D sizeof(struct ifreq32)); > CTASSERT(__offsetof(struct ifreq, ifr_ifru) =3D=3D > __offsetof(struct ifreq32, ifr_ifru)); > -#endif > > +struct ifgroupreq32 { > + char ifgr_name[IFNAMSIZ]; > + u_int ifgr_len; > + union { > + char ifgru_group[IFNAMSIZ]; > + uint32_t ifgru_groups; > + } ifgr_ifgru; > +}; > +#define _CASE_IOC_IFGROUPREQ_32(cmd) \ > + case _IOC_NEWTYPE((cmd), struct ifgroupreq32): > +#else > +#define _CASE_IOC_IFGROUPREQ_32(cmd) > +#endif /* COMPAT_FREEBSD32 */ > + > +#define CASE_IOC_IFGROUPREQ(cmd) \ > + _CASE_IOC_IFGROUPREQ_32(cmd) \ > + case (cmd) > + > union ifreq_union { > struct ifreq ifr; > #ifdef COMPAT_FREEBSD32 > @@ -142,6 +159,13 @@ union ifreq_union { > #endif > }; > > +union ifgroupreq_union { > + struct ifgroupreq ifgr; > +#ifdef COMPAT_FREEBSD32 > + struct ifgroupreq32 ifgr32; > +#endif > +}; > + > SYSCTL_NODE(_net, PF_LINK, link, CTLFLAG_RW, 0, "Link layers"); > SYSCTL_NODE(_net_link, 0, generic, CTLFLAG_RW, 0, "Generic link-manageme= nt"); > > @@ -1490,17 +1514,42 @@ if_delgroups(struct ifnet *ifp) > IFNET_WUNLOCK(); > } > > +static char * > +ifgr_group_get(void *ifgrp) > +{ > + union ifgroupreq_union *ifgrup; > + > + ifgrup =3D ifgrp; > +#ifdef COMPAT_FREEBSD32 > + if (SV_CURPROC_FLAG(SV_ILP32)) > + return (&ifgrup->ifgr32.ifgr_ifgru.ifgru_group[0]); > +#endif > + return (&ifgrup->ifgr.ifgr_ifgru.ifgru_group[0]); > +} > + > +static struct ifg_req * > +ifgr_groups_get(void *ifgrp) > +{ > + union ifgroupreq_union *ifgrup; > + > + ifgrup =3D ifgrp; > +#ifdef COMPAT_FREEBSD32 > + if (SV_CURPROC_FLAG(SV_ILP32)) > + return ((struct ifg_req *)(uintptr_t) > + ifgrup->ifgr32.ifgr_ifgru.ifgru_groups); > +#endif > + return (ifgrup->ifgr.ifgr_ifgru.ifgru_groups); > +} > + > /* > - * Stores all groups from an interface in memory pointed > - * to by data > + * Stores all groups from an interface in memory pointed to by ifgr. > */ > static int > -if_getgroup(struct ifgroupreq *data, struct ifnet *ifp) > +if_getgroup(struct ifgroupreq *ifgr, struct ifnet *ifp) > { > int len, error; > struct ifg_list *ifgl; > struct ifg_req ifgrq, *ifgp; > - struct ifgroupreq *ifgr =3D data; > > if (ifgr->ifgr_len =3D=3D 0) { > IF_ADDR_RLOCK(ifp); > @@ -1511,7 +1560,7 @@ if_getgroup(struct ifgroupreq *data, struct ifnet *= ifp > } > > len =3D ifgr->ifgr_len; > - ifgp =3D ifgr->ifgr_groups; > + ifgp =3D ifgr_groups_get(ifgr); > /* XXX: wire */ > IF_ADDR_RLOCK(ifp); > TAILQ_FOREACH(ifgl, &ifp->if_groups, ifgl_next) { > @@ -1535,12 +1584,11 @@ if_getgroup(struct ifgroupreq *data, struct ifnet= *ifp > } > > /* > - * Stores all members of a group in memory pointed to by data > + * Stores all members of a group in memory pointed to by igfr > */ > static int > -if_getgroupmembers(struct ifgroupreq *data) > +if_getgroupmembers(struct ifgroupreq *ifgr) > { > - struct ifgroupreq *ifgr =3D data; > struct ifg_group *ifg; > struct ifg_member *ifgm; > struct ifg_req ifgrq, *ifgp; > @@ -1563,7 +1611,7 @@ if_getgroupmembers(struct ifgroupreq *data) > } > > len =3D ifgr->ifgr_len; > - ifgp =3D ifgr->ifgr_groups; > + ifgp =3D ifgr_groups_get(ifgr); > TAILQ_FOREACH(ifgm, &ifg->ifg_members, ifgm_next) { > if (len < sizeof(ifgrq)) { > IFNET_RUNLOCK(); > @@ -2793,34 +2841,28 @@ ifhwioctl(u_long cmd, struct ifnet *ifp, caddr_t = data, > error =3D if_gethwaddr(ifp, ifr); > break; > > - case SIOCAIFGROUP: > - { > - struct ifgroupreq *ifgr =3D (struct ifgroupreq *)ifr; > - > + CASE_IOC_IFGROUPREQ(SIOCAIFGROUP): > error =3D priv_check(td, PRIV_NET_ADDIFGROUP); > if (error) > return (error); > - if ((error =3D if_addgroup(ifp, ifgr->ifgr_group))) > + if ((error =3D if_addgroup(ifp, > + ifgr_group_get((struct ifgroupreq *)data)))) > return (error); > break; > - } > > - case SIOCGIFGROUP: > - if ((error =3D if_getgroup((struct ifgroupreq *)ifr, ifp)= )) > + CASE_IOC_IFGROUPREQ(SIOCGIFGROUP): > + if ((error =3D if_getgroup((struct ifgroupreq *)data, ifp= ))) > return (error); > break; > > - case SIOCDIFGROUP: > - { > - struct ifgroupreq *ifgr =3D (struct ifgroupreq *)ifr; > - > + CASE_IOC_IFGROUPREQ(SIOCDIFGROUP): > error =3D priv_check(td, PRIV_NET_DELIFGROUP); > if (error) > return (error); > - if ((error =3D if_delgroup(ifp, ifgr->ifgr_group))) > + if ((error =3D if_delgroup(ifp, > + ifgr_group_get((struct ifgroupreq *)data)))) > return (error); > break; > - } > > default: > error =3D ENOIOCTL; > @@ -2926,7 +2968,7 @@ ifioctl(struct socket *so, u_long cmd, caddr_t data= , s > error =3D if_clone_list((struct if_clonereq *)data); > CURVNET_RESTORE(); > return (error); > - case SIOCGIFGMEMB: > + CASE_IOC_IFGROUPREQ(SIOCGIFGMEMB): > error =3D if_getgroupmembers((struct ifgroupreq *)data); > CURVNET_RESTORE(); > return (error); > > Modified: stable/11/sys/net/if.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- stable/11/sys/net/if.h Fri Apr 13 21:19:06 2018 (r332492) > +++ stable/11/sys/net/if.h Fri Apr 13 22:32:28 2018 (r332493) > @@ -512,8 +512,10 @@ struct ifgroupreq { > char ifgru_group[IFNAMSIZ]; > struct ifg_req *ifgru_groups; > } ifgr_ifgru; > +#ifndef _KERNEL > #define ifgr_group ifgr_ifgru.ifgru_group > #define ifgr_groups ifgr_ifgru.ifgru_groups > +#endif > }; > > /* > _______________________________________________ > svn-src-all@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-all > To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sat Apr 14 11:37:47 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51229F9789D; Sat, 14 Apr 2018 11:37:47 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mx.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 911B7826F4; Sat, 14 Apr 2018 11:37:45 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id w3EBbiNf089599; Sat, 14 Apr 2018 11:37:44 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id w3EBbi0U089598; Sat, 14 Apr 2018 04:37:44 -0700 (PDT) (envelope-from david) Date: Sat, 14 Apr 2018 04:37:43 -0700 From: David Wolfskill To: Magnus Ringman Cc: Brooks Davis , svn-src-stable@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: Re: svn commit: r332493 - stable/11/sys/net Message-ID: <20180414113743.GR63353@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Magnus Ringman , Brooks Davis , svn-src-stable@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-stable@freebsd.org, svn-src-stable-11@freebsd.org References: <201804132232.w3DMWSmI019256@repo.freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="E6EZ4qVJn/rWPYaj" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Apr 2018 11:37:47 -0000 --E6EZ4qVJn/rWPYaj Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Apr 14, 2018 at 01:31:28PM +0200, Magnus Ringman wrote: > Hi Brooks, this MFC missed your r331077 > (https://reviews.freebsd.org/D14706) thus stable buildkernel currently > breaks on missing those two macros. >=20 > (_IOC_NEWLEN and _IOC_NEWTYPE for searchability) >=20 > Sk=E5l, > Magnus > .... Aye; looks as if that would explain the kernel build failures I had this morning in stable/11, trying to update from r332465 -> r332496. Peace, david --=20 David H. Wolfskill david@catwhisker.org Donald Trump's criticism of others comes across as psychological projection. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --E6EZ4qVJn/rWPYaj Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEEzLfO+ReoAfQwZNd7FTnMQKBJ7hcFAlrR6AdfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEND QjdDRUY5MTdBODAxRjQzMDY0RDc3QjE1MzlDQzQwQTA0OUVFMTcACgkQFTnMQKBJ 7hf0PwgA0kLh0HJ7t3RBon/xPvwzgOnZsvfzYtWlT55yXEdkPD91wx8FpNldiZiD T5Vu4G8KUnGAyB2uAlmWesk6r3FikQEEszvCVnCF91ej0ems9naRaJdSEsRaIpyG sO51YaQQ8tDh4Ww/jve9dVVIuBYYEBaPVNzomZS6NP9onD85et6AT4MHdXZsQtY+ hMyoTFryA3AFQq1+OpPyvTUabVCqEqmVSUX5zVX/P+XKrhFMPdCTuFYdkK4+pCAI flfNzhvJqPniCRMCMjKULlpY078gvIUV+QiT3mMBdkPprYotpNxuaFq9NM9VgrXD aHwjasY6eKEmuVSR6fyFj7rVOjqwJg== =Jyh+ -----END PGP SIGNATURE----- --E6EZ4qVJn/rWPYaj-- From owner-freebsd-stable@freebsd.org Sat Apr 14 12:14:38 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62F42F9AF97 for ; Sat, 14 Apr 2018 12:14:38 +0000 (UTC) (envelope-from bmr@ringman.ch) Received: from mail-yw0-x243.google.com (mail-yw0-x243.google.com [IPv6:2607:f8b0:4002:c05::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7815F6A80A for ; Sat, 14 Apr 2018 12:14:37 +0000 (UTC) (envelope-from bmr@ringman.ch) Received: by mail-yw0-x243.google.com with SMTP id k9so227763ywa.4 for ; Sat, 14 Apr 2018 05:14:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ringman-ch.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-transfer-encoding; bh=rY6N3rQI39mrJcyNm9PLXqd1kPsZf4ikha0NChDHXQU=; b=pRdrYG16nWQVvWOv+SJdESkPpC1sdZdKDUYnrNPQya6f2cbgxOusGXawXpxUm5Qg5k AEXJis8zmF0cVKVAxUU00Q6//thF2ODp0yKpDjUm3FvD+K9nCLFnj7oxPzneypNorKv5 J1zMghvXmdsFbapqNP6vzmWhOVVYiM77JLd9Fd/iN/s9XJf72MgM+71LCg6EGBSo0a4+ rAhMfFUGjKwykDsiUGBpl6ndG0s0TUXgyIvcbTa+ES3Yeice4YlLFH5nTQLsfMeTIglz eHruIVgRejPrM5xaHwAgo3E7TNjikeEaTQfqtqr/chpQd6b3slLk1U5Ou2p1zGwOEJPf H8kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-transfer-encoding; bh=rY6N3rQI39mrJcyNm9PLXqd1kPsZf4ikha0NChDHXQU=; b=hAGRPCq2RdFgynWzUAuDUgcYazyagMZg1G4SwkaO0jE+uMLo4rG8lWGFIu/AHZtqpM HYubJXlJOhwmWXh/UfXnp+jSJ8QFOq9n5RI9tMwfjITEl/HT/l+LV10Jczgj8zQX9V7y iIUaX4b5UNPT10hrkQiArzAHuGtAyl+akrMfOogXlmPuHiQTHcANfu10WwrTCuTf/H7S 2rdC+xp8mlEQ2qnEgqt+BtF5tzJUr0UPnqur1qmk88wvWyKqegF2pnLbH5xjzDCGaHnn lh8fuBdQUCPm9UKF8BSlLDfZamnz3mtdczzTg3o/zyuL9ehqIlZI7F4y+ZOcMDtno/6C BqTA== X-Gm-Message-State: ALQs6tBRGgk1KRsjp4wF9AeoUo2c2paddCk9D7bOdNG68D+iel5Y82G1 z+EcQ0/s2xTDPCDPQ3B4vdDS8fXziypJMrgAYMhtLQ== X-Google-Smtp-Source: AIpwx486PB2fs0L7nZJXd9KvdV81b76W2uah4fEenlwFBtrzwNK2pvnCINlyS5EkZN+VEQHBgk2n1WClPyvXzKihX38= X-Received: by 10.13.216.205 with SMTP id a196mr6787707ywe.442.1523708076965; Sat, 14 Apr 2018 05:14:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.128.133 with HTTP; Sat, 14 Apr 2018 05:14:36 -0700 (PDT) In-Reply-To: <20180414113743.GR63353@albert.catwhisker.org> References: <201804132232.w3DMWSmI019256@repo.freebsd.org> <20180414113743.GR63353@albert.catwhisker.org> From: Magnus Ringman Date: Sat, 14 Apr 2018 14:14:36 +0200 Message-ID: Subject: Re: svn commit: r332493 - stable/11/sys/net To: David Wolfskill , Magnus Ringman , Brooks Davis , svn-src-stable@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org, freebsd-stable@freebsd.org, svn-src-stable-11@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Apr 2018 12:14:38 -0000 I don't think anyone has committed a fix yet (I'm not eligible, just lurking here.) Just manually integrate the missing change. At the top of your src tree: patch -p1 << __EOT --- head/sys/sys/ioccom.h 2017/11/20 19:43:44 326023 +++ head/sys/sys/ioccom.h 2018/03/16 22:23:04 331077 @@ -61,6 +61,10 @@ #define _IOW(g,n,t) _IOC(IOC_IN, (g), (n), sizeof(t)) /* this should be _IORW, but stdio got there first */ #define _IOWR(g,n,t) _IOC(IOC_INOUT, (g), (n), sizeof(t)) +/* Replace length/type in an ioctl command. */ +#define _IOC_NEWLEN(ioc, len) \ + (((~(IOCPARM_MASK << 16)) & (ioc)) | (((len) & IOCPARM_MASK) << 16)) +#define _IOC_NEWTYPE(ioc, type) _IOC_NEWLEN((ioc), sizeof(type)) #ifdef _KERNEL __EOT On Sat, Apr 14, 2018 at 1:37 PM, David Wolfskill wro= te: > On Sat, Apr 14, 2018 at 01:31:28PM +0200, Magnus Ringman wrote: >> Hi Brooks, this MFC missed your r331077 >> (https://reviews.freebsd.org/D14706) thus stable buildkernel currently >> breaks on missing those two macros. >> >> (_IOC_NEWLEN and _IOC_NEWTYPE for searchability) >> >> Sk=C3=A5l, >> Magnus >> .... > > Aye; looks as if that would explain the kernel build failures I had this > morning in stable/11, trying to update from r332465 -> r332496. > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Donald Trump's criticism of others comes across as psychological projecti= on. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. From owner-freebsd-stable@freebsd.org Sat Apr 14 12:54:36 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B089CF9E332; Sat, 14 Apr 2018 12:54:36 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33FBA74BA4; Sat, 14 Apr 2018 12:54:36 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io0-x22d.google.com with SMTP id d7so13123285ioc.11; Sat, 14 Apr 2018 05:54:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=HkybOt/pAPEQgxgxjTGZ43QknkXp0iHi6l785MPm6w0=; b=Uz0ZJ7YBBAFkwmzZO22QVOYY2idvDzcoFhfMVtMSvFRlhnpoK6WKuXnuaqNDVboZAN kh7X7DHrDQ6vat/Wi4y5kf8qHJQuxGMCrqoMi44ATEgSPGSqvOWXwnOl5pmW5p17dRtt WpP5qIEgYKQFbfWGkOw5A1P/lEbx2zq3VkwVEp1qP5td6164NtayivtZL/b/x9PM/7rc TkQmJl+CJbNi2uc6hUsK7ibyO9L2maSOjsNuDuIWloasc4ffWKP/4Uv7fpVthad55Poa 2PgMdXIgKEVb0sBzSvVrhWlPkBTsHeZn3RrYWQUHxC8KajmNpUSf0mYuWHHHJhZNlGlZ d4Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=HkybOt/pAPEQgxgxjTGZ43QknkXp0iHi6l785MPm6w0=; b=GVsmI8FNun3oOIj7Cmp2AmwMmjmS1gEJBHt6ZftRD7v5Vi0pxCQYzNIngJ2LWyTjFE IiCnjuL3kZvWygPjw+dkff0GN9xjn/RDOpvo+QyIDM2nPj6GhHNNOs8exqNvPCJ3Vi/L HD0q4L56NbZ8o8PQV5b3BGijjEYvIvQnkXKGC3O77HTMYw2ffa+Vzxsa8kjTJ3rpDk0R uHIpS/t5EFxY0x/AuvpLdCroBSdkCp83hKSuSD6p7KCHUx2YhradffQeQ/PB2PsDHpzn wY6sLSthHrHiN0igrTw038LNaYiCsF6nQsEPsnhSQPUhb3k2RIjBoHp0opAtZRCpIVjU eX0Q== X-Gm-Message-State: ALQs6tAnF9xNOCGZgEG11u45MFh/H1gXQBOQbv9/6tQhtr/XgyjRpsQY LWRlOG2xvX1yng+upxRSK7IPp8S1O4wnatB9fpQvtNiV X-Google-Smtp-Source: AIpwx49L1ajIl970yOcgRDuOb3sucGnQrwW26r+EAFOIT08hb8nXDeXl04Jnne0wd2m2BD7usRtfnNFDBF4xiBhZ4ug= X-Received: by 10.107.134.85 with SMTP id i82mr16437337iod.210.1523710475693; Sat, 14 Apr 2018 05:54:35 -0700 (PDT) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.130.197 with HTTP; Sat, 14 Apr 2018 05:54:15 -0700 (PDT) In-Reply-To: References: <201804132232.w3DMWSmI019256@repo.freebsd.org> From: Ed Maste Date: Sat, 14 Apr 2018 08:54:15 -0400 X-Google-Sender-Auth: N8Gjb2HEZ4WxU3xgarqYqg383Lk Message-ID: Subject: Re: svn commit: r332493 - stable/11/sys/net To: Magnus Ringman Cc: Brooks Davis , src-committers , svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org, freebsd-stable stable Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Apr 2018 12:54:36 -0000 On 14 April 2018 at 07:31, Magnus Ringman wrote: > Hi Brooks, this MFC missed your r331077 > (https://reviews.freebsd.org/D14706) thus stable buildkernel currently > breaks on missing those two macros. Thanks for identifying the missing commit Magnus. I've now merged it in r332502.