From owner-freebsd-hackers@freebsd.org Tue Nov 6 12:48:31 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC5CC1122B4C for ; Tue, 6 Nov 2018 12:48:30 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailout03.t-online.de (mailout03.t-online.de [194.25.134.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4AD574FC2 for ; Tue, 6 Nov 2018 12:48:29 +0000 (UTC) (envelope-from se@freebsd.org) Received: from fwd10.aul.t-online.de (fwd10.aul.t-online.de [172.20.26.152]) by mailout03.t-online.de (Postfix) with SMTP id 0FF1E4208777; Tue, 6 Nov 2018 13:48:22 +0100 (CET) Received: from Stefans-MBP-LAN.fritz.box (bpREusZcoh-zhlDv+lz9LU9+mUIYXJSYZ7tubNpOnrON81CTsiyKGVLpoyacf40QXV@[80.128.111.25]) by fwd10.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1gK0mF-2RZnSS0; Tue, 6 Nov 2018 13:48:15 +0100 Subject: Re: FreeBSD 11.x thread creation time is 9000+ microseconds on Intel Xeon Gold series CPU To: Steevan Rodrigues , mad@madpilot.net Cc: freebsd-hackers@freebsd.org, vhaisman@gmail.com References: <7bb8463e-aa5d-bade-7ae7-6c612bae65f9@gmail.com> From: Stefan Esser Openpgp: preference=signencrypt Autocrypt: addr=se@freebsd.org; prefer-encrypt=mutual; keydata= xsBNBFVxiRIBCADOLNOZBsqlplHUQ3tG782FNtVT33rQli9EjNt2fhFERHIo4NxHlWBpHLnU b0s4L/eItx7au0i7Gegv01A9LUMwOnAc9EFAm4EW3Wmoa6MYrcP7xDClohg/Y69f7SNpEs3x YATBy+L6NzWZbJjZXD4vqPgZSDuMcLU7BEdJf0f+6h1BJPnGuwHpsSdnnMrZeIM8xQ8PPUVQ L0GZkVojHgNUngJH6e21qDrud0BkdiBcij0M3TCP4GQrJ/YMdurfc8mhueLpwGR2U1W8TYB7 4UY+NLw0McThOCLCxXflIeF/Y7jSB0zxzvb/H3LWkodUTkV57yX9IbUAGA5RKRg9zsUtABEB AAHNKVN0ZWZhbiBFw59lciAoWWFob28hKSA8c3QuZXNzZXJAeWFob28uZGU+wsCWBBMBCgBA AhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AWIQSjceplnAvsyCtxUxNH67XvWv31RAUC WvLvqwUJCyUBEwAKCRBH67XvWv31REySCACc6vqcSFQCRyBRc2CV5ZBjbbnTy7VBoXbUS3/c 4Hn8I0YQ39q7//2z8vYsgLeM1mMXL4PUIU/0f0dBAFBLpxV7bntGzyCJls6SeGS/qcQKhqaI 6I7NcWg8OkIJIhUL6q238cS1ql9pU65fyHe0PP8JS08m81PDpX2/4wTE6h2jgYUy55eXRzoF MEjr1S8SSnidsBem27o7iWu9ltJsUtE86071iZlLzbuHv2nvucrjAV9cK9tHrxYT/YiY8QhT L48iWj2xIjLjg1ebmgIFZ2k881we/KTIoUugqOOR1gDSc4qwM8CA388cN3frjtl98CwhAT5T UV8tIDqri+/Z1AKwzsBNBFVxiRIBCACxI/aglzGVbnI6XHd0MTP05VK/fJub4hHdc+LQpz1M kVnCAhFbY9oecTB/togdKtfiloavjbFrb0nJhJnx57K+3SdSuu+znaQ4SlWiZOtXnkbpRWNU eMm+gtTDMSvloGAfr76RtFHskdDOLgXsHD70bKuMhlBxUCrSwGzHaD00q8iQPhJZ5itb3WPq z3B4IjiDAWTO2obD1wtAvSuHuUj/XJRsiKDKW3x13cfavkad81bZW4cpNwUv8XHLv/vaZPSA ly+hkY7NrDZydMMXVNQ7AJQufWuTJ0q7sImRcEZ5EIa98esJPey4O7C0vY405wjeyxpVZkpq ThDMurqtQFn1ABEBAAHCwHwEGAEKACYCGwwWIQSjceplnAvsyCtxUxNH67XvWv31RAUCWvLv qwUJCyUBGQAKCRBH67XvWv31RLnrB/9gzcRlpx71sDMosoZULWn7wysBJ/8AIEfIByRaHQe3 pn/KwE57pB+zFbbQqB7YzeZb7/UUgR4zU2ZbOcEfwDZcHUbj0B3fGRsS3t0uiLlAd8w0sBZb SxrqzjdpDjIbOZkxssqUmvrsN67UG1AFWH9aD24keBS7YjPBS8hLxPeYV+Xz6vUL8fRZje/Z JgiBMIwyj6g2lH/zkdnxBdC0iG1xxJOLTaghMMeQyCdH6ef8+VMyAlAJsMckbOTvx63tY8z7 DFcrnTJfbe1EziRilVsEaK8tTzJzhcTfos+f3eBYWEilxe5HzIhYKJeC7lmsSUcGwa6+9VRg a0ctmi9Z8OgX Message-ID: <0f7e583f-9ad9-f210-4bdb-1ace13ab8c5f@freebsd.org> Date: Tue, 6 Nov 2018 13:48:12 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-ID: bpREusZcoh-zhlDv+lz9LU9+mUIYXJSYZ7tubNpOnrON81CTsiyKGVLpoyacf40QXV X-TOI-MSGID: cf3ae718-8e93-4bf3-84e3-1e5f3a48c87f X-Rspamd-Queue-Id: D4AD574FC2 X-Spamd-Result: default: False [-103.14 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; ALLOW_DOMAIN_WHITELIST(-100.00)[freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.992,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mx66.freebsd.org]; NEURAL_HAM_SHORT(-1.00)[-0.995,0]; RCVD_IN_DNSWL_NONE(0.00)[81.134.25.194.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_GOOD(0.00)[81.134.25.194.rep.mailspike.net : 127.0.0.18]; IP_SCORE(-0.04)[asn: 3320(-0.20), country: DE(-0.01)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:3320, ipnet:194.25.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Nov 2018 12:48:31 -0000 Am 06.11.18 um 12:05 schrieb Steevan Rodrigues: > Thanks to all of you for the quick responses. > > I see this issue only on a particular server. > I do not see this issue on another server with slightly older processor > Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (dual socket 6 core per socket) > In this system the same program takes only about 800 usec > > I wonder whether it is problem with 11.x FreeBSD release on this particular > processor. > I came across two results (given below) from Phoronix OSbench test suite > which also show poor thread creation time for FreeBSD 11.x > > https://openbenchmarking.org/result/1804094-AR-1804096AR58 > https://openbenchmarking.org/result/1807114-RA-WINDOWSSE47 > > In this look for "Create Threads" row. This list shows, that FreeBSD-12 is fast, only FreeBSD-11 shows the long thread creation time (16000 vs 36 in whatever units), even though FreeBSD-12 was running with KPTI-Patches. This lets me think that the issue that caused these delays has been fixed, but not merged to FreeBSD-11, yet.# I guess that the cause for the delays is the synchronization IPIs between sockets, and that these might be very high due to some cores in deep sleep modes (with corresponding long wake-up delays). Regards, STefan