From owner-freebsd-current@freebsd.org Fri Dec 16 20:10:04 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FE96C8383C for ; Fri, 16 Dec 2016 20:10:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3DD57109 for ; Fri, 16 Dec 2016 20:10:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 3A47AC8383B; Fri, 16 Dec 2016 20:10:04 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39E75C8383A for ; Fri, 16 Dec 2016 20:10:04 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wj0-x232.google.com (mail-wj0-x232.google.com [IPv6:2a00:1450:400c:c01::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C25E4108; Fri, 16 Dec 2016 20:10:03 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wj0-x232.google.com with SMTP id v7so100677615wjy.2; Fri, 16 Dec 2016 12:10:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=aKLIczVWhuvI5tLGk683G7SzCAPzeQL2CB2jdD0SNBw=; b=KjpJQhgEoNWJIKoOC9656wa4RFkfGEEsTKlb/jxRF0dV6kDoSpB0iIey/ogPjyG5Mc XyW5bljUz275WbxDS4se7ZifRNvnwPFGLxb1lYrruQUhoxobuaFNOocZHe4N7vNeczgJ l6xbBiB4A7WQRnIB5wS2jzlqA+9QM+gQNNOef5j793jJyRt8nSEFQxPr+1Bft2TnNdI0 sK2Q9CGcSE0AOgCiViepD2ZAYQCAq5ySNME36QJDMV2YztT1q/+O1q++KF1xU8N9/WTs /O6C6SleX3+73iM+Xvw/oJ7rJGAi9mS+GMQyJ4tFEAQClV36gDOoUoT+eaqcyQMAOeJK a9fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=aKLIczVWhuvI5tLGk683G7SzCAPzeQL2CB2jdD0SNBw=; b=GF02ja0q/EFVtADhzwmd0l4ASTSt716jxYDt+eJA49b1HpDv/m5kwss/yznIbdPDcp 748iUNHuI2nLBvFO94W9qzbX24M2mzV6RqzOfAtZZCXdVUEx9G7Yh/nOs6Zd4a5f1o7F 04UMTgANaQIC54f/mMQ5abDXM7oKd8Wuwe4T+yd8J5lDVwElnQFm/mj+3W+gled3C15B 13P7nhLiq62o2f1moOnCTf8US9gl1oxQuKOiyNysejuYWQ4p3KEuXeRt7xP8UtPqEeDe LZz0PrrsLX49UOn8PMlN2ThTwzcKc/Rkbu/qKTbJygLCS5eG+odSe5GTpV//ylcROmmh Anww== X-Gm-Message-State: AIkVDXIIboe5UZZKRweZJSJmCMTfSCMJDqb1FVIDAqDH1McD/bxzkFGmolttzWr8bVbeFLaSzgWDhuKoDnD4bw== X-Received: by 10.194.187.103 with SMTP id fr7mr4114198wjc.99.1481919001904; Fri, 16 Dec 2016 12:10:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.44.1 with HTTP; Fri, 16 Dec 2016 12:10:01 -0800 (PST) In-Reply-To: <20161216194519.GA71398@onelab2.iet.unipi.it> References: <20161216021719.GA63374@onelab2.iet.unipi.it> <20161216194519.GA71398@onelab2.iet.unipi.it> From: Adrian Chadd Date: Fri, 16 Dec 2016 12:10:01 -0800 Message-ID: Subject: Re: best approximation of getcpu() ? To: Luigi Rizzo Cc: David Chisnall , Alan Somers , "current@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 20:10:04 -0000 On 16 December 2016 at 11:45, Luigi Rizzo wrote: > On Fri, Dec 16, 2016 at 09:29:15AM +0000, David Chisnall wrote: >> On 16 Dec 2016, at 03:10, Alan Somers wrote: >> > >> > What about pthread_setaffinity(3) and friends? You can use it to pin >> > a thread to a single CPU, and know that it will never migrate. >> >> This is not a useable solution for anything that needs to live in a libr= ary and also doesn???t solve the problem. >> >> The Linux get_cpu call() is used for caches that are somewhere between g= lobal and thread-local. Accessing them still requires a lock, but it???s v= ery likely to be uncontended (contention only happens when you???re context= switched at exactly the wrong time, or if a thread is migrated between cor= es in between the get_cpu() call and usage) and so you can use the userspac= e fast path for the lock and not suffer from cache contention effects. >> >> One x86, you can use cpuid from userspace and get the current core ID. = I have some code that does this and re-checks every few hundred accesses, s= toring the current CPU ID in a thread-local variable. Using the per-CPU ca= ches is a lot faster than using the global cache (and reduces contention on= the global cache). It would be great if we could have a syscall to do thi= s on FreeBSD (it would be even better if we could have specify a TLS variab= le that the kernel automatically updates for the userspace thread when the = scheduler migrates the thread between cores). > > indeed the following line seems to do the job for x86 > asm volatile("cpuid" : "=3Dd"(curcpu), "=3Da"(tmp), "=3Db"(tmp), = "=3Dc"(tmp) : "a"(0xb) ); > (there must be a better way to tell the compiler that eax, ebx, ecx, edx = are > all clobbered). > > 0xb is the CPUID function that returns the current APIC id for the > core (not necessarily matching the OS core-id) > > The only problem is that this instruction is serialising and slow, > seems to take some 70-100ns on several of my machines so you > cannot afford to call it at all times but need the value cached > somewhere. Exposing it as thread local storage, or a VDSO syscall, > would be nicer because the kernel knows when it is actually changing > value. The problem is your CPU ID can change in the middle of packet handling. So if you want it to be accurate, you need to bind your worker thread to a = CPU. -adrian