From owner-freebsd-threads@freebsd.org Thu Sep 24 18:11:39 2015 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58B5FA079AC for ; Thu, 24 Sep 2015 18:11:39 +0000 (UTC) (envelope-from schulz.j@gmail.com) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D9B41EC1 for ; Thu, 24 Sep 2015 18:11:39 +0000 (UTC) (envelope-from schulz.j@gmail.com) Received: by pacgz1 with SMTP id gz1so13024855pac.3 for ; Thu, 24 Sep 2015 11:11:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=6nX2tqxaDg7s44mFFLDx6QKkEiS8C2Rby3MGgIue3Fo=; b=QXAOyv2Iqy7HxP0RQpsJql60w6A9Wo60Et54C9BNFPQn1fz7KwgEWkhF+sWWxv6BRU 1axbVTRdw5iqYmDq02tl6CgpxZqxjks5/Jp4E/bPOFMHkzM/iqqFkDY4J6Q+REYiPZj2 BCtXl4N2MAml97YHmB19azJcUlFflZ2i7ntOOh8sa8dpjFSjPbRDuyI9zD/XxycI3yUL XVt19Jwopmszp4AEAjTxDSJ0yeN6c2CHnz3o9M5GBAlcNOL4ZN+QYy6WMYPGgKkk8uF6 0yfUojeViCYkaip+glb2uwQjyoKN7Y8E3aJ28jXYY7jdKY3B7fbuVZDGV6c8KUtwz8Nr kMag== X-Received: by 10.66.165.106 with SMTP id yx10mr1286690pab.102.1443118298868; Thu, 24 Sep 2015 11:11:38 -0700 (PDT) Received: from gmail.com (c-73-140-192-75.hsd1.wa.comcast.net. [73.140.192.75]) by smtp.gmail.com with ESMTPSA id yk5sm14867124pab.31.2015.09.24.11.11.36 for (version=TLSv1/SSLv3 cipher=OTHER); Thu, 24 Sep 2015 11:11:37 -0700 (PDT) Sender: Jason Schulz Date: Thu, 24 Sep 2015 11:11:27 -0700 From: Jason Schulz To: freebsd-threads@freebsd.org Subject: pthread_getaffinity_np issue Message-ID: <20150924181127.GC3104@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 18:11:39 -0000 I'm having an issue using the non-posix thread affinity methods. Specifically, I'm having trouble using CPU_COUNT. I'm trying to use the following code to get the current number of processors in a processes affinity... long cs = 0; #if defined(HAVE_LINUX) || defined(HAVE_FREEBSD) pthread_t this = pthread_self(); cpu_set_t cpus; CPU_ZERO(&cpus); int err = pthread_getaffinity_np(this, sizeof(cpus), &cpus); if (err) return err; cs = (long) CPU_COUNT(&cpus); However, I get the compiler error... clang -DHAVE_CONFIG_H -I. -g -O2 -D_THREAD_SAFE -pthread -pipe -std=c99 -D_XOPEN_SOURCE=700 -pedantic -Wall -Wextra -Wsign-conversion -Wconversion -Werror -MT thread.o -MD -MP -MF .deps/thread.Tpo -c -o thread.o thread.c thread.c:85:15: error: implicit declaration of function '__bitcountl' is invalid in C99 [-Werror,-Wimplicit-function-declaration] cs = (long) CPU_COUNT(&cpus); ^ /usr/include/sys/cpuset.h:63:24: note: expanded from macro 'CPU_COUNT' #define CPU_COUNT(p) BIT_COUNT(CPU_SETSIZE, p) ^ /usr/include/sys/bitset.h:185:14: note: expanded from macro 'BIT_COUNT' __count += __bitcountl((p)->__bits[__i]); \ ^ 1 error generated. I'm able to use CPU_ISSET to the same effect, but obviously I'd rather use CPU_COUNT. Suggestions? -Jason From owner-freebsd-threads@freebsd.org Thu Sep 24 18:32:53 2015 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B571EA085A8 for ; Thu, 24 Sep 2015 18:32:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 310591A06 for ; Thu, 24 Sep 2015 18:32:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id t8OIWkA8041618 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 24 Sep 2015 21:32:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua t8OIWkA8041618 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id t8OIWkjj041617; Thu, 24 Sep 2015 21:32:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 24 Sep 2015 21:32:46 +0300 From: Konstantin Belousov To: Jason Schulz Cc: freebsd-threads@freebsd.org Subject: Re: pthread_getaffinity_np issue Message-ID: <20150924183246.GK67105@kib.kiev.ua> References: <20150924181127.GC3104@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150924181127.GC3104@gmail.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 18:32:53 -0000 On Thu, Sep 24, 2015 at 11:11:27AM -0700, Jason Schulz wrote: > I'm having an issue using the non-posix thread affinity methods. Specifically, I'm having trouble using CPU_COUNT. I'm trying to use the following code to get the current number of processors in a processes affinity... > > > > long cs = 0; > > #if defined(HAVE_LINUX) || defined(HAVE_FREEBSD) > > pthread_t this = pthread_self(); > > cpu_set_t cpus; > > CPU_ZERO(&cpus); > > int err = pthread_getaffinity_np(this, sizeof(cpus), &cpus); > > if (err) > return err; > > cs = (long) CPU_COUNT(&cpus); > > > > However, I get the compiler error... > > > > clang -DHAVE_CONFIG_H -I. -g -O2 -D_THREAD_SAFE -pthread -pipe -std=c99 -D_XOPEN_SOURCE=700 -pedantic -Wall -Wextra -Wsign-conversion -Wconversion -Werror -MT thread.o -MD -MP -MF .deps/thread.Tpo -c -o thread.o thread.c > thread.c:85:15: error: implicit declaration of function '__bitcountl' is invalid in C99 [-Werror,-Wimplicit-function-declaration] > cs = (long) CPU_COUNT(&cpus); > ^ > /usr/include/sys/cpuset.h:63:24: note: expanded from macro 'CPU_COUNT' > #define CPU_COUNT(p) BIT_COUNT(CPU_SETSIZE, p) > ^ > /usr/include/sys/bitset.h:185:14: note: expanded from macro 'BIT_COUNT' > __count += __bitcountl((p)->__bits[__i]); \ > ^ > 1 error generated. > > > > I'm able to use CPU_ISSET to the same effect, but obviously I'd rather use CPU_COUNT. Suggestions? > Show the minimal example demonstrating the issue, specify the version of the system and which you use. My guess is that you did not added #include before including sys/cpuset.h. From owner-freebsd-threads@freebsd.org Thu Sep 24 19:11:31 2015 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCDC6A088F7 for ; Thu, 24 Sep 2015 19:11:31 +0000 (UTC) (envelope-from schulz.j@gmail.com) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F1CF1B9C for ; Thu, 24 Sep 2015 19:11:31 +0000 (UTC) (envelope-from schulz.j@gmail.com) Received: by pacex6 with SMTP id ex6so80891247pac.0 for ; Thu, 24 Sep 2015 12:11:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Ms5H+5Z29G/S1ulGA62nuaLtbKCr+QMqgvUtKk07Z2A=; b=gwThkmwgtPUNC4SllKEYG+CSj+EXeU3Kesvn+QbZ2OzH0X5G6PxhiroElIEjXDnwPr /1sbUAHdYhMFti4vts+FcyYU5byAmAWGJZD1S8ZbjcVTn/sSjl34IEIfvjX5x4LqYcXH d0V+M01PbkCYtGHdlK1VYs5IQz540n+7XnNjLsl7b8Iqgq/BsAi2fsbSEXadWrXpNBhs SvYAlKEWHLHhXPOXRsyo95cgQM/G14Ckm60zrzHsM92wQZdrVt5WiVWNDSlNijJ77f0i D+5qTyfD9LecWydoCX0W53rmYooX6I2vSb1WgPo8Cj7Yl1BsQ2v2SpLm04REI4nNDawN Rogw== X-Received: by 10.66.236.10 with SMTP id uq10mr1532696pac.55.1443121891314; Thu, 24 Sep 2015 12:11:31 -0700 (PDT) Received: from gmail.com (c-73-140-192-75.hsd1.wa.comcast.net. [73.140.192.75]) by smtp.gmail.com with ESMTPSA id xd10sm15046631pab.25.2015.09.24.12.11.29 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 24 Sep 2015 12:11:30 -0700 (PDT) Sender: Jason Schulz Date: Thu, 24 Sep 2015 12:11:21 -0700 From: Jason Schulz To: Konstantin Belousov Cc: freebsd-threads@freebsd.org Subject: Re: pthread_getaffinity_np issue Message-ID: <20150924191121.GG3104@gmail.com> References: <20150924181127.GC3104@gmail.com> <20150924183246.GK67105@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20150924183246.GK67105@kib.kiev.ua> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 19:11:31 -0000 Hey Konstantin, Thanks for the quick response. The version I'm developing with is the latest release. [jason@fb ~/development/git/uxcn/yafd]$ uname -a FreeBSD fb 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 I was able to narrow it down to the following, #include #include #include int main(int argc, char* argv[argc]) { long cs = 0; pthread_t this = pthread_self(); cpuset_t cpus; CPU_ZERO(&cpus); int err = pthread_getaffinity_np(this, sizeof(cpus), &cpus); if (err) return err; cs = (long) CPU_COUNT(&cpus); printf("num cpus... %li\n", cs); return 0; } Compiling with '-D_XOPEN_SOURCE=700', [jason@fb /tmp]$ clang -D_XOPEN_SOURCE=700 -pthread -o num_cpus num_cpus.c num_cpus.c:21:15: warning: implicit declaration of function '__bitcountl' is invalid in C99 [-Wimplicit-function-declaration] cs = (long) CPU_COUNT(&cpus); ^ /usr/include/sys/cpuset.h:63:24: note: expanded from macro 'CPU_COUNT' #define CPU_COUNT(p) BIT_COUNT(CPU_SETSIZE, p) ^ /usr/include/sys/bitset.h:185:14: note: expanded from macro 'BIT_COUNT' __count += __bitcountl((p)->__bits[__i]); \ ^ 1 warning generated. /tmp/num_cpus-560a33.o: In function `main': num_cpus.c:(.text+0xbd): undefined reference to `__bitcountl' clang: error: linker command failed with exit code 1 (use -v to see invocation) However compiling without '-D_XOPEN_SOURCE=700' works, [jason@fb /tmp]$ clang -pthread -o num_cpus num_cpus.c [jason@fb /tmp]$ Should I file a bug report? -J On Thu, Sep 24, 2015 at 09:32:46PM +0300, Konstantin Belousov wrote: > On Thu, Sep 24, 2015 at 11:11:27AM -0700, Jason Schulz wrote: > > I'm having an issue using the non-posix thread affinity methods. Specifically, I'm having trouble using CPU_COUNT. I'm trying to use the following code to get the current number of processors in a processes affinity... > > > > > > > > long cs = 0; > > > > #if defined(HAVE_LINUX) || defined(HAVE_FREEBSD) > > > > pthread_t this = pthread_self(); > > > > cpu_set_t cpus; > > > > CPU_ZERO(&cpus); > > > > int err = pthread_getaffinity_np(this, sizeof(cpus), &cpus); > > > > if (err) > > return err; > > > > cs = (long) CPU_COUNT(&cpus); > > > > > > > > However, I get the compiler error... > > > > > > > > clang -DHAVE_CONFIG_H -I. -g -O2 -D_THREAD_SAFE -pthread -pipe -std=c99 -D_XOPEN_SOURCE=700 -pedantic -Wall -Wextra -Wsign-conversion -Wconversion -Werror -MT thread.o -MD -MP -MF .deps/thread.Tpo -c -o thread.o thread.c > > thread.c:85:15: error: implicit declaration of function '__bitcountl' is invalid in C99 [-Werror,-Wimplicit-function-declaration] > > cs = (long) CPU_COUNT(&cpus); > > ^ > > /usr/include/sys/cpuset.h:63:24: note: expanded from macro 'CPU_COUNT' > > #define CPU_COUNT(p) BIT_COUNT(CPU_SETSIZE, p) > > ^ > > /usr/include/sys/bitset.h:185:14: note: expanded from macro 'BIT_COUNT' > > __count += __bitcountl((p)->__bits[__i]); \ > > ^ > > 1 error generated. > > > > > > > > I'm able to use CPU_ISSET to the same effect, but obviously I'd rather use CPU_COUNT. Suggestions? > > > > Show the minimal example demonstrating the issue, specify the version > of the system and which you use. My guess is that you did not added > #include before including sys/cpuset.h. From owner-freebsd-threads@freebsd.org Thu Sep 24 19:26:33 2015 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF352A072C3 for ; Thu, 24 Sep 2015 19:26:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 669AB14AF; Thu, 24 Sep 2015 19:26:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id t8OJQNvq054573 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 24 Sep 2015 22:26:24 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua t8OJQNvq054573 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id t8OJQNUc054572; Thu, 24 Sep 2015 22:26:23 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 24 Sep 2015 22:26:23 +0300 From: Konstantin Belousov To: Jason Schulz Cc: freebsd-threads@freebsd.org, jhb@freebsd.org Subject: Re: pthread_getaffinity_np issue Message-ID: <20150924192623.GL67105@kib.kiev.ua> References: <20150924181127.GC3104@gmail.com> <20150924183246.GK67105@kib.kiev.ua> <20150924191121.GG3104@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150924191121.GG3104@gmail.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 19:26:33 -0000 On Thu, Sep 24, 2015 at 12:11:21PM -0700, Jason Schulz wrote: > Hey Konstantin, > > Thanks for the quick response. The version I'm developing with is the latest release. > > [jason@fb ~/development/git/uxcn/yafd]$ uname -a > FreeBSD fb 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > > I was able to narrow it down to the following, > > #include > > #include > #include > > int main(int argc, char* argv[argc]) { > > long cs = 0; > > pthread_t this = pthread_self(); > > cpuset_t cpus; > > CPU_ZERO(&cpus); > > int err = pthread_getaffinity_np(this, sizeof(cpus), &cpus); > > if (err) > return err; > > cs = (long) CPU_COUNT(&cpus); > > printf("num cpus... %li\n", cs); > > return 0; > } > > Compiling with '-D_XOPEN_SOURCE=700', > > [jason@fb /tmp]$ clang -D_XOPEN_SOURCE=700 -pthread -o num_cpus num_cpus.c > num_cpus.c:21:15: warning: implicit declaration of function '__bitcountl' is invalid in C99 [-Wimplicit-function-declaration] > cs = (long) CPU_COUNT(&cpus); > ^ > /usr/include/sys/cpuset.h:63:24: note: expanded from macro 'CPU_COUNT' > #define CPU_COUNT(p) BIT_COUNT(CPU_SETSIZE, p) > ^ > /usr/include/sys/bitset.h:185:14: note: expanded from macro 'BIT_COUNT' > __count += __bitcountl((p)->__bits[__i]); \ > ^ > 1 warning generated. > /tmp/num_cpus-560a33.o: In function `main': > num_cpus.c:(.text+0xbd): undefined reference to `__bitcountl' > clang: error: linker command failed with exit code 1 (use -v to see invocation) > > However compiling without '-D_XOPEN_SOURCE=700' works, > > [jason@fb /tmp]$ clang -pthread -o num_cpus num_cpus.c > [jason@fb /tmp]$ > > Should I file a bug report? So your issue only appears when you specify _XOPEN_SOURCE. This is reasonable, because __bitcount stuff in sys/types.h was put under _BSD_VISIBLE protection. I am not sure why and I Cc:ed the author of the change. IMO all symbols from __bitcount* are in the private namespace and could be made unconditionally accessible, to avoid surprises. I put the patch at the end of the message which does this. That said, pthread_np.h is not specified by POSIX/X/Open, so your use of the header and affinity functionality is not covered by _XOPEN_SOURCE contract. The consequences of mixing implementation-specific headers and request for the standard-compliance can be arbitrary. > > -J > > On Thu, Sep 24, 2015 at 09:32:46PM +0300, Konstantin Belousov wrote: > > On Thu, Sep 24, 2015 at 11:11:27AM -0700, Jason Schulz wrote: > > > I'm having an issue using the non-posix thread affinity methods. Specifically, I'm having trouble using CPU_COUNT. I'm trying to use the following code to get the current number of processors in a processes affinity... > > > > > > > > > > > > long cs = 0; > > > > > > #if defined(HAVE_LINUX) || defined(HAVE_FREEBSD) > > > > > > pthread_t this = pthread_self(); > > > > > > cpu_set_t cpus; > > > > > > CPU_ZERO(&cpus); > > > > > > int err = pthread_getaffinity_np(this, sizeof(cpus), &cpus); > > > > > > if (err) > > > return err; > > > > > > cs = (long) CPU_COUNT(&cpus); > > > > > > > > > > > > However, I get the compiler error... > > > > > > > > > > > > clang -DHAVE_CONFIG_H -I. -g -O2 -D_THREAD_SAFE -pthread -pipe -std=c99 -D_XOPEN_SOURCE=700 -pedantic -Wall -Wextra -Wsign-conversion -Wconversion -Werror -MT thread.o -MD -MP -MF .deps/thread.Tpo -c -o thread.o thread.c > > > thread.c:85:15: error: implicit declaration of function '__bitcountl' is invalid in C99 [-Werror,-Wimplicit-function-declaration] > > > cs = (long) CPU_COUNT(&cpus); > > > ^ > > > /usr/include/sys/cpuset.h:63:24: note: expanded from macro 'CPU_COUNT' > > > #define CPU_COUNT(p) BIT_COUNT(CPU_SETSIZE, p) > > > ^ > > > /usr/include/sys/bitset.h:185:14: note: expanded from macro 'BIT_COUNT' > > > __count += __bitcountl((p)->__bits[__i]); \ > > > ^ > > > 1 error generated. > > > > > > > > > > > > I'm able to use CPU_ISSET to the same effect, but obviously I'd rather use CPU_COUNT. Suggestions? > > > > > > > Show the minimal example demonstrating the issue, specify the version > > of the system and which you use. My guess is that you did not added > > #include before including sys/cpuset.h. diff --git a/sys/sys/types.h b/sys/sys/types.h index 4a66a4e..4dfd8ef 100644 --- a/sys/sys/types.h +++ b/sys/sys/types.h @@ -290,9 +290,6 @@ typedef _Bool bool; * The following are all things that really shouldn't exist in this header, * since its purpose is to provide typedefs, not miscellaneous doodads. */ -#if __BSD_VISIBLE - -#include #ifdef __POPCNT__ #define __bitcount64(x) __builtin_popcountll((__uint64_t)(x)) @@ -356,6 +353,10 @@ __bitcount64(__uint64_t _x) #define __bitcount(x) __bitcount32((unsigned int)(x)) #endif +#if __BSD_VISIBLE + +#include + /* * minor() gives a cookie instead of an index since we don't want to * change the meanings of bits 0-15 or waste time and space shifting From owner-freebsd-threads@freebsd.org Thu Sep 24 20:53:06 2015 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73AF5A08A8B for ; Thu, 24 Sep 2015 20:53:06 +0000 (UTC) (envelope-from schulz.j@gmail.com) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4457B12EA; Thu, 24 Sep 2015 20:53:06 +0000 (UTC) (envelope-from schulz.j@gmail.com) Received: by pacex6 with SMTP id ex6so82971671pac.0; Thu, 24 Sep 2015 13:53:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=8bL7Hz+BKt3ptQ44Ed47SpIrvPFw60PeL6mBYuHKMJA=; b=UxUIhUlUf/qf0aCFoCZKAS+XGS1O2BtQZWTWiXADvWmw7Q5jUAswuwEGUH9QUUd6uy hxqDng4lSzFoUYO0y88vWYjZXScEOWkjzHZSWWWcr/Jaw6d1heW2DZ5npmSzPb2Wzvh7 buZxzpEJwP55kQS72akez+yJIt2VxBwzvPOd1m8+WZ5aC65TF8T/FVUBmT4uFoCMz8K5 d7Ad799KdbagwFFnTnUPdCNVwBsYeh2LxGmGEkMKLGbpmerQYJ8cEqIBD04W5mzYUzWo CwLwAiorULoFC1d+dWe5qRZkM5aoltFvAuNVbTkASWNlKH4SIb3ny6S7lO5DyCpCNTqD yu7g== X-Received: by 10.66.221.193 with SMTP id qg1mr2094701pac.103.1443127985421; Thu, 24 Sep 2015 13:53:05 -0700 (PDT) Received: from gmail.com (c-73-140-192-75.hsd1.wa.comcast.net. [73.140.192.75]) by smtp.gmail.com with ESMTPSA id w5sm100936pbs.31.2015.09.24.13.53.03 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 24 Sep 2015 13:53:04 -0700 (PDT) Sender: Jason Schulz Date: Thu, 24 Sep 2015 13:52:55 -0700 From: Jason Schulz To: Konstantin Belousov Cc: freebsd-threads@freebsd.org, jhb@freebsd.org Subject: Re: pthread_getaffinity_np issue Message-ID: <20150924205255.GI3104@gmail.com> References: <20150924181127.GC3104@gmail.com> <20150924183246.GK67105@kib.kiev.ua> <20150924191121.GG3104@gmail.com> <20150924192623.GL67105@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20150924192623.GL67105@kib.kiev.ua> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Sep 2015 20:53:06 -0000 On Thu, Sep 24, 2015 at 10:26:23PM +0300, Konstantin Belousov wrote: > On Thu, Sep 24, 2015 at 12:11:21PM -0700, Jason Schulz wrote: > > Hey Konstantin, > > > > Thanks for the quick response. The version I'm developing with is the latest release. > > > > [jason@fb ~/development/git/uxcn/yafd]$ uname -a > > FreeBSD fb 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > > > > I was able to narrow it down to the following, > > > > #include > > > > #include > > #include > > > > int main(int argc, char* argv[argc]) { > > > > long cs = 0; > > > > pthread_t this = pthread_self(); > > > > cpuset_t cpus; > > > > CPU_ZERO(&cpus); > > > > int err = pthread_getaffinity_np(this, sizeof(cpus), &cpus); > > > > if (err) > > return err; > > > > cs = (long) CPU_COUNT(&cpus); > > > > printf("num cpus... %li\n", cs); > > > > return 0; > > } > > > > Compiling with '-D_XOPEN_SOURCE=700', > > > > [jason@fb /tmp]$ clang -D_XOPEN_SOURCE=700 -pthread -o num_cpus num_cpus.c > > num_cpus.c:21:15: warning: implicit declaration of function '__bitcountl' is invalid in C99 [-Wimplicit-function-declaration] > > cs = (long) CPU_COUNT(&cpus); > > ^ > > /usr/include/sys/cpuset.h:63:24: note: expanded from macro 'CPU_COUNT' > > #define CPU_COUNT(p) BIT_COUNT(CPU_SETSIZE, p) > > ^ > > /usr/include/sys/bitset.h:185:14: note: expanded from macro 'BIT_COUNT' > > __count += __bitcountl((p)->__bits[__i]); \ > > ^ > > 1 warning generated. > > /tmp/num_cpus-560a33.o: In function `main': > > num_cpus.c:(.text+0xbd): undefined reference to `__bitcountl' > > clang: error: linker command failed with exit code 1 (use -v to see invocation) > > > > However compiling without '-D_XOPEN_SOURCE=700' works, > > > > [jason@fb /tmp]$ clang -pthread -o num_cpus num_cpus.c > > [jason@fb /tmp]$ > > > > Should I file a bug report? > > So your issue only appears when you specify _XOPEN_SOURCE. This is reasonable, > because __bitcount stuff in sys/types.h was put under _BSD_VISIBLE protection. > I am not sure why and I Cc:ed the author of the change. > > IMO all symbols from __bitcount* are in the private namespace and could > be made unconditionally accessible, to avoid surprises. I put the patch > at the end of the message which does this. > > That said, pthread_np.h is not specified by POSIX/X/Open, so your use of > the header and affinity functionality is not covered by _XOPEN_SOURCE > contract. The consequences of mixing implementation-specific headers > and request for the standard-compliance can be arbitrary. > > > > > -J > > > > On Thu, Sep 24, 2015 at 09:32:46PM +0300, Konstantin Belousov wrote: > > > On Thu, Sep 24, 2015 at 11:11:27AM -0700, Jason Schulz wrote: > > > > I'm having an issue using the non-posix thread affinity methods. Specifically, I'm having trouble using CPU_COUNT. I'm trying to use the following code to get the current number of processors in a processes affinity... > > > > > > > > > > > > > > > > long cs = 0; > > > > > > > > #if defined(HAVE_LINUX) || defined(HAVE_FREEBSD) > > > > > > > > pthread_t this = pthread_self(); > > > > > > > > cpu_set_t cpus; > > > > > > > > CPU_ZERO(&cpus); > > > > > > > > int err = pthread_getaffinity_np(this, sizeof(cpus), &cpus); > > > > > > > > if (err) > > > > return err; > > > > > > > > cs = (long) CPU_COUNT(&cpus); > > > > > > > > > > > > > > > > However, I get the compiler error... > > > > > > > > > > > > > > > > clang -DHAVE_CONFIG_H -I. -g -O2 -D_THREAD_SAFE -pthread -pipe -std=c99 -D_XOPEN_SOURCE=700 -pedantic -Wall -Wextra -Wsign-conversion -Wconversion -Werror -MT thread.o -MD -MP -MF .deps/thread.Tpo -c -o thread.o thread.c > > > > thread.c:85:15: error: implicit declaration of function '__bitcountl' is invalid in C99 [-Werror,-Wimplicit-function-declaration] > > > > cs = (long) CPU_COUNT(&cpus); > > > > ^ > > > > /usr/include/sys/cpuset.h:63:24: note: expanded from macro 'CPU_COUNT' > > > > #define CPU_COUNT(p) BIT_COUNT(CPU_SETSIZE, p) > > > > ^ > > > > /usr/include/sys/bitset.h:185:14: note: expanded from macro 'BIT_COUNT' > > > > __count += __bitcountl((p)->__bits[__i]); \ > > > > ^ > > > > 1 error generated. > > > > > > > > > > > > > > > > I'm able to use CPU_ISSET to the same effect, but obviously I'd rather use CPU_COUNT. Suggestions? > > > > > > > > > > Show the minimal example demonstrating the issue, specify the version > > > of the system and which you use. My guess is that you did not added > > > #include before including sys/cpuset.h. > > diff --git a/sys/sys/types.h b/sys/sys/types.h > index 4a66a4e..4dfd8ef 100644 > --- a/sys/sys/types.h > +++ b/sys/sys/types.h > @@ -290,9 +290,6 @@ typedef _Bool bool; > * The following are all things that really shouldn't exist in this header, > * since its purpose is to provide typedefs, not miscellaneous doodads. > */ > -#if __BSD_VISIBLE > - > -#include > > #ifdef __POPCNT__ > #define __bitcount64(x) __builtin_popcountll((__uint64_t)(x)) > @@ -356,6 +353,10 @@ __bitcount64(__uint64_t _x) > #define __bitcount(x) __bitcount32((unsigned int)(x)) > #endif > > +#if __BSD_VISIBLE > + > +#include > + > /* > * minor() gives a cookie instead of an index since we don't want to > * change the meanings of bits 0-15 or waste time and space shifting Affinity functionality (cpusets, etc...) definitely seems to be outside the scope of POSIX. sysconf isn't, but my opinion is that it's more considerate to query cpus within the affinity group rather than just the raw number of processors. Honestly I want the software to be able run as many versions of FreeBSD as possible anyway, so I'll probably be using CPU_ISSET, but using CPU_COUNT would probably be a little cleaner though. For simple stuff at least, I'm trying to keep it as platform agnostic as possible. -J From owner-freebsd-threads@freebsd.org Fri Sep 25 16:50:50 2015 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 21FADA089FD for ; Fri, 25 Sep 2015 16:50:50 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0040C11FF for ; Fri, 25 Sep 2015 16:50:50 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 0DD45B923; Fri, 25 Sep 2015 12:50:49 -0400 (EDT) From: John Baldwin To: Konstantin Belousov Cc: Jason Schulz , freebsd-threads@freebsd.org Subject: Re: pthread_getaffinity_np issue Date: Fri, 25 Sep 2015 09:37:55 -0700 Message-ID: <5496698.OIvuy2PFgu@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-PRERELEASE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20150924192623.GL67105@kib.kiev.ua> References: <20150924181127.GC3104@gmail.com> <20150924191121.GG3104@gmail.com> <20150924192623.GL67105@kib.kiev.ua> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 25 Sep 2015 12:50:49 -0400 (EDT) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Sep 2015 16:50:50 -0000 On Thursday, September 24, 2015 10:26:23 PM Konstantin Belousov wrote: > So your issue only appears when you specify _XOPEN_SOURCE. This is reasonable, > because __bitcount stuff in sys/types.h was put under _BSD_VISIBLE protection. > I am not sure why and I Cc:ed the author of the change. > > IMO all symbols from __bitcount* are in the private namespace and could > be made unconditionally accessible, to avoid surprises. I put the patch > at the end of the message which does this. It was not on purpose, more to put it in the section with other "doodads". If it is safe to move out from under BSD_VISIBLE then your patch is fine with me. The only person I can think to ask would be bde@, but my guess is your change is correct. -- John Baldwin