From owner-freebsd-arch@FreeBSD.ORG Tue Apr 28 22:39:33 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9770997F; Tue, 28 Apr 2015 22:39:33 +0000 (UTC) Received: from st11p02mm-asmtp001.mac.com (st11p02mm-asmtpout001.mac.com [17.172.220.236]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6AF69105B; Tue, 28 Apr 2015 22:39:33 +0000 (UTC) Received: from st11p02mm-spool001.mac.com ([17.172.220.246]) by st11p02mm-asmtp001.mac.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTP id <0NNJ000Z8DHMJ260@st11p02mm-asmtp001.mac.com>; Tue, 28 Apr 2015 21:39:25 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68,1.0.33,0.0.0000 definitions=2015-04-28_07:2015-04-28,2015-04-28,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1504280242 MIME-version: 1.0 Received: from localhost ([17.172.220.163]) by st11p02mm-spool001.mac.com (Oracle Communications Messaging Server 7.0.5.33.0 64bit (built Aug 27 2014)) with ESMTP id <0NNJ00FF4DHMBP10@st11p02mm-spool001.mac.com>; Tue, 28 Apr 2015 21:39:22 +0000 (GMT) To: Adrian Chadd Cc: "freebsd-arch@freebsd.org" From: Rui Paulo Subject: Re: RFT: numa policy branch Date: Tue, 28 Apr 2015 21:39:22 +0000 (GMT) X-Mailer: iCloud MailClient15B.8196069 MailServer15B.18830 X-Originating-IP: [12.218.212.178] Message-id: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2015 22:39:33 -0000 On Apr 26, 2015, at 01:30 PM, Adrian Chadd wrote:=0A=0A= Hi!=0A=0AAnother update:=0A=0A* updated to recent -HEAD;=0A* numactl now c= an set memory policy and cpuset domain information - so=0Ait's easy to say= "this runs in memory domain X and cpu domain Y" in=0Aone pass with it;=0A= =C2=A0=0AThat works, but --mempolicy=3Dfirst-touch should ignore the --mem= domain argument (or print an error) if it's present.=0A=0A* the locality m= atrix is now available. Here's an example from scott's=0A2x haswell v3, wi= th cluster-on-die enabled:=0A=0Avm.phys_locality:=0A0: 10 21 31 31=0A1: 21= 10 31 31=0A2: 31 31 10 21=0A3: 31 31 21 10=0A=0AAnd on the westmere-ex bo= x, with no SLIT table:=0A=0Avm.phys_locality:=0A0: -1 -1 -1 -1=0A1: -1 -1 = -1 -1=0A2: -1 -1 -1 -1=0A3: -1 -1 -1 -1=0A=C2=A0=0AThis worked for us on I= vyBridge a SLIT table.=0A=0A* I've tested in on westmere-ex (4x socket), s= andybridge, ivybridge,=0Ahaswell v3 and haswell v3 cluster on die.=0A* I'v= e discovered that our implementation of libgomp (from gcc-4.2) is=0Avery o= ld and doesn't include some of the thread control environment=0Avariables,= grr.=0A* .. and that the gcc libgomp code doesn't at all have freebsd thr= ead=0Aaffinity routines, so I added them to gcc-4.8.=0A=C2=A0=0AI used gcc= 4.9=0A=0AI'd appreciate any reviews / testing people are able to provide.= I'm=0Aabout at the functionality point where I'd like to submit it for=0A= formal review and try to land it in -HEAD.=0A=C2=A0=0AThere's a bug in the= default sysctl policy. =C2=A0You're calling strcat on an uninitialised st= ring, so it produces garbage output. =C2=A0We also hit the a panic when ou= r application starts allocation many GBs of memory. =C2=A0In this case, th= e memory is split between two sockets and I think it's crashing like you d= escribed on IRC.=0A=0A=0A=