From owner-freebsd-arch@FreeBSD.ORG Wed Oct 8 01:37:04 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 433CBA84 for ; Wed, 8 Oct 2014 01:37:04 +0000 (UTC) Received: from mail-ig0-x22e.google.com (mail-ig0-x22e.google.com [IPv6:2607:f8b0:4001:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 16D17EAE for ; Wed, 8 Oct 2014 01:37:04 +0000 (UTC) Received: by mail-ig0-f174.google.com with SMTP id a13so233987igq.7 for ; Tue, 07 Oct 2014 18:37:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=d7rVKvqgIZ4ECIvpzNd75YgpffPI8Sd0Dr1iBJwN+PA=; b=pQTbh/HtPZIqjoqtsbFcZX2xda/Y/0EiW+FKDYM1nSMXwO13OcoBs6CwpY908y3If9 WqbVmJYmYZtykIl4Lri8A1JbOe0dalE9nuZ+wy2BSsBHT8qKnWsQDhfm/A52h1cxpurr ZWCg5mAnF47bHl+dETGjIZvxOLzsmFay+vcRh0kwzhNZPyz1tKb2sx6bCFyf7JiZj4Eq KGOM04QdAzkooW0Ta66IS+6HKu/9HsyjMLxdskK3kErv6HHzrJG6h9demI+2rVcUbAGn LVhYrq1JFgsSb1p5RZ4J++7vnh2KhbkoHsD2S3j6PqEUEfRofbiuHao0Edax+BJ8wDUp V4kg== MIME-Version: 1.0 X-Received: by 10.50.72.3 with SMTP id z3mr11437457igu.36.1412732223433; Tue, 07 Oct 2014 18:37:03 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.50.78.4 with HTTP; Tue, 7 Oct 2014 18:37:03 -0700 (PDT) Date: Tue, 7 Oct 2014 18:37:03 -0700 X-Google-Sender-Auth: bQmrTdQAkr3ExcwJ6310eOfFDMc Message-ID: Subject: [rfc] enumerating device / bus domain information From: Adrian Chadd To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 01:37:04 -0000 Hi, Right now we're not enumerating any NUMA domain information about devices. The more recent intel NUMA stuff has some extra affinity information for devices that (eventually) will allow us to bind kernel/user threads and/or memory allocation to devices to keep access local. There's a penalty for DMAing in/out of remote memory, so we'll want to figure out what counts as "Local" for memory allocation and perhaps constrain the CPU set that worker threads for a device run on. This patch adds a few things: * it adds a bus_if.m method for fetching the VM domain ID of a given device; or ENOENT if it's not in a VM domain; * it adds some hooks to print the numa-domain out of a device if it exists; * it adds hooks in srat.c to store the original proximity-id values and uses them to map PXM to FreeBSD VM domain IDs; * the ACPI code now has bus methods to enumerate which PXM (and thus which VM domain) a device is on. The review for it is here: https://reviews.freebsd.org/D906 Please ignore the vm_phys.c patch; it's purely for experimenting on my side and won't be committed part of this work. Thanks, -a