From owner-freebsd-arch@FreeBSD.ORG Thu Feb 19 17:16:06 2015 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96C9726D for ; Thu, 19 Feb 2015 17:16:06 +0000 (UTC) Received: from mail-pd0-f172.google.com (mail-pd0-f172.google.com [209.85.192.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 66BDA204 for ; Thu, 19 Feb 2015 17:16:06 +0000 (UTC) Received: by pdbnh10 with SMTP id nh10so1016962pdb.11 for ; Thu, 19 Feb 2015 09:16:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=OZEuXycPx4ol2P4iUK6ltTjEcMQOn/nYcntTCEmHXKQ=; b=KvyPU7EfdmZ70iYF2Ok9R2P98xmADR9l7FwM6O6Rxyc5T62zDDIQmkzFyHwtPjebB1 IXP2BuPCbJoJWFSdo9rCoRcXAG4Tl4gJbx3kiJSIQbOdQlgFO2f2aPE1uaVvGw7CAkXp uQT8zF2hlp5vC+tPw2LDB57LvAdGZfKn4rGgJ+gOfQXzMVEJo89jfwlaohl6uAYOOdUA XYrTVnXvnmy0nSBGmf6HpvnX1wjtbGrB3oU18obiqJPgGhO+rGMmNgTU/N6H5HeNAyeb yv+FgUvX6JIhqyZUlpUAKBUPL4y2mUrtYvu7taPP9Rs+2u2s9tkE3mL6HfwTGW4LQUJC nUkw== X-Gm-Message-State: ALoCoQkhFpXCSCjqwJol55i+9K5etQnO+oSbB9sOte856F+binIzcVZr/cLf6bm6DV4QdkaotrRT X-Received: by 10.68.68.235 with SMTP id z11mr9370110pbt.77.1424366159772; Thu, 19 Feb 2015 09:15:59 -0800 (PST) Received: from lgmac-jlieser.corp.netflix.com ([69.53.236.236]) by mx.google.com with ESMTPSA id sq6sm24269562pbc.40.2015.02.19.09.15.58 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 19 Feb 2015 09:15:58 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: RFC: bus_get_cpus(9) From: Warner Losh In-Reply-To: <2650364.MV3AvSBuVe@ralph.baldwin.cx> Date: Thu, 19 Feb 2015 10:15:56 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1848011.eGOHhpCEMm@ralph.baldwin.cx> <6147240.5Rne9DUXyM@ralph.baldwin.cx> <2650364.MV3AvSBuVe@ralph.baldwin.cx> To: John Baldwin X-Mailer: Apple Mail (2.2070.6) Cc: "freebsd-arch@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Feb 2015 17:16:06 -0000 > On Feb 19, 2015, at 10:03 AM, John Baldwin wrote: >=20 > On Thursday, February 19, 2015 08:28:34 AM Adrian Chadd wrote: >> On 19 February 2015 at 07:37, John Baldwin wrote: >>> There's nothing preventing the RSS code from calling bus_get_cpus() >>> internally to populate the info it returns in its APIs. >>>=20 >>> That is, I imagine something like: >>>=20 >>> #ifdef RSS >>>=20 >>> queue_info =3D fetch_rss_info(dev); >>> for (queue in queue_info) { >>>=20 >>> create queue for CPU queue->cpu >>>=20 >>> } >>>=20 >>> #else >>>=20 >>> /* Use bus_get_cpus directly and do 1:1 */ >>>=20 >>> #endif >>>=20 >>> That is, I think RSS should provide a layer on top of new-bus, not = be a >>> bus_foo API. At some point all drivers might only have the #ifdef = RSS >>> case >>> and not use bus_get_cpus() directly at all, but it doesn't seem like = the >>> RSS API is quite there yet. >>=20 >> I wasn't suggesting we have RSS as a newbus method, just that drivers >> don't necessarily need to call the bus method and iterate themselves. >>=20 >> I was suggesting that we do what i've done for rss, but as a generic >> "how should devices create queues and map them to cpusets / interrupt >> locality" and that calls the bus method(s) to discover topology and >> query local-interrupt and local-memory sets to do things >> appropriately. >>=20 >> Then RSS is just a flavour of that API call - network drivers could >> either be RSS aware and call it to get the mapping, or call some >> higher level bus API call to do the "generic" hints or whatever based >> mapping. >=20 > Can you provide a sample API (function prototype, etc.)? Aside from = RSS=20 > (which will have its own API for other reasons), I don't know of = another use=20 > case that is well understood enough to let us build an abstraction on = yet (we=20 > all know the perils of abstracting from one use case), so I'm hesitant = to go=20 > much further than "these are the best place to do interrupts=E2=80=9D. Newer LSI cards could benefit from that, but the rest of the storage = stack may need tweaks to allow for true multi-queue implementations. = Interrupts would be part of that. Warner=