From owner-freebsd-hackers@FreeBSD.ORG Wed May 2 21:16:26 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5E581106566C for ; Wed, 2 May 2012 21:16:26 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 306788FC0A for ; Wed, 2 May 2012 21:16:26 +0000 (UTC) Received: by pbbro2 with SMTP id ro2so1781155pbb.13 for ; Wed, 02 May 2012 14:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=K+uFOpqsI0LvX0cbH1m3EDXOh6G6IGdR56yK9Y76zv4=; b=QMgl3ujOYRq+OGMZy9HL0Pi+THdXU+WnDWvUuZb9DEov09YZYGY6F7YNGhsePpfZp2 66CowE6QL8djARmIgwf5ECFYrmIk/JV1MT1ZObPUan/gfV0eFsRcBv8sivMXJEtahi6X WNOsk+YPz+WgfgWy+9hMQWB7sxXrKT3XJG7my1EjF07Bz5L2wm8O3t9KVchzzByDEyby 40d7WxCy2SMZ8YsImHeXqeJ4BvdMeM6M6KE5tw8tYG4nA9zKQWwTnw+CqhaqsLWJ9LCT yW5LEpsrrguvfFOTrrCxOt7GRBgtawh/DGbVjETYDq+sNhsgEu5iWilR+f8NZW1eli6r dbGQ== Received: by 10.68.238.38 with SMTP id vh6mr1048040pbc.50.1335993380536; Wed, 02 May 2012 14:16:20 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPS id y5sm3002471pbk.5.2012.05.02.14.16.18 (version=SSLv3 cipher=OTHER); Wed, 02 May 2012 14:16:18 -0700 (PDT) Sender: Navdeep Parhar Message-ID: <4FA1A421.6020601@FreeBSD.org> Date: Wed, 02 May 2012 14:16:17 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:12.0) Gecko/20120502 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: CPU selection for ithreads on 8.3 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 May 2012 21:16:26 -0000 There seems to be a regression in 8.3 in the way the kernel selects CPUs for interrupts. For example, cxgb(4) on 8.3 ends up with all its ithreads on the same CPU (CPU7 in this case). 12 root -68 - 0K 816K WAIT 7 0:55 0.00% intr{irq279: cxgbc0} 12 root -68 - 0K 816K WAIT 7 0:52 0.00% intr{irq275: cxgbc0} 12 root -68 - 0K 816K WAIT 7 0:47 0.00% intr{irq278: cxgbc0} 12 root -68 - 0K 816K WAIT 7 0:43 0.00% intr{irq277: cxgbc0} 12 root -68 - 0K 816K WAIT 7 0:43 0.00% intr{irq282: cxgbc0} 12 root -68 - 0K 816K WAIT 7 0:41 0.00% intr{irq281: cxgbc0} 12 root -68 - 0K 816K WAIT 7 0:32 0.00% intr{irq276: cxgbc0} 12 root -68 - 0K 816K WAIT 7 0:31 0.00% intr{irq280: cxgbc0} Back in the day there used to be code in cxgb to bind different interrupts to different CPUs but it was removed because the kernel distributed them across CPUs anyway. So what changed? This appears 8.3 specific. I don't see it on head and I don't have a 9 system readily available right now. Regards, Navdeep