From owner-freebsd-arch@FreeBSD.ORG Fri Feb 14 22:57:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 472FFBC9; Fri, 14 Feb 2014 22:57:21 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CF900105A; Fri, 14 Feb 2014 22:57:20 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id w7so21352854qcr.0 for ; Fri, 14 Feb 2014 14:57:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=dyaQZ1O5yEnbLCb25tmt99o5o9IQoLAU+LgPmywsIvQ=; b=BUmkYv3u0Q9Xt+0CwwVZ1zjlj7/Z5XHysaSZabgZ4ieowgLRp2MlZWasAQ+0t8qGmJ +gKztJK/q3xEUaZieEQubZ43El//eQCcOgIR+ho+24CMWQgaxSpIgnN8v4fGBjWexrKR mZDsutgmTcp3vpyAJ9+/oSben9byJgSVJr7v/VLZG+xkVdunxtfSqWfoQ77mUV8IzNfz 67g4stqyIrqyv6PXnl6qFcfcr5VVVecby9FfbAXUEsL0I2wEd8GJiEUMQAwCc1FZhRk9 ZFjobHyNK7jE2o6z79juyAGQSIt2NqHJ+V/Ki0XIJUdD+2BKva2Gmz3Qp4HzOUGUv3/U 3IoQ== MIME-Version: 1.0 X-Received: by 10.224.61.2 with SMTP id r2mr18094159qah.49.1392418640030; Fri, 14 Feb 2014 14:57:20 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Fri, 14 Feb 2014 14:57:19 -0800 (PST) In-Reply-To: References: <201402141139.49158.jhb@freebsd.org> <201402141318.44743.jhb@freebsd.org> Date: Fri, 14 Feb 2014 14:57:19 -0800 X-Google-Sender-Auth: vlg3IRlPpAsLpOvGMj4V9O8uvdc Message-ID: Subject: Re: can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code? From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 22:57:21 -0000 [snip] So it turns out that the threads somehow migrating between CPUs during flowtable_lookup_common() is the clock swi(s), which I'm guessing are driving the per-CPU TCP callwheel timeouts. It turns out the per-CPU clock swis aren't CPU-pinned, so they can be preempted and migrated. I'm not sure if this is correct behaviour. I'll experiment with pinning these to their base CPU and see if this causes issues. Thanks, -a