From owner-freebsd-net@freebsd.org Fri Sep 11 09:46:30 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A72EDA0166F for ; Fri, 11 Sep 2015 09:46:30 +0000 (UTC) (envelope-from trtrmitya@gmail.com) Received: from mail-lb0-x234.google.com (mail-lb0-x234.google.com [IPv6:2a00:1450:4010:c04::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E51D102A for ; Fri, 11 Sep 2015 09:46:30 +0000 (UTC) (envelope-from trtrmitya@gmail.com) Received: by lbcjc2 with SMTP id jc2so36719365lbc.0 for ; Fri, 11 Sep 2015 02:46:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=I8LK/wOa9uM0e/1TwA3EC3ei96GuMOqwAl9ryoxxAyU=; b=P2naPShfREErYXwWTSgdm1Vt+wMMOe3ebaluQIc04A1K0OfPOH3TjXjqVOfdk2ioCE VnqKoPLaxyYDwD5dx/+bkQvR8IDKwzjxjJC5gNP5lZOEUbcydAH71P1Qi9rjJtLJbMkX A4S+VFw4fOwI0RysmjuetslKnRREug2kRLVXkKwSis5VoeJ/drUhteY7bzS1oekYAJ1t UaFblj5HhO6w5iztdARlSIYC2sU8Gq5pNgnBqt4h1By2vCrZcfsz9sDotkyo6GxeTwhi DwVcNHQwzNuuTNjzhNP2tATx4QHgw+Evq19fs/EotWam9Oj5rbiVnQsHvFj9pvNfRWaY ehOQ== X-Received: by 10.152.29.167 with SMTP id l7mr40839883lah.57.1441964788118; Fri, 11 Sep 2015 02:46:28 -0700 (PDT) Received: from ?IPv6:2a02:6b8::408:6468:ef0e:8c24:602b? ([2a02:6b8:0:408:6468:ef0e:8c24:602b]) by smtp.gmail.com with ESMTPSA id po6sm113180lbc.33.2015.09.11.02.46.27 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Sep 2015 02:46:27 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: netisr process eats 100% cpu From: Dmitry Sivachenko In-Reply-To: Date: Fri, 11 Sep 2015 12:46:25 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: FreeBSD Net , hiren@strugglingcoder.info X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Sep 2015 09:46:30 -0000 > hiren panchasara wrote: >=20 > Unsure at the moment if loopback is causing the trouble for you or = not. > See: (please keep me CC'ed, I am not subscribed to -net) > > = https://lists.freebsd.org/pipermail/freebsd-net/2015-February/041239.html > > Yes, this thread looks similar. > You may want to try: > 1) pmcstat and see if you can catch something What particular should I look for? Here is first lines of pmcstat -T -S = instructions -w 1: PMC: [INSTR_RETIRED_ANY] Samples: 157198 (100.0%) , 0 unresolved %SAMP IMAGE FUNCTION CALLERS 13.2 kernel cpu_search_highest cpu_search_highest:12.0 = sched_idletd:1.2 8.3 kernel ipfw_chk ipfw_check_packet 3.1 myprogram memsetAVX _ZN12TLz4Compress7DoWriteEPKv 2.3 kernel tcp_output tcp_usr_send:1.0 = tcp_do_segment:0.9 > 2) disable checksum on localhost I tried, but nothing has changed. > 3) look at netisr settings. sysctl net.isr o/p and how it looks under > netstat -Q. I am not sure if adding more threads to netisr via What should I look for? > net.isr.numthreads would help. (Note its a loader.conf variable) This netisr load looks parasitical to me (as I noted, moving haproxy to = a separate machine does not burn CPU cycles on netisr, why is localhost = special?) Even if adding more threads to netisr would boost network utilization, = wouldn't those CPU cycles spent for netisr just be a waste of energy? I = have other tasks for these CPU. > > As I said, I am not sure whats up. Just throwing some ideas here. Thanks!=