From owner-freebsd-net@FreeBSD.ORG Mon Jul 28 20:43:42 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85707E24 for ; Mon, 28 Jul 2014 20:43:42 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 405A320B5 for ; Mon, 28 Jul 2014 20:43:42 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id k15so8772797qaq.27 for ; Mon, 28 Jul 2014 13:43:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=htgdwhB6eTt6wa1XoZa556+gsj1qIiojsGi9feVAYoc=; b=BzGjWM1gjD+iHJnB1IfE/cv8zqaTJ/8V17rExt84iKh8I/130ymiSS4t/tbcSTtYj8 Koy4WYrwFQUsYnho/D02AWVgcY4VzxlOptblHcK9IPvKSvFFQ5ZjqdQOnGlGwnZDaYmv 6C2KIB2zF75DQ+AhhKj/73BEvVjEoIbZh/GAbtqC/zmTQXP8rr7O5IWCmUQx3nd0/ePE nGDqxxctqvjJXyFQrx/pyPhsRjuH3cduST2pQnOagYRVSir/jvudTDryYbttHe0dlB9b U8Q4SykBCm47e/u+uoAMsSYMfQgc6HTozIbWf5SMnd7oNdM8S6DNSbUB25ikYDWtPl2w r0Ew== MIME-Version: 1.0 X-Received: by 10.140.38.169 with SMTP id t38mr63787378qgt.3.1406580221322; Mon, 28 Jul 2014 13:43:41 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.1.6 with HTTP; Mon, 28 Jul 2014 13:43:41 -0700 (PDT) In-Reply-To: References: <53CE80DD.9090109@gmail.com> <53CEB090.7030701@gmail.com> <53CEB670.9060600@gmail.com> <53CEB9B5.7020609@gmail.com> <83597B15-63B3-4AD7-A458-00B67C9E5396@neville-neil.com> Date: Mon, 28 Jul 2014 13:43:41 -0700 X-Google-Sender-Auth: M4BJJUBZ-7hCftemg4Zz2rK5nGA Message-ID: Subject: Re: fastforward/routing: a 3 million packet-per-second system? From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , John Jasen , Navdeep Parhar X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2014 20:43:42 -0000 On 28 July 2014 13:37, Ryan Stone wrote: > On Sun, Jul 27, 2014 at 4:42 PM, George Neville-Neil > wrote: >> Chiming in late, but don't you mean instruction-retired instead of >> CPU_CLK_UNHALTED_CORE? >> >> Best, >> George > > In my experience instruction-retired gives very misleading profiler > output in most cases. The problem is that instruction-retired gives > equal weight to all instructions, which means that it does not take > into account instructions with long latencies because they (for > example) missed the cache. CPU_CLK_UNHALTED_CORE (or its alias, > unhalted-cycles) is a much better event because it is a nearer proxy > for time-based sampling, which is really what you're interested in > when trying to reduce runtime of processes. Right. It is a union of all the things that screw with you - frontend stall, backend/retire stall, microcode operation stall, FPU length stall, branch misprediction stalls, L3 miss (ie, memory) stall, cache ping-ponging stalls. Figuring out -which- of those above are the problem requires a little further digging. > My one big complaint with unhalted-cycles is that it does not take > into effect CPU time spent in busy-wait loops that use the pause > instruction, so it vastly unweights time spent adaptively spinning on > kernel mutexes, for instance. Well, it depends if you want to know about the places that it's spending in busy-wait loops using PAUSE or not. (Are there any flags / modifiers that have the CPU not count that?) > I'm also not sure what it does when the > CPU is adjusting its frequency, but that's not a case that I ever have > to deal with personally. That's the difference between _CORE and _REF. -a