From owner-svn-src-all@freebsd.org Wed Oct 12 14:46:01 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06FC0C0F2AD; Wed, 12 Oct 2016 14:46:01 +0000 (UTC) (envelope-from jonlooney@gmail.com) Received: from mail-qt0-f178.google.com (mail-qt0-f178.google.com [209.85.216.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C1F5ECEC; Wed, 12 Oct 2016 14:46:00 +0000 (UTC) (envelope-from jonlooney@gmail.com) Received: by mail-qt0-f178.google.com with SMTP id q7so18108693qtq.1; Wed, 12 Oct 2016 07:46:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=OY/35HCrP0/bGZwm5v3OtK0BvJsvrvLla9zhJu8QLc8=; b=dtttkcI2Ra3Uq2AeO/+SeVv03qJ8/W+9JDNfjtqPo4v9Bqo/AP+0Z7UnZAisHJFxx/ QwJFL6Ir9IfLXbHuJ7apDZKMRsESH4ysViz7gxs0cTwfQvK0jj06A9AXsWKqbffxaZp1 O2sfS317zGtK88GZuCd/xotIlqodZcBXSqifKu/aHzDLpgVAX2YM9pIpCdpv00TJZUHm o7zlmzVx3mOJCny29xe7cv1/BHj1b2WiXMrVR7mQdY54C++ixRcYI1S5Fo8lial9gJ3W CoW7NRx30RsPbzHLB/t2geZftd0xxUGfmzUm3BFO2Lo9oPsKuLEoOIzTXekgXJPECpO+ I0Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=OY/35HCrP0/bGZwm5v3OtK0BvJsvrvLla9zhJu8QLc8=; b=hUK5BCRtBbLZYvNiyRW6yFhtBgYr5jsJeoMqvUlZXvKW/5QCjaNp3P93eIXTcOHLtr O4WP5C3dMnSOAW4QY4Tr9Q0PCOqbftpC1CwaYyLjQ7IjM/umDXYD984mgRyTtMVAkBfS fLEWRE463uloiZ4r3rWBMk+hLOp6AO4ZJV0MW+yOf+fBEJxmnmPaXqdJ7JcGxRZ6pLUc SRyjbi9P9+LTFjyP3gcfcisa3jJCnbBbcTGxljq5X2PB3secmkCMFFjNk5MQ50H4ocsk 4Lkc1hamv/I3sQaap66RsLwkP55S4L3wBMeb3bsC2YwRZJoEd2Hx3uQADItFMLO3FSoE gdpg== X-Gm-Message-State: AA6/9RmCQCh3KKmPXmIf6c2s5DIB75MdHJggwH9oiH2sC4CaX6KF8EkSuC9Cu7JIxr39dKOICWTbQpN3oVIPKg== X-Received: by 10.237.41.39 with SMTP id s36mr1524139qtd.156.1476283493414; Wed, 12 Oct 2016 07:44:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.200.53.208 with HTTP; Wed, 12 Oct 2016 07:44:52 -0700 (PDT) In-Reply-To: <20161012094403.GA57876@zxy.spb.ru> References: <201610120216.u9C2Gga8041814@repo.freebsd.org> <20161012094403.GA57876@zxy.spb.ru> From: Jonathan Looney Date: Wed, 12 Oct 2016 10:44:52 -0400 Message-ID: Subject: Re: svn commit: r307082 - in head: . sys/amd64/conf sys/arm/conf sys/arm64/conf sys/conf sys/i386/conf sys/mips/conf sys/modules/cc sys/modules/khelp sys/netinet sys/netinet/tcp_stacks sys/pc98/conf sy... To: Slawa Olhovchenkov Cc: "Jonathan T. Looney" , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 14:46:01 -0000 On Wed, Oct 12, 2016 at 5:44 AM, Slawa Olhovchenkov wrote: > Do you perform estimate of performane impact of this overhead? > > Somewhat, but not precisely. It will depend on (at least) a few factors: a. The hardware * How fast is your CPU? Your L1/L2/L3 cache? * How many CPUs? * How smart is the hardware prefetching? * How fast is the memory bus? * etc. b. Whether VNET is enabled * If VNET is enabled, the hook lookup will require more instructions to find the correct list c. Which TCP stack you are using * The hook lookup is implemented as an external function in the non-default stacks, so that requires the overhead of a function call d. The probability that the number of hooks is in your CPU cache * This is a combination of the hardware and the exact workload on the device On a test system running a fairly heavy load of TCP traffic using the default TCP stack, without VNET, and without any TCP hooks installed, the input/output hhook functions accounted for approximately .075% of the "busy" CPU cycles during one of my measurements. That certainly is not a large number, but it is large enough to be measurable. And, I think the hardware I used for the measurement is tuned for high-performance, so this may be close to the "best case" measurement. I suspect that systems with a large number of short-lived sessions will see a larger improvement from the deletion of khelp_init_osd() from the session setup path. (I didn't measure this.) I also suspect that systems with VNET will see a larger improvement. I also suspect that systems with slower memory busses, smaller caches, etc. will see a larger improvement. But, this is very hard to measure precisely in a generic sense. The one concrete thing we *can* measure is that the functions add some number of instructions to the session setup, session teardown, input, and output code paths. If you need those instructions to achieve the desired functionality (in this case, probably a congestion-control algorithm), then you probably want those instructions. If not, then you may want to delete them. This was the impetus to add a kernel option to give you the ability to decide whether you want to include the functionality. I hope this answer helps explain some more about the change. Jonathan