From owner-freebsd-net@FreeBSD.ORG Thu Apr 23 23:12:30 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58880904; Thu, 23 Apr 2015 23:12:30 +0000 (UTC) Received: from forward8l.mail.yandex.net (forward8l.mail.yandex.net [IPv6:2a02:6b8:0:1819::8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Certum Level IV CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1709F1156; Thu, 23 Apr 2015 23:12:29 +0000 (UTC) Received: from smtp3h.mail.yandex.net (smtp3h.mail.yandex.net [84.201.186.20]) by forward8l.mail.yandex.net (Yandex) with ESMTP id 07C801A4024C; Fri, 24 Apr 2015 02:12:17 +0300 (MSK) Received: from smtp3h.mail.yandex.net (localhost [127.0.0.1]) by smtp3h.mail.yandex.net (Yandex) with ESMTP id EACB21B43393; Fri, 24 Apr 2015 02:12:16 +0300 (MSK) Received: from unknown (unknown [2a02:6b8:0:6::a8]) by smtp3h.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id KQkKyMmqXC-CGjWa3iQ; Fri, 24 Apr 2015 02:12:16 +0300 (using TLSv1.2 with cipher AES128-SHA (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1429830736; bh=CAR1M/iJNFn3hi7ngdx8C+R+/5ezkcGZvw2Pbsc2+wg=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=bCfam4hhSm0YGZSWl5vkcopep6+suELJrbjhvuixEIQfcuS1oFbNET+nElrk2jDmf /S2uY7WI+WQnkBKe4c95Jh3Pm85+urtYEatI1ndPxQGsk4316ZXaAtYxSz5erFgNYJ acTHLX2F3sdpDGbZSrZHnWQg5awO5kw8+WTOYPLs= Authentication-Results: smtp3h.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <55397BE1.7090403@yandex.ru> Date: Fri, 24 Apr 2015 02:10:25 +0300 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Sydney Meyer , freebsd-net@freebsd.org CC: "Robert N. M. Watson" , George Neville-Neil Subject: Re: IPSec Performance under Xen References: In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Apr 2015 23:12:30 -0000 On 24.04.2015 01:00, Sydney Meyer wrote: > Hello, > > I have set up 2 VM's under Xen running each one IPSec-Endpoint. > Everything seems to work fine, but (measured with benchmarks/iperf) > the performance drops from ~10 Gb/s on a non-IPSec-Kernel to ~200 > Mb/s with IPSec compiled in, regardless of whether actually using > IPSec or not. > > I have read about reasoning why IPSec isn't enabled in GENERIC, but > wanted to ask if this is the kind of performance hit one has to > expect. Hi, I have a guess. Since you use iperf, I think the main bottleneck there is the fact, that the socket has a PCB. When you have compiled IPSEC in the kernel, it enables the code, that does initialization of PCB's security policy inp_sp via ipsec_init_policy(). Then each packet that has associated PCB (iperf uses sockets, so it has PCB) on output goes through a bunch of checks, that includes several lookups with taking of exclusive locks. Even if you don't use any security policies ALL packets that have associated PCB will go through such UNNEEDED checks. I am not very familiar with this code, but maybe George or Robert can answer why we do this for every PCB? Why not initialize inp_sp only when application does needed configuration via setsockopt(IP_IPSEC_POLICY)? -- WBR, Andrey V. Elsukov