From owner-freebsd-virtualization@FreeBSD.ORG Mon Dec 15 15:17:27 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 285B91065672 for ; Mon, 15 Dec 2008 15:17:27 +0000 (UTC) (envelope-from jason.fines@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.247]) by mx1.freebsd.org (Postfix) with ESMTP id BE3898FC0C for ; Mon, 15 Dec 2008 15:17:26 +0000 (UTC) (envelope-from jason.fines@gmail.com) Received: by an-out-0708.google.com with SMTP id c2so923701anc.13 for ; Mon, 15 Dec 2008 07:17:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type:references; bh=H0LtWARHOlCcYr/KwVLan7xdh4wrAgmUluZDL5Ev+BE=; b=lr+LYEhJEysuSm8yR5tr2wI+HnM9PQa3eETxrkgVZBPgDmhMk2e1vkmCbuB5oRqUOd CYfaCVBePSqdc4AdpVHEjMrfS0lGo4416iK03K1jo5H30EasfEKPlm8m3/n4yM9Te4f1 +/RM6wMzGux29Bv7BIxVIRbQviWISB3A36XIg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:references; b=hl9FMQNjH4GW/ElCUMi94rA/zy+6K2w1W2HXvqkxheyciqhdfCxP5nrhMQoW5qdmeU YzjIoRAUVSsuqk9Jsuwi1Jvzzvo2hZA0qkHEcMzVTLduyLAtCoMQ6tcCMnnYrUW7DQ3O P7v2TGx9BgD67TR0MHd5GdlAVVrTExUDhW0+U= Received: by 10.100.172.17 with SMTP id u17mr4542338ane.105.1229354245496; Mon, 15 Dec 2008 07:17:25 -0800 (PST) Received: by 10.100.9.13 with HTTP; Mon, 15 Dec 2008 07:17:25 -0800 (PST) Message-ID: <5e6025b70812150717h500ab3c4tb1319dee1572f711@mail.gmail.com> Date: Mon, 15 Dec 2008 10:17:25 -0500 From: "Jason Fines" To: "Jorge Sanchez" In-Reply-To: MIME-Version: 1.0 References: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-virtualization@freebsd.org Subject: Re: Question About TCP Reassembly Inside VImages X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Dec 2008 15:17:27 -0000 Hi Jorge, Sorry about the delay in my response. I have also been able to set maxqlen, but have been unsuccessful in setting maxsegments. I just recently tried upgrading to 7_RELENG with the vimage package http://imunes.tel.fer.hr/virtnet/vimage_7-20081015.tgz however I am still experiencing the same problems. I even went so far as to set the variable 'V_tcp_reass_maxseg' explicitly to 4096 on lines 111, and 122 of tcp_reass.c in the kernel source code, however, the maxsegments sysctl variable is still zero in my vimages!!! Have you tried anything else? I am also prohibited from using this amazing technology because of this issue. Does anyone else have any information about this? Marko, would upgrading FreeBSD 8.0 make any difference? Any help would be greatly appreciated. Thanks, Jason On Thu, Nov 27, 2008 at 11:57 AM, Jorge Sanchez wrote: > > > > > Hola Jason, > > I also observed a similar phenomenon on my system's vimages. I have several > thousands dropped packets due to "insufficient memory" (the counter you > mention in netstat -m for me is also incremented in the > net.inet.tcp.reass.overflows read-only sysctl variable) and I routinely have > TCP connections dropped within vimages because of it. I think that the TCP > packet reassembly queue length is essentially zero once options VIMAGE is > enabled... which would explain my problems when trying to contact hosts that > are on flaky links or are situated very far away hop-wise. > > I think there is something very wrong with the TCP reassembly when options > VIMAGE is enabled. Did you try increasing the net.inet.reass.maxqlen or > net.inet.reass.maxsegments sysctls? I was able to increase maxqlen but > maxsegments must be set in loader.conf and this value is not inherited into > any vimages I create after booting! :-( > > If you come up with a fix, I would appreciate it too since this prevents me > from performing realistic TCP testing within the virtualization framework. > > Adios! > Jorge > > > From: jason.fines@gmail.com > To: > freebsd-virtualization@freebsd.org > Subject: Re: Question About TCP Reassembly Inside VImages > Date: Sat, 22 Nov 2008 08:52:16 -0500 > Thanks Julian, > > I suspect you are correct as nmbclusters is a system wide sysctl variable > set at boot time and although V_tcp_reass_maxseg is set per vimage it is > the > result of a constant operation done on nmbclusters (nmbclusters / 16). > > What I've described is what I suspect is the root of my problem. The > manifestation of this problem is that TCP packets passing through my > vimage(s) are not reassembled when they are out of order and I get an > exceptionally high value reported by netstat -m stating that packets were > dropped due to "insufficient memory". Posts I've found on the net point to > the reassembly queue length, which in the vimages is zero for some reason. > > Perhaps this additional information will help clarify my exact problem. > > Thanks, > Jason > > On Sat, Nov 22, 2008 at 5:12 AM, Julian Elischer >wrote: > > > Jason Fines wrote: > > > >> Hello all, > >> > >> I've got a question about setting the sysctl variable > >> net.inet.tcp.reass.maxsegments to a non-zero value inside my vimages. > I'm > >> currently running the FreeBSD 7 with the VIMAGE package available at > >> http://imunes.tel.fer.hr/virtnet/vimage_7-20081015.tgz. > >> > >> My problem is with TCP reassembly support inside of the vimages, namely > >> with > >> the tcp.reass.maxsegments sysctl variable. I've tracked down where in > the > >> code the variable is set to line 122 in tcp_reass_init() of > >> netinet/tcp_reass.c: "V_tcp_reass_maxseg = nmbclusters / 16;". The line > >> clearly reads that maxsegments should be set to "nmbclusters /16", in > the > >> main OS (not in any vimage) the value is correctly set to 1/16 of what > my > >> nmbclusters sysctl variable is set to. However, inside all my vimages > >> nmbclusters is set correctly, while reass.maxsegments is incorrectly set > >> to > >> zero!!! > >> > > > > V_tcp_reass_maxseg is a macro that hides the fact that > > tcp_reass_maxseg is a PER Vimage variable. > > > > Part of the patch > > is to make some sysctls be per-vimage. I do not know exactly > > about that one.. I suspect it is actually a read-only > > whole-system value, and not per vimage. > > > > > > > > > > > >> Is it possible that nmbclusters when read on line 122 of > >> netinet/tcp_reass.c > >> is zero? Has anyone else experienced this problem? Is TCP reassembly > not > >> supported/tested inside vimages? > >> > >> Any help in this area would be greatly appreciated. > >> > >> Thanks, > >> Jason > >> > >> P.S. This technology is phenomenal, and thanks to everyone who is > involved > >> developing it. > >> _______________________________________________ > >> freebsd-virtualization at freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > >> To unsubscribe, send any mail to " > >> freebsd-virtualization-unsubscribe at freebsd.org" > >> > > > _________________________________________________________________ > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" >