From owner-freebsd-virtualization@FreeBSD.ORG Thu Feb 19 20:56:46 2009 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C75A106567A for ; Thu, 19 Feb 2009 20:56:46 +0000 (UTC) (envelope-from prvs=julian=2945f491c@elischer.org) Received: from smtp-outbound.ironport.com (smtp-outbound.ironport.com [63.251.108.112]) by mx1.freebsd.org (Postfix) with ESMTP id 84E0A8FC17 for ; Thu, 19 Feb 2009 20:56:46 +0000 (UTC) (envelope-from prvs=julian=2945f491c@elischer.org) Received: from unknown (HELO julian-mac.elischer.org) ([10.251.60.36]) by smtp-outbound.ironport.com with ESMTP; 19 Feb 2009 12:28:21 -0800 Message-ID: <499DC0F7.10709@elischer.org> Date: Thu, 19 Feb 2009 12:28:39 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209) MIME-Version: 1.0 To: FreeBSD virtualization mailing list , Marko Zec , "Bjoern A. Zeeb" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Vimage next step X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Feb 2009 20:56:47 -0000 I've been doing performance testing on the 'non-vimage' 'structified' case VS the original 'globals' case and have not been able to see any really significant differences (though I have seen very slight differences in the distribution of results). SO I think we are in the position of moving forward to the next steps. I think that just means checking in the rest of the vimage tree from what I have seen. Then we can play with it a bit and then proceed to the jail/vimage merge stuff that Jamie (and bz) are working on. One thing I'd like to do is make the following changes: 1/ evaluate the ordering of teh items in the vimage structures to see if there are items that should be clusterred for cache reasons. 2/ remove all sub structures from the vimage structures and replace them with pointers. This is because puting them in directly in the vimage structures will make our lives harder due to ABI issues. If they are independently allocated (*) then we don't need to worry about them changing in size. (*) actually they could still be allocated as a blob but we would access them as if they are separate. comments? Julian From owner-freebsd-virtualization@FreeBSD.ORG Thu Feb 19 23:26:04 2009 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7AB33106566C; Thu, 19 Feb 2009 23:26:04 +0000 (UTC) (envelope-from prvs=julian=2945f491c@elischer.org) Received: from smtp-outbound.ironport.com (smtp-outbound.ironport.com [63.251.108.112]) by mx1.freebsd.org (Postfix) with ESMTP id 604378FC08; Thu, 19 Feb 2009 23:26:04 +0000 (UTC) (envelope-from prvs=julian=2945f491c@elischer.org) Received: from unknown (HELO julian-mac.elischer.org) ([10.251.60.79]) by smtp-outbound.ironport.com with ESMTP; 19 Feb 2009 15:26:04 -0800 Message-ID: <499DEA9D.70105@elischer.org> Date: Thu, 19 Feb 2009 15:26:21 -0800 From: Julian Elischer User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209) MIME-Version: 1.0 To: FreeBSD virtualization mailing list , Marko Zec , "Bjoern A. Zeeb" References: <499DC0F7.10709@elischer.org> In-Reply-To: <499DC0F7.10709@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Vimage next step X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Feb 2009 23:26:04 -0000 Julian Elischer wrote: > I've been doing performance testing on the 'non-vimage' 'structified' > case VS the original 'globals' case and have not been able to see any > really significant differences (though I have seen very slight > differences in the distribution of results). > > SO I think we are in the position of moving forward to the next steps. > > I think that just means checking in the rest of the vimage tree > from what I have seen. > > Then we can play with it a bit and then proceed to the > jail/vimage merge stuff that Jamie (and bz) are working on. > > One thing I'd like to do is make the following changes: > > 1/ evaluate the ordering of the items in the vimage structures to see if > there are items that should be clustered for cache reasons. > > 2/ remove all sub structures from the vimage structures > and replace them with pointers. This is because puting them in > directly in the vimage structures will make our lives harder due to ABI > issues. If they are independently allocated (*) then we don't need to > worry about them changing in size. for example, the ipsec struct starts with: int _ipsec_debug; struct ipsecstat _ipsec4stat; struct secpolicy _ip4_def_policy; int _ip4_esp_trans_deflev; int _ip4_esp_net_deflev; This effectively fixes the size of the ipsecstat and secpolicy structures. I would like instead to have: int _ipsec_debug; struct ipsecstat *_ipsec4stat; struct secpolicy *_ip4_def_policy; int _ip4_esp_trans_deflev; int _ip4_esp_net_deflev; and have the initializer function allocate those separately. > > > (*) actually they could still be allocated as a blob but we would access > them as if they are separate. > > comments? > > Julian > > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Fri Feb 20 10:07:37 2009 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8655D1065672 for ; Fri, 20 Feb 2009 10:07:37 +0000 (UTC) (envelope-from zec@freebsd.org) Received: from labs4.cc.fer.hr (labs4.cc.fer.hr [161.53.72.24]) by mx1.freebsd.org (Postfix) with ESMTP id 10C798FC19 for ; Fri, 20 Feb 2009 10:07:36 +0000 (UTC) (envelope-from zec@freebsd.org) Received: from sluga.fer.hr (sluga.cc.fer.hr [161.53.72.14]) by labs4.cc.fer.hr (8.14.2/8.14.2) with ESMTP id n1K9t5wX005656; Fri, 20 Feb 2009 10:55:05 +0100 (CET) Received: from [192.168.200.100] ([161.53.19.79]) by sluga.fer.hr with Microsoft SMTPSVC(6.0.3790.3959); Fri, 20 Feb 2009 10:54:38 +0100 From: Marko Zec To: Julian Elischer Date: Fri, 20 Feb 2009 10:54:34 +0100 User-Agent: KMail/1.9.10 References: <499DC0F7.10709@elischer.org> <499DEA9D.70105@elischer.org> In-Reply-To: <499DEA9D.70105@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200902201054.35147.zec@freebsd.org> X-OriginalArrivalTime: 20 Feb 2009 09:54:38.0212 (UTC) FILETIME=[3E2DDC40:01C99341] X-Scanned-By: MIMEDefang 2.64 on 161.53.72.24 Cc: "Bjoern A. Zeeb" , FreeBSD virtualization mailing list Subject: Re: Vimage next step X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Feb 2009 10:07:37 -0000 On Friday 20 February 2009 00:26:21 Julian Elischer wrote: > Julian Elischer wrote: > > I've been doing performance testing on the 'non-vimage' > > 'structified' case VS the original 'globals' case and have not been > > able to see any really significant differences (though I have seen > > very slight differences in the distribution of results). > > > > SO I think we are in the position of moving forward to the next > > steps. > > > > I think that just means checking in the rest of the vimage tree > > from what I have seen. > > > > Then we can play with it a bit and then proceed to the > > jail/vimage merge stuff that Jamie (and bz) are working on. > > > > One thing I'd like to do is make the following changes: > > > > 1/ evaluate the ordering of the items in the vimage structures to > > see if there are items that should be clustered for cache reasons. > > > > 2/ remove all sub structures from the vimage structures > > and replace them with pointers. This is because puting them in > > directly in the vimage structures will make our lives harder due to > > ABI issues. If they are independently allocated (*) then we don't > > need to worry about them changing in size. > > for example, the ipsec struct starts with: > > int _ipsec_debug; > struct ipsecstat _ipsec4stat; > struct secpolicy _ip4_def_policy; > > int _ip4_esp_trans_deflev; > int _ip4_esp_net_deflev; > > This effectively fixes the size of the ipsecstat and secpolicy > structures. I would like instead to have: > int _ipsec_debug; > struct ipsecstat *_ipsec4stat; > struct secpolicy *_ip4_def_policy; > > int _ip4_esp_trans_deflev; > int _ip4_esp_net_deflev; > > and have the initializer function allocate those separately. > > > (*) actually they could still be allocated as a blob but we would > > access them as if they are separate. > > > > comments? I'm working on distilling the initializer functions from vimage to vimage-commit2 branch, and this will probably be a cause of enough controversy for itself, so that I'd propose to wait a few more days before engaging in other changes / optimizations. With my pointy-hat for losing lots of time on, I still think that so far we have been more or less successfull in merging bits over to svn without causing major damage to other people, so perhaps it might sense to continue one step at the time. Of course, I agree that reducing the impact of changes in random structures to the layout of vnet_* containers by using pointers instead of embedded structs would certainly be an improvement in ABI maintenance terms, but we shouldn't loose the performance perspective in mind, given that we've already introduced an additional level of indirection. Ordering of items in vnet_ containers is definitely something that could and should be played with sooner than later, though I can't / won't promise to spend significant amount of time on that in the next few weeks :) Marko