From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 05:58:42 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73372106567B for ; Mon, 9 Jun 2008 05:58:42 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outS.internet-mail-service.net (outs.internet-mail-service.net [216.240.47.242]) by mx1.freebsd.org (Postfix) with ESMTP id 59E5D8FC1C for ; Mon, 9 Jun 2008 05:58:42 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id D5C53249B; Sun, 8 Jun 2008 22:58:52 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 8DFB12D600F; Sun, 8 Jun 2008 22:58:41 -0700 (PDT) Message-ID: <484CC690.9020303@elischer.org> Date: Sun, 08 Jun 2008 22:58:40 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: FreeBSD Current , freebsd-virtualization@freebsd.org, Marko Zec Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 05:58:42 -0000 At the BSDCAn devsummit we discussed how to proceed with committing Vimage to -current. the Milestones included something like: June 8 (today) Headsup.... June 15 commit changes that add macros for vnet (network module) and vinet(inet virtualisation) with macros defined in such a way to make 0 actual differences. provable by md5 etc. Documentat s/hostname/g//V_hostname/ #define V_hostname hostname 2 weeks settle time, next step prepared, tested and reviewed. June 29 Add changes to convert all globals to members of per-module structures. Done in a reversible way (i.e. compilable out). Macros defined so that depending on compile options structures or globals are used (one global structure). Performance implications of using structures are evaluated. Structures possibly tuned. Initialisation routines added, checked and tuned. example: #if VIMAGE_USE_STRUCTS #define V_hostname sys_globals.hostname ... #else #define V_hostname hostname ... #endif July 13 globals removed in vnet, vinet. ifdefs and compile option removed or scaled back to make code clean to read again. Destructor routines added where needed. Remaining "NULL Macros" (compile to nothing at this point) committed to reduce the size of the MEAT diffs. Review of Meat diffs formally under way for final comment. example: #define INIT_VNET_INET(x) /* nothing */ add "INIT_VNET_INET(curvnet);"(and similar) where needed. remove globals (e.g. 'hostname') July 21 JAIL+Vimage framework committed. e.g. add new syscall, program, etc. (part one of meat diffs) structures still only global instances. vimage inhansed jails can be created but act jus tlike normal jails? July 28 Ability to created > 1 vimage enabled. Vimage enhanced jails now have private network stacks etc. August start on converting more modules as needed and time allows. Marko and I have been working towards splitting up the current diffs (which do the whole thing) so allow this schedule to be followed. We may or may not be ready for the June 15 step by then, but if not it may be a week there-after. So this should be considered the heads-up. discussion will be on freebsd-virtualization@ and the perforce branch that we have as a current working system is branch 'vimage'. //depot/projects/vimage/... diffs can be found at: http://www.freebsd.org/~julian/vimage.diff and it are usually fairly up to date. From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 09:57:03 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D55CD1065670; Mon, 9 Jun 2008 09:57:03 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9E4EE8FC22; Mon, 9 Jun 2008 09:57:02 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <484CFE6E.7040305@FreeBSD.org> Date: Mon, 09 Jun 2008 11:57:02 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Julian Elischer References: <484CC690.9020303@elischer.org> In-Reply-To: <484CC690.9020303@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Marko Zec , FreeBSD Current , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 09:57:03 -0000 Julian Elischer wrote: > At the BSDCAn devsummit we discussed how to proceed with committing > Vimage to -current. > > the Milestones included something like: > > June 8 (today) Headsup.... > > June 15 commit changes that add macros for vnet > (network module) and vinet(inet virtualisation) > with macros defined in such a way to make 0 actual > differences. provable by md5 etc. > Documentat > s/hostname/g//V_hostname/ > #define V_hostname hostname > 2 weeks settle time, next step prepared, tested > and reviewed. ... > diffs can be found at: > http://www.freebsd.org/~julian/vimage.diff and it are usually > fairly up to date. Did Marko fix the panic I saw back in May? I wasn't even able to boot a vimage kernel yet, let alone begin testing it :) Kris From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 17:07:20 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7196106564A for ; Mon, 9 Jun 2008 17:07:20 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from gritton.org (gritton.org [161.58.222.4]) by mx1.freebsd.org (Postfix) with ESMTP id A2DB18FC2C for ; Mon, 9 Jun 2008 17:07:20 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from guppy.corp.verio.net (fw.oremut02.us.wh.verio.net [198.65.168.24]) (authenticated bits=0) by gritton.org (8.13.6.20060614/8.13.6) with ESMTP id m59H7JWt074438 for ; Mon, 9 Jun 2008 11:07:19 -0600 (MDT) Message-ID: <484D6342.1080901@gritton.org> Date: Mon, 09 Jun 2008 11:07:14 -0600 From: James Gritton User-Agent: Thunderbird 2.0.0.9 (X11/20080228) MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.92.1/7413/Mon Jun 9 10:06:39 2008 on gritton.org X-Virus-Status: Clean Subject: jail_set X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 17:07:20 -0000 I've gotten the first stage working of the extensible name-based jail settings framework, with a patch available at http://gritton.org/jail_set.diff This is based around a new jail_set() system call, much like nmount() - in fact it even uses the same vfs options calls. It allows for modules (the existing "prison services" hooks that zfs use) to have be controlled via this interface, both to enable or disable the entire module, or to have their own module-specific parameters. The old jail() system call still exists and is compatible with this setup - it just becomes a stub to jail_set with the "path", "hostname", and "ip_number" parameters. There's also a sysctl tree security.jail.jid, that shows all parameters for current jails, once again with hooks for per-module parameters. The expectation is that vimage's vnet and vinet will become prison services under this framework, and the other more minor vimage bits will be rolled in as well. This would fit in with the goals of the 21 Jul deadline in Julian's recently posted schedule. Work still to do: Allow for hierarchical jails (which vimage needs). Actually integrate this with vimage. Integrate with other subsystems, more for proof of concept than anything else. SYSV IPC perhaps, since I've already done similar work on them. Or replace the one-off "pr_linux" hook on the prison structure with the standard services hook. Perhaps add a jail_get() system call, to read jail parameters. Currently, they can be read via sysctl, but that might not be the best way around this. If there's both a jail_set and jail_get, there may be no need for the extra effort of the sysctl tree. - Jamie From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 17:41:13 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48BF81065673 for ; Mon, 9 Jun 2008 17:41:13 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outC.internet-mail-service.net (outc.internet-mail-service.net [216.240.47.226]) by mx1.freebsd.org (Postfix) with ESMTP id 357ED8FC15 for ; Mon, 9 Jun 2008 17:41:13 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id E13872379; Mon, 9 Jun 2008 10:41:12 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 74AFF2D6011; Mon, 9 Jun 2008 10:41:12 -0700 (PDT) Message-ID: <484D6B38.3020207@elischer.org> Date: Mon, 09 Jun 2008 10:41:12 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Kris Kennaway References: <484CC690.9020303@elischer.org> <484CFE6E.7040305@FreeBSD.org> In-Reply-To: <484CFE6E.7040305@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Marko Zec , FreeBSD Current , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 17:41:13 -0000 Kris Kennaway wrote: > Julian Elischer wrote: >> At the BSDCAn devsummit we discussed how to proceed with committing >> Vimage to -current. >> >> the Milestones included something like: >> >> June 8 (today) Headsup.... >> >> June 15 commit changes that add macros for vnet >> (network module) and vinet(inet virtualisation) >> with macros defined in such a way to make 0 actual >> differences. provable by md5 etc. >> Documentat >> s/hostname/g//V_hostname/ >> #define V_hostname hostname >> 2 weeks settle time, next step prepared, tested >> and reviewed. > ... > >> diffs can be found at: >> http://www.freebsd.org/~julian/vimage.diff and it are usually >> fairly up to date. > > Did Marko fix the panic I saw back in May? I wasn't even able to boot a > vimage kernel yet, let alone begin testing it :) is this the ipv6 panic? I must admit I have not heard.. he was looking at it back thne and has committed stuff since then.. I use it with IPV4 quite successfully quite often. note that the first commits are pretty much quaranteed to not have that problem.. as they are effective NOPs I'll get back to you on it.. > > Kris From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 18:34:59 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B9641065673; Mon, 9 Jun 2008 18:34:59 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 18AC88FC24; Mon, 9 Jun 2008 18:34:57 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <484D77D2.9050102@FreeBSD.org> Date: Mon, 09 Jun 2008 20:34:58 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Julian Elischer References: <484CC690.9020303@elischer.org> <484CFE6E.7040305@FreeBSD.org> <484D6B38.3020207@elischer.org> In-Reply-To: <484D6B38.3020207@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Marko Zec , FreeBSD Current , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 18:34:59 -0000 Julian Elischer wrote: > Kris Kennaway wrote: >> Julian Elischer wrote: >>> At the BSDCAn devsummit we discussed how to proceed with committing >>> Vimage to -current. >>> >>> the Milestones included something like: >>> >>> June 8 (today) Headsup.... >>> >>> June 15 commit changes that add macros for vnet >>> (network module) and vinet(inet virtualisation) >>> with macros defined in such a way to make 0 actual >>> differences. provable by md5 etc. >>> Documentat >>> s/hostname/g//V_hostname/ >>> #define V_hostname hostname >>> 2 weeks settle time, next step prepared, tested >>> and reviewed. >> ... >> >>> diffs can be found at: >>> http://www.freebsd.org/~julian/vimage.diff and it are usually >>> fairly up to date. >> >> Did Marko fix the panic I saw back in May? I wasn't even able to boot >> a vimage kernel yet, let alone begin testing it :) > > is this the ipv6 panic? > I must admit I have not heard.. > he was looking at it back thne and has committed stuff since then.. > > I use it with IPV4 quite successfully quite often. > > note that the first commits are pretty much quaranteed to not have that > problem.. as they are effective NOPs > > > I'll get back to you on it.. Yes, the panic occurs when one runs a vimage kernel on a CVS world. It's presumably a case of incomplete validation of the input from userland. I'd still like someone to validate the initial commits and establish a framework for ongoing testing, because we've seen cases recently where things as simple as structure alignment changes can have >30% performance impact, so if it's not entirely a NOP then there is still potential for trouble. Kris From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 18:53:44 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 104DA106566C for ; Mon, 9 Jun 2008 18:53:44 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outN.internet-mail-service.net (outn.internet-mail-service.net [216.240.47.237]) by mx1.freebsd.org (Postfix) with ESMTP id ECAF68FC21 for ; Mon, 9 Jun 2008 18:53:42 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id AA73D2487; Mon, 9 Jun 2008 11:53:42 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 6A1332D600D; Mon, 9 Jun 2008 11:53:42 -0700 (PDT) Message-ID: <484D7C36.2030607@elischer.org> Date: Mon, 09 Jun 2008 11:53:42 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Kris Kennaway References: <484CC690.9020303@elischer.org> <484CFE6E.7040305@FreeBSD.org> <484D6B38.3020207@elischer.org> <484D77D2.9050102@FreeBSD.org> In-Reply-To: <484D77D2.9050102@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Marko Zec , FreeBSD Current , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 18:53:44 -0000 Kris Kennaway wrote: > Julian Elischer wrote: >> Kris Kennaway wrote: >>> Julian Elischer wrote: >>>> At the BSDCAn devsummit we discussed how to proceed with committing >>>> Vimage to -current. >>>> >>>> the Milestones included something like: >>>> >>>> June 8 (today) Headsup.... >>>> >>>> June 15 commit changes that add macros for vnet >>>> (network module) and vinet(inet virtualisation) >>>> with macros defined in such a way to make 0 actual >>>> differences. provable by md5 etc. >>>> Documentat >>>> s/hostname/g//V_hostname/ >>>> #define V_hostname hostname >>>> 2 weeks settle time, next step prepared, tested >>>> and reviewed. >>> ... >>> >>>> diffs can be found at: >>>> http://www.freebsd.org/~julian/vimage.diff and it are usually >>>> fairly up to date. >>> >>> Did Marko fix the panic I saw back in May? I wasn't even able to >>> boot a vimage kernel yet, let alone begin testing it :) >> >> is this the ipv6 panic? >> I must admit I have not heard.. >> he was looking at it back thne and has committed stuff since then.. >> >> I use it with IPV4 quite successfully quite often. >> >> note that the first commits are pretty much quaranteed to not have >> that problem.. as they are effective NOPs >> >> >> I'll get back to you on it.. > > Yes, the panic occurs when one runs a vimage kernel on a CVS world. It's > presumably a case of incomplete validation of the input from userland. > > I'd still like someone to validate the initial commits and establish a > framework for ongoing testing, because we've seen cases recently where > things as simple as structure alignment changes can have >30% we have set asside a step to confirm this.. but initial testing has shown no impact. > performance impact, so if it's not entirely a NOP then there is still > potential for trouble. > > Kris From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 19:03:22 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E530210657F2; Mon, 9 Jun 2008 19:03:22 +0000 (UTC) (envelope-from zec@fer.hr) Received: from xaqua.tel.fer.hr (xaqua.tel.fer.hr [161.53.19.25]) by mx1.freebsd.org (Postfix) with ESMTP id A17AF8FC24; Mon, 9 Jun 2008 19:03:22 +0000 (UTC) (envelope-from zec@fer.hr) Received: by xaqua.tel.fer.hr (Postfix, from userid 20006) id ED0CD9B64A; Mon, 9 Jun 2008 20:46:09 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on xaqua.tel.fer.hr X-Spam-Level: X-Spam-Status: No, score=-4.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.7 Received: from [192.168.200.111] (zec2.tel.fer.hr [161.53.19.79]) by xaqua.tel.fer.hr (Postfix) with ESMTP id D48009B644; Mon, 9 Jun 2008 20:46:08 +0200 (CEST) From: Marko Zec To: freebsd-current@freebsd.org Date: Mon, 9 Jun 2008 20:46:02 +0200 User-Agent: KMail/1.9.7 References: <484CC690.9020303@elischer.org> <484CFE6E.7040305@FreeBSD.org> In-Reply-To: <484CFE6E.7040305@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200806092046.02847.zec@fer.hr> X-Mailman-Approved-At: Mon, 09 Jun 2008 19:10:03 +0000 Cc: Julian Elischer , freebsd-virtualization@freebsd.org, FreeBSD Current Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 19:03:23 -0000 On Monday 09 June 2008 11:57:02 Kris Kennaway wrote: > Julian Elischer wrote: > > At the BSDCAn devsummit we discussed how to proceed with committing > > Vimage to -current. > > > > the Milestones included something like: > > > > June 8 (today) Headsup.... > > > > June 15 commit changes that add macros for vnet > > (network module) and vinet(inet virtualisation) > > with macros defined in such a way to make 0 actual > > differences. provable by md5 etc. > > Documentat > > s/hostname/g//V_hostname/ > > #define V_hostname hostname > > 2 weeks settle time, next step prepared, tested > > and reviewed. > > ... > > > diffs can be found at: > > http://www.freebsd.org/~julian/vimage.diff and it are usually > > fairly up to date. > > Did Marko fix the panic I saw back in May? I wasn't even able to > boot a vimage kernel yet, let alone begin testing it :) I just submitted a p4 change that allows for a machine to boot with rpc.lockd enabled, but haven't tested it with any NFS mounts yet. But do you really need to have rpc.lockd running to be able to do any testing? Pls. note that the vimage branch tracking HEAD has other misterious lockups that I haven't been able to track down yet. The user/zec/vimage_7 branch is in much better shape... Marko From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 19:11:04 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F64A1065672; Mon, 9 Jun 2008 19:11:04 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7A6408FC16; Mon, 9 Jun 2008 19:11:02 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <484D8046.7020403@FreeBSD.org> Date: Mon, 09 Jun 2008 21:11:02 +0200 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Marko Zec References: <484CC690.9020303@elischer.org> <484CFE6E.7040305@FreeBSD.org> <200806092046.02847.zec@fer.hr> In-Reply-To: <200806092046.02847.zec@fer.hr> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Julian Elischer , freebsd-virtualization@freebsd.org, FreeBSD Current Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 19:11:04 -0000 Marko Zec wrote: > On Monday 09 June 2008 11:57:02 Kris Kennaway wrote: >> Julian Elischer wrote: >>> At the BSDCAn devsummit we discussed how to proceed with committing >>> Vimage to -current. >>> >>> the Milestones included something like: >>> >>> June 8 (today) Headsup.... >>> >>> June 15 commit changes that add macros for vnet >>> (network module) and vinet(inet virtualisation) >>> with macros defined in such a way to make 0 actual >>> differences. provable by md5 etc. >>> Documentat >>> s/hostname/g//V_hostname/ >>> #define V_hostname hostname >>> 2 weeks settle time, next step prepared, tested >>> and reviewed. >> ... >> >>> diffs can be found at: >>> http://www.freebsd.org/~julian/vimage.diff and it are usually >>> fairly up to date. >> Did Marko fix the panic I saw back in May? I wasn't even able to >> boot a vimage kernel yet, let alone begin testing it :) > > I just submitted a p4 change that allows for a machine to boot with > rpc.lockd enabled, but haven't tested it with any NFS mounts yet. But > do you really need to have rpc.lockd running to be able to do any > testing? Probably not in an absolute sense, but it presumably is something that must be fixed anyway :) > Pls. note that the vimage branch tracking HEAD has other misterious > lockups that I haven't been able to track down yet. The > user/zec/vimage_7 branch is in much better shape... OK, I will wait until the HEAD branch is in good shape. Please let me know when I can start testing. Kris From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 19:25:09 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C280210657BA for ; Mon, 9 Jun 2008 19:25:09 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.cksoft.de (mail.cksoft.de [62.111.66.27]) by mx1.freebsd.org (Postfix) with ESMTP id CE34B8FC15 for ; Mon, 9 Jun 2008 19:25:07 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from localhost (amavis.str.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 38AD741C732; Mon, 9 Jun 2008 21:25:06 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([62.111.66.27]) by localhost (amavis.str.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id poTYhAYJxNEp; Mon, 9 Jun 2008 21:25:05 +0200 (CEST) Received: by mail.cksoft.de (Postfix, from userid 66) id B02FE41C72C; Mon, 9 Jun 2008 21:25:05 +0200 (CEST) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 4981044487F; Mon, 9 Jun 2008 19:24:42 +0000 (UTC) Date: Mon, 9 Jun 2008 19:24:42 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Julian Elischer In-Reply-To: <484CC690.9020303@elischer.org> Message-ID: <20080609174826.Q83875@maildrop.int.zabbadoz.net> References: <484CC690.9020303@elischer.org> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Marko Zec , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 19:25:09 -0000 On Sun, 8 Jun 2008, Julian Elischer wrote: > At the BSDCAn devsummit we discussed how to proceed with committing Vimage to > -current. > > the Milestones included something like: > > June 8 (today) Headsup.... > > June 15 commit changes that add macros for vnet > (network module) and vinet(inet virtualisation) > with macros defined in such a way to make 0 actual > differences. provable by md5 etc. > Documentat > s/hostname/g//V_hostname/ > #define V_hostname hostname > 2 weeks settle time, next step prepared, tested > and reviewed. For which part were you talking about a sed/awk script to use? Can we have a diff for just this part (once it is avail?) [schedule] * I am missing the BIG HEADS UP somewhere for all the people with outstanding work so that they will not re-do any integration multiple times. * I am missing the developers and users documentation in the schedule. > diffs can be found at: > http://www.freebsd.org/~julian/vimage.diff and it are usually > fairly up to date. I am just starting to skip through the patch, not doing a close review atm (not checking functional changes, etc. at all), and even this is hard at the end of the day... Are sys/ddb/db_command.c related in any way to this? sys/kern/init_main.c has an extra whitespace before the LIST_FIRST. sys/kern/kern_linker.c Isizeof(lookup) should be 4 space indent not 2 tabs. Do we need those changes like sys/kern/kern_switch.c ? sys/kern/kern_sysctl.c has indentation problems in the @@ -1322,7 +1421,17 @@ junk sys/kern/kern_timeout.c has an extra whitespace sys/kern/kern_vimage.c says "XXX RCS tag goes here" so add it. sys/kern/kern_vimage.c has // comments no-no - " - s,#define NAME,#define\tNAME,g - " - vnet_mod_register,vnet_mod_register_multi,(more)... declarations - " - adds a new suser() call. - " - in vi_symlookup() 2nsd line of for, remove a space - " - thinks like this scare me: /* A brute force check whether there's enough mem for a new vimage */ especially if its freed again instantly - " - near Detach / free per-module state instances remove whitespace - " - vi_free() remove a \t before the break; - " - db_show_vnets should probably check for db_pager_quit sys/kern/kern_xxx.c the printf looks like debugging? sys/kern/sys_socket.c has an unrelated whitespace change sys/kern/uipc_domain.c removes a comment I am entirely sure it can be removed. - " - why do we need to change net_init_domain(?here?) just to cast again? sys/kern/uipc_socket.c junk @@ -1284,13 +1314,17 @@ s,\t, , - " - if (how != SHUT_RD) { int error; add \n sys/kern/vfs_lookup.c adds something called IMUNES_SYMLINK_HACK which should either be renamed or removed. sys/modules/Makefile does not look like it belongs there sys/modules/netgraph/Makefile looks really strange, can we fix that? sys/modules/netgraph/pipe/Makefile has an extra space sys/modules/netgraph/wormhole/Makefile has an extra space sys/net/bpf.c adds an IMUNES_BPF_HACK, and defines it - either rename or remove; also has whitespace issues and debugging printfs in there (that should not compile). sys/net/if.c @@ -292,31 +317,73 @@ junk if (IS_DEFAULT_VNET(curvnet)) { ... needs an extra \t, no? doesn't look nice; there are more of those in this file; maybe not yet; not before the #ifdefs go. - " - SYSINT .. if_attachdomain was a wrong ws change - " - junk @@ -1842,6 +1971,24 @@ adds another suser() - " - at the end there are two unrelated/wrong ws changes sys/net/if_ethersubr.c ether_reassign() has whitespace issues - " - SYSCTL_V_INT for ether_ipfw 2nd line indent looks wrong sys/net/if_gif.c SYSCTL_V_INT 2nd line, parallel_tunnels indent - " - gifmodevent() empty line wrongly removed sys/net/if_gif.h #define\tNAME sys/net/if_gre.c is there a reason to rename the local variables? sys/net/if_loop.c I cannot see a difference for vnet_loif_iattach w/ or w/o the #ifdef. Should the outer one go? - " - is there a need to move the loif check up in lo_clone_destroy? - " - junk @@ -190,7 +266,7 @@ use 4 spaces sys/net/if_mib.c SYSCTL_V_INT fix ws sys/net/if_var.h do we need to move if_index? sys/net/route.c static uma_zone_t rtzone; has an uneeded ws change - " - rtable_init() ws wrong - " - is that realted to more MRT changes or why are functions split and shuffled? - " - there were and still are more ws problems around V_rt_tables - " - return 0; ws problem - " - rtable_idetach() ws problem and more and the return sys/net/rtsock.c rnh =\n ... whitespace next line sys/net/vnet.h XXX RCS tag goes here do so - " - struct vnet_net has ws issues with the _ether_ipfw line - " - #define\tNAME I am running out of battery, so I am going to continue with the next ~20%+- in sys/net80211/**, l 6556 tomorrow. General: values in return statements should be enclosed in parentheses. General: function declarations K&R vs. ANSI vs ... General: you are adding 92 lines with XXX, 18 say "locking", 2 say WRONG, 10 say RCS, (other), ... can we get (most of) them fixed before committing? (fixed, not removed) /bz -- Bjoern A. Zeeb Stop bit received. Insert coin for new game. From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 19:47:12 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5673106567D for ; Mon, 9 Jun 2008 19:47:12 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outU.internet-mail-service.net (outu.internet-mail-service.net [216.240.47.244]) by mx1.freebsd.org (Postfix) with ESMTP id 93F288FC1C for ; Mon, 9 Jun 2008 19:47:12 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 863422425; Mon, 9 Jun 2008 12:47:12 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 918FC2D623F; Mon, 9 Jun 2008 12:46:58 -0700 (PDT) Message-ID: <484D88AC.2000402@elischer.org> Date: Mon, 09 Jun 2008 12:46:52 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: FreeBSD Net , freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: vimage include files X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 19:47:12 -0000 The current vimage code adds a handful of new include files.. e.g. vnet.h for vimage related defines that are related to general networking stuff vinet for vimage related defines that are related to inet. however eventually these defines would move to other files. For example I think every single file that includes vinet.h already includes netinet/in.h so these definitions move into that file. My question however comes with vnet.h 95% of the files that use it also include so possibly they could go there, but a few of them don't. they are: uipc_socket.c (sets a reference counter in the vnet structure) raw_cb.c accesses V_rawcb_list raw_usrreq.c accesses V_rawcb_list tcp_output.c lots of stuff of course but doesn't use if.h tcp_timer.c ditto vnet appears to be needed jsut for the SYSCTL_V_ stuff. (marko?) netipsec/keysock.c no need for if.h so there is no one place where all of the vnet structure is in scope but if.h is the closest. so should we: keep vnet.h? split it up a bit to make it more in scope? Find/make an include file for "networking in general?" is there such a file? As I said if.h seems the closest. From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 20:06:52 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E82C91065679 for ; Mon, 9 Jun 2008 20:06:52 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outT.internet-mail-service.net (outt.internet-mail-service.net [216.240.47.243]) by mx1.freebsd.org (Postfix) with ESMTP id D0E1E8FC27 for ; Mon, 9 Jun 2008 20:06:52 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 8AD692439; Mon, 9 Jun 2008 13:06:52 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 0698F2D6A85; Mon, 9 Jun 2008 13:06:51 -0700 (PDT) Message-ID: <484D8D5B.9010500@elischer.org> Date: Mon, 09 Jun 2008 13:06:51 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: "Bjoern A. Zeeb" References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> In-Reply-To: <20080609174826.Q83875@maildrop.int.zabbadoz.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Marko Zec , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 20:06:53 -0000 Bjoern A. Zeeb wrote: > On Sun, 8 Jun 2008, Julian Elischer wrote: > >> At the BSDCAn devsummit we discussed how to proceed with committing >> Vimage to -current. >> >> the Milestones included something like: >> >> June 8 (today) Headsup.... >> >> June 15 commit changes that add macros for vnet >> (network module) and vinet(inet virtualisation) >> with macros defined in such a way to make 0 actual >> differences. provable by md5 etc. >> Documentat >> s/hostname/g//V_hostname/ >> #define V_hostname hostname >> 2 weeks settle time, next step prepared, tested >> and reviewed. > > For which part were you talking about a sed/awk script to use? > Can we have a diff for just this part (once it is avail?) We are generating this now (the script..it'll be tcl it looks like :-) I have a "hand made" version of the diff that I'm generating for comparison. It is made by getting the diffs in the "vimage-commit" branch. run the perl script in the base of that branch to get the diff generated for you. What I'm doing is creating a diff that will compile with no changes, and yet includes as much of the full diff as possible. note: that branch doesn't compile yet as I haven't defined all the macros yet, but the ain is that it will, and should produce a "No differences" binary.. > > > [schedule] > > * I am missing the BIG HEADS UP somewhere for all the people with > outstanding work so that they will not re-do any integration multiple > times. > > * I am missing the developers and users documentation in the schedule. > > > >> diffs can be found at: >> http://www.freebsd.org/~julian/vimage.diff and it are usually >> fairly up to date. > > I am just starting to skip through the patch, not doing a close review > atm (not checking functional changes, etc. at all), and even this is > hard at the end of the day... > > Are sys/ddb/db_command.c related in any way to this? > > sys/kern/init_main.c has an extra whitespace before the LIST_FIRST. > > sys/kern/kern_linker.c Isizeof(lookup) should be 4 space indent not 2 > tabs. > > Do we need those changes like sys/kern/kern_switch.c ? > > sys/kern/kern_sysctl.c has indentation problems in the > @@ -1322,7 +1421,17 @@ junk > > sys/kern/kern_timeout.c has an extra whitespace > > sys/kern/kern_vimage.c says "XXX RCS tag goes here" so add it. > sys/kern/kern_vimage.c has // comments no-no > - " - s,#define NAME,#define\tNAME,g > - " - vnet_mod_register,vnet_mod_register_multi,(more)... declarations > - " - adds a new suser() call. > - " - in vi_symlookup() 2nsd line of for, remove a space > - " - thinks like this scare me: > /* A brute force check whether there's enough mem for a new vimage */ > especially if its freed again instantly > - " - near Detach / free per-module state instances remove whitespace > - " - vi_free() remove a \t before the break; > - " - db_show_vnets should probably check for db_pager_quit > > sys/kern/kern_xxx.c the printf looks like debugging? > > sys/kern/sys_socket.c has an unrelated whitespace change > > sys/kern/uipc_domain.c removes a comment I am entirely sure it can be > removed. > - " - why do we need to change net_init_domain(?here?) just to cast > again? > > sys/kern/uipc_socket.c junk @@ -1284,13 +1314,17 @@ s,\t, , > - " - if (how != SHUT_RD) { int error; add \n > > sys/kern/vfs_lookup.c adds something called IMUNES_SYMLINK_HACK which > should either be renamed or removed. > > sys/modules/Makefile does not look like it belongs there > > sys/modules/netgraph/Makefile looks really strange, can we fix that? > > sys/modules/netgraph/pipe/Makefile has an extra space > > sys/modules/netgraph/wormhole/Makefile has an extra space > > sys/net/bpf.c adds an IMUNES_BPF_HACK, and defines it - either > rename or remove; also has whitespace issues and debugging > printfs in there (that should not compile). > > sys/net/if.c @@ -292,31 +317,73 @@ junk if (IS_DEFAULT_VNET(curvnet)) { > ... needs an extra \t, no? doesn't look nice; there are more > of those in this file; maybe not yet; not before the #ifdefs go. > - " - SYSINT .. if_attachdomain was a wrong ws change > - " - junk @@ -1842,6 +1971,24 @@ adds another suser() > - " - at the end there are two unrelated/wrong ws changes > > sys/net/if_ethersubr.c ether_reassign() has whitespace issues > - " - SYSCTL_V_INT for ether_ipfw 2nd line indent looks wrong > > sys/net/if_gif.c SYSCTL_V_INT 2nd line, parallel_tunnels indent > - " - gifmodevent() empty line wrongly removed > > sys/net/if_gif.h #define\tNAME > > sys/net/if_gre.c is there a reason to rename the local variables? > > sys/net/if_loop.c I cannot see a difference for vnet_loif_iattach > w/ or w/o the #ifdef. Should the outer one go? > - " - is there a need to move the loif check up in lo_clone_destroy? > - " - junk @@ -190,7 +266,7 @@ use 4 spaces > > sys/net/if_mib.c SYSCTL_V_INT fix ws > > sys/net/if_var.h do we need to move if_index? > > sys/net/route.c static uma_zone_t rtzone; has an uneeded ws change > - " - rtable_init() ws wrong > - " - is that realted to more MRT changes or why are functions split > and shuffled? > - " - there were and still are more ws problems around V_rt_tables > - " - return 0; ws problem > - " - rtable_idetach() ws problem and more and the return > > sys/net/rtsock.c rnh =\n ... whitespace next line > > sys/net/vnet.h XXX RCS tag goes here do so > - " - struct vnet_net has ws issues with the _ether_ipfw line > - " - #define\tNAME > > > > I am running out of battery, so I am going to continue with the > next ~20%+- in sys/net80211/**, l 6556 tomorrow. > > > General: values in return statements should be enclosed in parentheses. > > General: function declarations K&R vs. ANSI vs ... > > General: you are adding 92 lines with XXX, 18 say "locking", 2 say > WRONG, 10 say RCS, (other), ... can we get (most of) them fixed before > committing? (fixed, not removed) > > > /bz > From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 20:13:18 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6AB5B10657B1 for ; Mon, 9 Jun 2008 20:13:18 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outU.internet-mail-service.net (outu.internet-mail-service.net [216.240.47.244]) by mx1.freebsd.org (Postfix) with ESMTP id 52E248FC0C for ; Mon, 9 Jun 2008 20:13:18 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 40734245A; Mon, 9 Jun 2008 13:13:18 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id E58942D6C43; Mon, 9 Jun 2008 13:13:17 -0700 (PDT) Message-ID: <484D8EDD.3040103@elischer.org> Date: Mon, 09 Jun 2008 13:13:17 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: "Bjoern A. Zeeb" References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> In-Reply-To: <20080609174826.Q83875@maildrop.int.zabbadoz.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Marko Zec , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 20:13:18 -0000 Bjoern A. Zeeb wrote: > On Sun, 8 Jun 2008, Julian Elischer wrote: > >> At the BSDCAn devsummit we discussed how to proceed with committing >> Vimage to -current. >> >> the Milestones included something like: >> >> June 8 (today) Headsup.... >> >> June 15 commit changes that add macros for vnet >> (network module) and vinet(inet virtualisation) >> with macros defined in such a way to make 0 actual >> differences. provable by md5 etc. >> Documentat >> s/hostname/g//V_hostname/ >> #define V_hostname hostname >> 2 weeks settle time, next step prepared, tested >> and reviewed. > > For which part were you talking about a sed/awk script to use? > Can we have a diff for just this part (once it is avail?) > > > [schedule] > > * I am missing the BIG HEADS UP somewhere for all the people with > outstanding work so that they will not re-do any integration multiple > times. > > * I am missing the developers and users documentation in the schedule. > > > >> diffs can be found at: >> http://www.freebsd.org/~julian/vimage.diff and it are usually >> fairly up to date. > > I am just starting to skip through the patch, not doing a close review > atm (not checking functional changes, etc. at all), and even this is > hard at the end of the day... > > Are sys/ddb/db_command.c related in any way to this? > > sys/kern/init_main.c has an extra whitespace before the LIST_FIRST. > > sys/kern/kern_linker.c Isizeof(lookup) should be 4 space indent not 2 > tabs. > > Do we need those changes like sys/kern/kern_switch.c ? This diff there includes experimental changes to virtualise things like load average, and they will not be part of the commit. so ignore anything that smells like "scheduler" > > sys/kern/kern_sysctl.c has indentation problems in the > @@ -1322,7 +1421,17 @@ junk > > sys/kern/kern_timeout.c has an extra whitespace yes we (I) will be trying to clean that sort of thing.. > > sys/kern/kern_vimage.c says "XXX RCS tag goes here" so add it. SNV tag? > sys/kern/kern_vimage.c has // comments no-no > - " - s,#define NAME,#define\tNAME,g > - " - vnet_mod_register,vnet_mod_register_multi,(more)... declarations > - " - adds a new suser() call. > - " - in vi_symlookup() 2nsd line of for, remove a space > - " - thinks like this scare me: > /* A brute force check whether there's enough mem for a new vimage */ > especially if its freed again instantly > - " - near Detach / free per-module state instances remove whitespace > - " - vi_free() remove a \t before the break; > - " - db_show_vnets should probably check for db_pager_quit > > sys/kern/kern_xxx.c the printf looks like debugging? > > sys/kern/sys_socket.c has an unrelated whitespace change > > sys/kern/uipc_domain.c removes a comment I am entirely sure it can be > removed. > - " - why do we need to change net_init_domain(?here?) just to cast > again? > > sys/kern/uipc_socket.c junk @@ -1284,13 +1314,17 @@ s,\t, , > - " - if (how != SHUT_RD) { int error; add \n > > sys/kern/vfs_lookup.c adds something called IMUNES_SYMLINK_HACK which > should either be renamed or removed. > > sys/modules/Makefile does not look like it belongs there > > sys/modules/netgraph/Makefile looks really strange, can we fix that? > > sys/modules/netgraph/pipe/Makefile has an extra space > > sys/modules/netgraph/wormhole/Makefile has an extra space > > sys/net/bpf.c adds an IMUNES_BPF_HACK, and defines it - either > rename or remove; also has whitespace issues and debugging > printfs in there (that should not compile). > > sys/net/if.c @@ -292,31 +317,73 @@ junk if (IS_DEFAULT_VNET(curvnet)) { > ... needs an extra \t, no? doesn't look nice; there are more > of those in this file; maybe not yet; not before the #ifdefs go. > - " - SYSINT .. if_attachdomain was a wrong ws change > - " - junk @@ -1842,6 +1971,24 @@ adds another suser() > - " - at the end there are two unrelated/wrong ws changes > > sys/net/if_ethersubr.c ether_reassign() has whitespace issues > - " - SYSCTL_V_INT for ether_ipfw 2nd line indent looks wrong > > sys/net/if_gif.c SYSCTL_V_INT 2nd line, parallel_tunnels indent > - " - gifmodevent() empty line wrongly removed > > sys/net/if_gif.h #define\tNAME > > sys/net/if_gre.c is there a reason to rename the local variables? > > sys/net/if_loop.c I cannot see a difference for vnet_loif_iattach > w/ or w/o the #ifdef. Should the outer one go? > - " - is there a need to move the loif check up in lo_clone_destroy? > - " - junk @@ -190,7 +266,7 @@ use 4 spaces > > sys/net/if_mib.c SYSCTL_V_INT fix ws > > sys/net/if_var.h do we need to move if_index? > > sys/net/route.c static uma_zone_t rtzone; has an uneeded ws change > - " - rtable_init() ws wrong > - " - is that realted to more MRT changes or why are functions split > and shuffled? > - " - there were and still are more ws problems around V_rt_tables > - " - return 0; ws problem > - " - rtable_idetach() ws problem and more and the return > > sys/net/rtsock.c rnh =\n ... whitespace next line > > sys/net/vnet.h XXX RCS tag goes here do so > - " - struct vnet_net has ws issues with the _ether_ipfw line > - " - #define\tNAME > > > > I am running out of battery, so I am going to continue with the > next ~20%+- in sys/net80211/**, l 6556 tomorrow. > > > General: values in return statements should be enclosed in parentheses. > > General: function declarations K&R vs. ANSI vs ... > > General: you are adding 92 lines with XXX, 18 say "locking", 2 say > WRONG, 10 say RCS, (other), ... can we get (most of) them fixed before > committing? (fixed, not removed) thanks max Our hope is to generate a set of patches derived from that we have now rather than commit what we have exactly, so we hope we can get your cleanups included as we create those diffs. feel free to use p4 to fix things yourself if you want to.. > > > /bz > From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 20:15:01 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 979611065671 for ; Mon, 9 Jun 2008 20:15:01 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outG.internet-mail-service.net (outg.internet-mail-service.net [216.240.47.230]) by mx1.freebsd.org (Postfix) with ESMTP id 8065F8FC1C for ; Mon, 9 Jun 2008 20:15:01 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 6B5582433; Mon, 9 Jun 2008 13:15:01 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 235DC2D6C57; Mon, 9 Jun 2008 13:15:01 -0700 (PDT) Message-ID: <484D8F44.60603@elischer.org> Date: Mon, 09 Jun 2008 13:15:00 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: "Bjoern A. Zeeb" References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> In-Reply-To: <484D8EDD.3040103@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Marko Zec , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 20:15:01 -0000 Julian Elischer wrote: > Bjoern A. Zeeb wrote: >> On Sun, 8 Jun 2008, Julian Elischer wrote: >> >>> At the BSDCAn devsummit we discussed how to proceed with committing >>> Vimage to -current. >>> >>> the Milestones included something like: >>> >>> June 8 (today) Headsup.... >>> >>> June 15 commit changes that add macros for vnet >>> (network module) and vinet(inet virtualisation) >>> with macros defined in such a way to make 0 actual >>> differences. provable by md5 etc. >>> Documentat >>> s/hostname/g//V_hostname/ >>> #define V_hostname hostname >>> 2 weeks settle time, next step prepared, tested >>> and reviewed. >> >> For which part were you talking about a sed/awk script to use? >> Can we have a diff for just this part (once it is avail?) >> >> >> [schedule] >> >> * I am missing the BIG HEADS UP somewhere for all the people with >> outstanding work so that they will not re-do any integration multiple >> times. >> >> * I am missing the developers and users documentation in the schedule. >> >> >> >>> diffs can be found at: >>> http://www.freebsd.org/~julian/vimage.diff and it are usually >>> fairly up to date. >> >> I am just starting to skip through the patch, not doing a close review >> atm (not checking functional changes, etc. at all), and even this is >> hard at the end of the day... >> >> Are sys/ddb/db_command.c related in any way to this? >> >> sys/kern/init_main.c has an extra whitespace before the LIST_FIRST. >> >> sys/kern/kern_linker.c Isizeof(lookup) should be 4 space indent not 2 >> tabs. >> >> Do we need those changes like sys/kern/kern_switch.c ? > > This diff there includes experimental changes to virtualise things > like load average, and they will not be part of the commit. > so ignore anything that smells like "scheduler" > >> >> sys/kern/kern_sysctl.c has indentation problems in the >> @@ -1322,7 +1421,17 @@ junk >> >> sys/kern/kern_timeout.c has an extra whitespace > > yes we (I) will be trying to clean that sort of thing.. > >> >> sys/kern/kern_vimage.c says "XXX RCS tag goes here" so add it. > > SNV tag? > >> sys/kern/kern_vimage.c has // comments no-no >> - " - s,#define NAME,#define\tNAME,g >> - " - vnet_mod_register,vnet_mod_register_multi,(more)... declarations >> - " - adds a new suser() call. >> - " - in vi_symlookup() 2nsd line of for, remove a space >> - " - thinks like this scare me: >> /* A brute force check whether there's enough mem for a new vimage */ >> especially if its freed again instantly >> - " - near Detach / free per-module state instances remove whitespace >> - " - vi_free() remove a \t before the break; >> - " - db_show_vnets should probably check for db_pager_quit >> >> sys/kern/kern_xxx.c the printf looks like debugging? >> >> sys/kern/sys_socket.c has an unrelated whitespace change >> >> sys/kern/uipc_domain.c removes a comment I am entirely sure it can be >> removed. >> - " - why do we need to change net_init_domain(?here?) just to cast >> again? >> >> sys/kern/uipc_socket.c junk @@ -1284,13 +1314,17 @@ s,\t, , >> - " - if (how != SHUT_RD) { int error; add \n >> >> sys/kern/vfs_lookup.c adds something called IMUNES_SYMLINK_HACK which >> should either be renamed or removed. >> >> sys/modules/Makefile does not look like it belongs there >> >> sys/modules/netgraph/Makefile looks really strange, can we fix that? >> >> sys/modules/netgraph/pipe/Makefile has an extra space >> >> sys/modules/netgraph/wormhole/Makefile has an extra space >> >> sys/net/bpf.c adds an IMUNES_BPF_HACK, and defines it - either >> rename or remove; also has whitespace issues and debugging >> printfs in there (that should not compile). >> >> sys/net/if.c @@ -292,31 +317,73 @@ junk if (IS_DEFAULT_VNET(curvnet)) { >> ... needs an extra \t, no? doesn't look nice; there are more >> of those in this file; maybe not yet; not before the #ifdefs go. >> - " - SYSINT .. if_attachdomain was a wrong ws change >> - " - junk @@ -1842,6 +1971,24 @@ adds another suser() >> - " - at the end there are two unrelated/wrong ws changes >> >> sys/net/if_ethersubr.c ether_reassign() has whitespace issues >> - " - SYSCTL_V_INT for ether_ipfw 2nd line indent looks wrong >> >> sys/net/if_gif.c SYSCTL_V_INT 2nd line, parallel_tunnels indent >> - " - gifmodevent() empty line wrongly removed >> >> sys/net/if_gif.h #define\tNAME >> >> sys/net/if_gre.c is there a reason to rename the local variables? >> >> sys/net/if_loop.c I cannot see a difference for vnet_loif_iattach >> w/ or w/o the #ifdef. Should the outer one go? >> - " - is there a need to move the loif check up in lo_clone_destroy? >> - " - junk @@ -190,7 +266,7 @@ use 4 spaces >> >> sys/net/if_mib.c SYSCTL_V_INT fix ws >> >> sys/net/if_var.h do we need to move if_index? >> >> sys/net/route.c static uma_zone_t rtzone; has an uneeded ws change >> - " - rtable_init() ws wrong >> - " - is that realted to more MRT changes or why are functions split >> and shuffled? >> - " - there were and still are more ws problems around V_rt_tables >> - " - return 0; ws problem >> - " - rtable_idetach() ws problem and more and the return >> >> sys/net/rtsock.c rnh =\n ... whitespace next line >> >> sys/net/vnet.h XXX RCS tag goes here do so >> - " - struct vnet_net has ws issues with the _ether_ipfw line >> - " - #define\tNAME >> >> >> >> I am running out of battery, so I am going to continue with the >> next ~20%+- in sys/net80211/**, l 6556 tomorrow. >> >> >> General: values in return statements should be enclosed in parentheses. >> >> General: function declarations K&R vs. ANSI vs ... >> >> General: you are adding 92 lines with XXX, 18 say "locking", 2 say >> WRONG, 10 say RCS, (other), ... can we get (most of) them fixed before >> committing? (fixed, not removed) > > thanks max I'm an idiot.. I got mixed up between two emails.. "thanks Bjoern" I should have said.. > > Our hope is to generate a set of patches derived from that we have now > rather than commit what we have exactly, so we hope we can get your > cleanups included as we create those diffs. > > feel free to use p4 to fix things yourself if you want to.. > > >> >> >> /bz >> > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 20:30:08 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18B681065687; Mon, 9 Jun 2008 20:30:08 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.cksoft.de (mail.cksoft.de [62.111.66.27]) by mx1.freebsd.org (Postfix) with ESMTP id C8A148FC1C; Mon, 9 Jun 2008 20:30:07 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from localhost (amavis.str.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 4689041C74D; Mon, 9 Jun 2008 22:30:06 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([62.111.66.27]) by localhost (amavis.str.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id 9oAwqmjdCSrT; Mon, 9 Jun 2008 22:30:06 +0200 (CEST) Received: by mail.cksoft.de (Postfix, from userid 66) id F0F7341C76D; Mon, 9 Jun 2008 22:30:05 +0200 (CEST) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 550FF44487F; Mon, 9 Jun 2008 20:28:49 +0000 (UTC) Date: Mon, 9 Jun 2008 20:28:49 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Julian Elischer In-Reply-To: <484D8EDD.3040103@elischer.org> Message-ID: <20080609202807.V83875@maildrop.int.zabbadoz.net> References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Marko Zec , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 20:30:08 -0000 On Mon, 9 Jun 2008, Julian Elischer wrote: > feel free to use p4 to fix things yourself if you want to.. to which of the --n++ branches? vimage as in //depot/projects/vimage/src/sys/... ? -- Bjoern A. Zeeb Stop bit received. Insert coin for new game. From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 20:53:30 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3C25106567A for ; Mon, 9 Jun 2008 20:53:30 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outK.internet-mail-service.net (outk.internet-mail-service.net [216.240.47.234]) by mx1.freebsd.org (Postfix) with ESMTP id A0F528FC1C for ; Mon, 9 Jun 2008 20:53:30 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 66BE924B6; Mon, 9 Jun 2008 13:53:30 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 40C872D6006; Mon, 9 Jun 2008 13:53:30 -0700 (PDT) Message-ID: <484D9849.4030403@elischer.org> Date: Mon, 09 Jun 2008 13:53:29 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: "Bjoern A. Zeeb" References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> <20080609202807.V83875@maildrop.int.zabbadoz.net> In-Reply-To: <20080609202807.V83875@maildrop.int.zabbadoz.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Marko Zec , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 20:53:30 -0000 Bjoern A. Zeeb wrote: > On Mon, 9 Jun 2008, Julian Elischer wrote: > >> feel free to use p4 to fix things yourself if you want to.. > > to which of the --n++ branches? > > vimage as in //depot/projects/vimage/src/sys/... ? yep > > From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 21:49:04 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 45467106567D for ; Mon, 9 Jun 2008 21:49:04 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from gritton.org (gritton.org [161.58.222.4]) by mx1.freebsd.org (Postfix) with ESMTP id 092248FC2A for ; Mon, 9 Jun 2008 21:49:03 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from guppy.corp.verio.net (fw.oremut02.us.wh.verio.net [198.65.168.24]) (authenticated bits=0) by gritton.org (8.13.6.20060614/8.13.6) with ESMTP id m59Ln0MY006243; Mon, 9 Jun 2008 15:49:00 -0600 (MDT) Message-ID: <484DA546.9060005@gritton.org> Date: Mon, 09 Jun 2008 15:48:54 -0600 From: James Gritton User-Agent: Thunderbird 2.0.0.9 (X11/20080228) MIME-Version: 1.0 To: Julian Elischer References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> In-Reply-To: <484D8EDD.3040103@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.92.1/7416/Mon Jun 9 15:10:04 2008 on gritton.org X-Virus-Status: Clean Cc: freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 21:49:10 -0000 Could we have a list of what isn't expected to actually commit? So the scheduler stuff is out. Is that all of the struct vcpu? Parts of struct vprocg? I see some scheduling bits in both. Aside from vnet/vinet and the doomed scheduling bits, I see not much besides the hostname, domain name, and morphing symlinks. Are these staying? The hostname is already in jails ,and the domainname makes sense in my new jail framework - the morphing symlinks might be something best left for later. Ideally, for integration purposes, the vnet/vinet would hang off jails that have pretty much the same capability as the vimage structure, and then other bits could be added later. I don't want to worry about trying to integrate features that aren't in the final cut anyway. - Jamie Julian Elischer wrote: > ... > > This diff there includes experimental changes to virtualise things > like load average, and they will not be part of the commit. > so ignore anything that smells like "scheduler" From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 22:15:36 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EAE9106564A for ; Mon, 9 Jun 2008 22:15:36 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outZ.internet-mail-service.net (outz.internet-mail-service.net [216.240.47.249]) by mx1.freebsd.org (Postfix) with ESMTP id 78E4C8FC19 for ; Mon, 9 Jun 2008 22:15:36 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 3881C24B6; Mon, 9 Jun 2008 15:15:36 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 064662D6010; Mon, 9 Jun 2008 15:15:35 -0700 (PDT) Message-ID: <484DAB87.6040706@elischer.org> Date: Mon, 09 Jun 2008 15:15:35 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: James Gritton References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> <484DA546.9060005@gritton.org> In-Reply-To: <484DA546.9060005@gritton.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 22:15:36 -0000 James Gritton wrote: > Could we have a list of what isn't expected to actually commit? So the > scheduler stuff is out. Is that all of the struct vcpu? Parts of > struct vprocg? I see some scheduling bits in both. > > Aside from vnet/vinet and the doomed scheduling bits, I see not much > besides the hostname, domain name, and morphing symlinks. Are these > staying? The hostname is already in jails ,and the domainname makes > sense in my new jail framework - the morphing symlinks might be > something best left for later. domain name and hostname both stay.. Hostname is tricky because both jail and vimage expect to change it.. though jail only really expects it to be virtualised to the user rather than REALLY VIRTUALISED. The morphing symlinks are an experimental feature. The verio guys have some work in that direction too that they want to work on.... hmmm that's not you is it? Loadavg etc. is not "out for ever" just "not in the first commit set." as they have not been extensively tested, and probably need more work. > > Ideally, for integration purposes, the vnet/vinet would hang off jails > that have pretty much the same capability as the vimage structure, and > then other bits could be added later. I don't want to worry about > trying to integrate features that aren't in the final cut anyway. the aim is that vimage and jail structures would merge. as the for "final cut", the schedule only covers initial commits of the vnet code, but once the framework is in place more functionality would be added. > > - Jamie > > > Julian Elischer wrote: >> ... >> >> This diff there includes experimental changes to virtualise things >> like load average, and they will not be part of the commit. >> so ignore anything that smells like "scheduler" From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 22:20:53 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16B351065675 for ; Mon, 9 Jun 2008 22:20:53 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from gritton.org (gritton.org [161.58.222.4]) by mx1.freebsd.org (Postfix) with ESMTP id B9BA18FC15 for ; Mon, 9 Jun 2008 22:20:52 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from guppy.corp.verio.net (fw.oremut02.us.wh.verio.net [198.65.168.24]) (authenticated bits=0) by gritton.org (8.13.6.20060614/8.13.6) with ESMTP id m59MKoQD009821; Mon, 9 Jun 2008 16:20:51 -0600 (MDT) Message-ID: <484DACBD.50109@gritton.org> Date: Mon, 09 Jun 2008 16:20:45 -0600 From: James Gritton User-Agent: Thunderbird 2.0.0.9 (X11/20080228) MIME-Version: 1.0 To: Julian Elischer References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> <484DA546.9060005@gritton.org> <484DAB87.6040706@elischer.org> In-Reply-To: <484DAB87.6040706@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.92.1/7416/Mon Jun 9 15:10:04 2008 on gritton.org X-Virus-Status: Clean Cc: freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 22:20:53 -0000 Julian Elischer wrote: > James Gritton wrote: >> Could we have a list of what isn't expected to actually commit? So >> the scheduler stuff is out. Is that all of the struct vcpu? Parts >> of struct vprocg? I see some scheduling bits in both. >> >> Aside from vnet/vinet and the doomed scheduling bits, I see not much >> besides the hostname, domain name, and morphing symlinks. Are these >> staying? The hostname is already in jails ,and the domainname makes >> sense in my new jail framework - the morphing symlinks might be >> something best left for later. > > domain name and hostname both stay.. > Hostname is tricky because both jail and vimage expect to change it.. > though jail only really expects it to be virtualised to the user > rather than REALLY VIRTUALISED. I notice there are some differences between the two approaches, and plan to keep the hostname as virtualized as possible. But really, the differences are few and easily merged. > The morphing symlinks are an experimental feature. > The verio guys have some work in that direction too that they want to > work on.... hmmm that's not you is it? Yeah, could be. So while it's a feature I understand and like, I still prefer it remain for later. > Loadavg etc. is not "out for ever" just "not in the first commit set." > as they have not been extensively tested, and probably need more work. > > >> >> Ideally, for integration purposes, the vnet/vinet would hang off >> jails that have pretty much the same capability as the vimage >> structure, and then other bits could be added later. I don't want to >> worry about trying to integrate features that aren't in the final cut >> anyway. > > the aim is that vimage and jail structures would merge. > > as the for "final cut", the schedule only covers initial commits of > the vnet code, but once the framework is in place more functionality > would be added. "Final cut" was a poor choice of words - I too am talking about the first commit that covers the vnet code. - Jamie From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 23:03:30 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A1C31065678 for ; Mon, 9 Jun 2008 23:03:30 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [91.103.162.4]) by mx1.freebsd.org (Postfix) with ESMTP id B8D508FC14 for ; Mon, 9 Jun 2008 23:03:29 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from localhost (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id B9D6519E023; Tue, 10 Jun 2008 00:47:46 +0200 (CEST) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id F2EA119E019; Tue, 10 Jun 2008 00:47:43 +0200 (CEST) Message-ID: <484DB357.80103@quip.cz> Date: Tue, 10 Jun 2008 00:48:55 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 X-Accept-Language: cz, cs, en, en-us MIME-Version: 1.0 To: Julian Elischer References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> <484DA546.9060005@gritton.org> <484DAB87.6040706@elischer.org> In-Reply-To: <484DAB87.6040706@elischer.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 23:03:30 -0000 Julian Elischer wrote: > James Gritton wrote: > >> Could we have a list of what isn't expected to actually commit? So >> the scheduler stuff is out. Is that all of the struct vcpu? Parts of >> struct vprocg? I see some scheduling bits in both. >> >> Aside from vnet/vinet and the doomed scheduling bits, I see not much >> besides the hostname, domain name, and morphing symlinks. Are these >> staying? The hostname is already in jails ,and the domainname makes >> sense in my new jail framework - the morphing symlinks might be >> something best left for later. > > > domain name and hostname both stay.. > Hostname is tricky because both jail and vimage expect to change it.. > though jail only really expects it to be virtualised to the user > rather than REALLY VIRTUALISED. > > The morphing symlinks are an experimental feature. > The verio guys have some work in that direction too that they want to > work on.... hmmm that's not you is it? Are there somebody who can shed some light on "what is planned in Jail / Vimage" for near future? I read many times "we talked about it at the developers summit" or "we will publish them on a wiki", but it is a long time ago and real informations are still kind of secret. I am working on page http://wiki.freebsd.org/Jails and I will be glad to publish as more informations as I can. For example, what is morphing symlinks? Or I know Verio has VPS on FreeBSD with fair-share resource management - are there some plans to have it in the src tree? Are you in contact with them? I think there are many developers working / thinking on some virtualization stuff but diverged and users (potential testers) know almost nothing about this 'work-in-progress'. > Loadavg etc. is not "out for ever" just "not in the first commit set." > as they have not been extensively tested, and probably need more work. What we can expect from loadavg and "scheduler" stuff? >> Ideally, for integration purposes, the vnet/vinet would hang off jails >> that have pretty much the same capability as the vimage structure, and >> then other bits could be added later. I don't want to worry about >> trying to integrate features that aren't in the final cut anyway. > > > the aim is that vimage and jail structures would merge. Are there some good examples of how things "will work" in vimage+jail world? (not from developers view, but from users view) > as the for "final cut", the schedule only covers initial commits of > the vnet code, but once the framework is in place more functionality > would be added. > > >> >> - Jamie >> >> >> Julian Elischer wrote: >> >>> ... >>> >>> This diff there includes experimental changes to virtualise things >>> like load average, and they will not be part of the commit. >>> so ignore anything that smells like "scheduler" Miroslav Lachman From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 23:11:10 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E5481065671 for ; Mon, 9 Jun 2008 23:11:10 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outB.internet-mail-service.net (outb.internet-mail-service.net [216.240.47.225]) by mx1.freebsd.org (Postfix) with ESMTP id 05C1F8FC15 for ; Mon, 9 Jun 2008 23:11:09 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id C787C2415; Mon, 9 Jun 2008 16:11:09 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id A43632D600D; Mon, 9 Jun 2008 16:11:09 -0700 (PDT) Message-ID: <484DB88C.1020403@elischer.org> Date: Mon, 09 Jun 2008 16:11:08 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: James Gritton References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> <484DA546.9060005@gritton.org> <484DAB87.6040706@elischer.org> <484DACBD.50109@gritton.org> In-Reply-To: <484DACBD.50109@gritton.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org Subject: Vimage commit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 23:11:10 -0000 I have another branch in p4, called vimage-commit //repos/projects/vimage-commit/src/sys/... that currently is SUPPOSED to create an identical binary to -current. It contains A LARGE PART of the commits. but only everything that compiles away to nothing if defined that way. I always update vimage and vimage-commit together, so that a diff between the two branches produces all the changes in vimage that do NOT just evaluate to nothing.. There is a perl script in the base called makemeat.pl produces such a diff. so to recap: //repos/projects/vimage/... is the tree that currently contains the full vimage diff. It is derived from -current. In the base directory is update.sh that keeps it merged with -current, and makediff.pl that generates a diff from -current. //repos/projects/vimage-commit/... is the tree that currently contains the partial vimage diff that "evaluates to nothing". It too is derived from -current. In the base directory is update.sh that keeps it merged with -current, and makediff.pl that generates a diff from -current. There is also a makemeat.pl that generates a diff between the two branches. This shows all the interesting stuff. From owner-freebsd-virtualization@FreeBSD.ORG Mon Jun 9 23:30:44 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BDA4F106566B for ; Mon, 9 Jun 2008 23:30:44 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outN.internet-mail-service.net (outn.internet-mail-service.net [216.240.47.237]) by mx1.freebsd.org (Postfix) with ESMTP id A041C8FC1B for ; Mon, 9 Jun 2008 23:30:44 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 4086A2415; Mon, 9 Jun 2008 16:30:44 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 11CFD2D600E; Mon, 9 Jun 2008 16:30:44 -0700 (PDT) Message-ID: <484DBD23.6000501@elischer.org> Date: Mon, 09 Jun 2008 16:30:43 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> <484DA546.9060005@gritton.org> <484DAB87.6040706@elischer.org> <484DB357.80103@quip.cz> In-Reply-To: <484DB357.80103@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-jails@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jun 2008 23:30:44 -0000 Miroslav Lachman wrote: > Julian Elischer wrote: > >> James Gritton wrote: >> >>> Could we have a list of what isn't expected to actually commit? So >>> the scheduler stuff is out. Is that all of the struct vcpu? Parts >>> of struct vprocg? I see some scheduling bits in both. >>> >>> Aside from vnet/vinet and the doomed scheduling bits, I see not much >>> besides the hostname, domain name, and morphing symlinks. Are these >>> staying? The hostname is already in jails ,and the domainname makes >>> sense in my new jail framework - the morphing symlinks might be >>> something best left for later. >> >> >> domain name and hostname both stay.. >> Hostname is tricky because both jail and vimage expect to change it.. >> though jail only really expects it to be virtualised to the user >> rather than REALLY VIRTUALISED. >> >> The morphing symlinks are an experimental feature. >> The verio guys have some work in that direction too that they want to >> work on.... hmmm that's not you is it? > > Are there somebody who can shed some light on "what is planned in Jail / > Vimage" for near future? I read many times "we talked about it at the > developers summit" or "we will publish them on a wiki", but it is a long > time ago and real informations are still kind of secret. > I am working on page http://wiki.freebsd.org/Jails and I will be glad to > publish as more informations as I can. I am sorry that I didn't realise that page existed earlier, and it reminds me that maybe we should be using the jails mailing list if we plan to make this all merged in with jails.. > For example, what is morphing symlinks? Or I know Verio has VPS on > FreeBSD with fair-share resource management - are there some plans to > have it in the src tree? Are you in contact with them? I will try edit the (2 hour) video of the discussion on the topic and get it up on the web asap.. basically we are trying to pull everyone together onto the basis of a merged vimage/jail implementation. It seems a big job, having read your wiki page! > > I think there are many developers working / thinking on some > virtualization stuff but diverged and users (potential testers) know > almost nothing about this 'work-in-progress'. yes I agree. > >> Loadavg etc. is not "out for ever" just "not in the first commit set." >> as they have not been extensively tested, and probably need more work. > > What we can expect from loadavg and "scheduler" stuff? being able for example to see which jails have which load-averages and thus being able to see who is using the resources. Scheduler partitioning is a bigger problem and I wouldn't care to try do it yet.. The Verio guys have some interesting stuff in this direction which may be useful to us but we need to do some more talking. > >>> Ideally, for integration purposes, the vnet/vinet would hang off >>> jails that have pretty much the same capability as the vimage >>> structure, and then other bits could be added later. I don't want to >>> worry about trying to integrate features that aren't in the final cut >>> anyway. >> >> >> the aim is that vimage and jail structures would merge. > > Are there some good examples of how things "will work" in vimage+jail > world? (not from developers view, but from users view) Not such a document at the moment. but I plan to try write some docs over the next couple of weeks.. basically, it is much like being in a jail except that you ahve your own firewall, routing tables and interfaces. You can run your own ipsec tunnels and whatever. You can also set the tcp sysctls yourself (for example) to turn on or off things like mtu discovery or rando portnumbers.. (etc.etc.) you can also make a jail inside your jail if you want to and let it run in a different virtual machine.. with a different IP space etc. which brings up a point which is that we need to provide a way to limit the damage that a vimage based jail can do by assigning Ip addresses.. i.e. we need to add some constraint based on the current jail code's ip constraints to allow a sub jail to only be allowed to set some addresses... We discussed this at BSDCan but didn't resolve it. > From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 00:55:31 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48CDE1065675 for ; Tue, 10 Jun 2008 00:55:31 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from gritton.org (gritton.org [161.58.222.4]) by mx1.freebsd.org (Postfix) with ESMTP id 0A1F88FC12 for ; Tue, 10 Jun 2008 00:55:30 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from glorfindel.gritton.org (c-76-27-80-223.hsd1.ut.comcast.net [76.27.80.223]) (authenticated bits=0) by gritton.org (8.13.6.20060614/8.13.6) with ESMTP id m5A0tT8H029338; Mon, 9 Jun 2008 18:55:29 -0600 (MDT) Message-ID: <484DD100.6030904@gritton.org> Date: Mon, 09 Jun 2008 18:55:28 -0600 From: James Gritton User-Agent: Thunderbird 2.0.0.9 (X11/20080228) MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> <484DA546.9060005@gritton.org> <484DAB87.6040706@elischer.org> <484DB357.80103@quip.cz> In-Reply-To: <484DB357.80103@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.92.1/7416/Mon Jun 9 15:10:04 2008 on gritton.org X-Virus-Status: Clean Cc: freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2008 00:55:31 -0000 Miroslav Lachman wrote: > Are there somebody who can shed some light on "what is planned in Jail > / Vimage" for near future? I read many times "we talked about it at > the developers summit" or "we will publish them on a wiki", but it is > a long time ago and real informations are still kind of secret. > I am working on page http://wiki.freebsd.org/Jails and I will be glad > to publish as more informations as I can. > For example, what is morphing symlinks? Or I know Verio has VPS on > FreeBSD with fair-share resource management - are there some plans to > have it in the src tree? Are you in contact with them? My own place in things is currently not adding features so much as working on an underlying framework where these different things can be added in an extensible and hopefully backward-compatible way. Yes, this comes from talk at the recent develop summit, where there concern (including my own) that there's already a "jail" system and new and improved virtualization should remain part of that. Vimage is where the momentum is to get something actually committed in the near future, so I'm especially gearing to work with that - also, I want to use what it has to offer. As for the Verio stuff (that's me), last I heard there's still lawyers involved in slowing things down, but after there will be two steps. One is just showing what we have to give, and then the next more important step is integrating it with the setup I'm working on now, instead of our own like-jail-but-not-quite configuration. Vimage will be in place by then, so this is something for later, but to plan for now. - Jamie From owner-freebsd-virtualization@FreeBSD.ORG Tue Jun 10 20:45:08 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 64D591065674 for ; Tue, 10 Jun 2008 20:45:08 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.cksoft.de (mail.cksoft.de [62.111.66.27]) by mx1.freebsd.org (Postfix) with ESMTP id 1C2D68FC1B for ; Tue, 10 Jun 2008 20:45:08 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from localhost (amavis.str.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 5580241C75E; Tue, 10 Jun 2008 22:45:06 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([62.111.66.27]) by localhost (amavis.str.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id mfuCYROCpF4u; Tue, 10 Jun 2008 22:45:05 +0200 (CEST) Received: by mail.cksoft.de (Postfix, from userid 66) id D7CCC41C75D; Tue, 10 Jun 2008 22:45:05 +0200 (CEST) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 41CA044487F; Tue, 10 Jun 2008 20:43:02 +0000 (UTC) Date: Tue, 10 Jun 2008 20:43:02 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Julian Elischer In-Reply-To: <484DBD23.6000501@elischer.org> Message-ID: <20080610204001.M83875@maildrop.int.zabbadoz.net> References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <484D8EDD.3040103@elischer.org> <484DA546.9060005@gritton.org> <484DAB87.6040706@elischer.org> <484DB357.80103@quip.cz> <484DBD23.6000501@elischer.org> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-virtualization@freebsd.org, freebsd-jails@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2008 20:45:08 -0000 On Mon, 9 Jun 2008, Julian Elischer wrote: Hi, > I am sorry that I didn't realise that page existed earlier, as most people haven't realised that these wiki pages have exists (for years) ... http://wiki.freebsd.org/Image > and it reminds me that maybe we should be using the jails mailing list if we > plan to make this all merged in with jails.. For a plan for the mailing lists see my very first posting to this one: http://lists.freebsd.org/pipermail/freebsd-virtualization/2008-May/000000.html /bz -- Bjoern A. Zeeb Stop bit received. Insert coin for new game. From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 14:56:43 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81D201065670; Wed, 11 Jun 2008 14:56:43 +0000 (UTC) (envelope-from zec@freebsd.org) Received: from xaqua.tel.fer.hr (xaqua.tel.fer.hr [161.53.19.25]) by mx1.freebsd.org (Postfix) with ESMTP id DE5A18FC12; Wed, 11 Jun 2008 14:56:42 +0000 (UTC) (envelope-from zec@freebsd.org) Received: by xaqua.tel.fer.hr (Postfix, from userid 20006) id 5644D9B653; Wed, 11 Jun 2008 16:23:25 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on xaqua.tel.fer.hr X-Spam-Level: X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.7 Received: from [192.168.200.111] (zec2.tel.fer.hr [161.53.19.79]) by xaqua.tel.fer.hr (Postfix) with ESMTP id 186589B652; Wed, 11 Jun 2008 16:23:17 +0200 (CEST) From: Marko Zec To: "Bjoern A. Zeeb" Date: Wed, 11 Jun 2008 16:22:58 +0200 User-Agent: KMail/1.9.7 References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> In-Reply-To: <20080609174826.Q83875@maildrop.int.zabbadoz.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200806111622.58194.zec@freebsd.org> X-Mailman-Approved-At: Wed, 11 Jun 2008 15:36:30 +0000 Cc: freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2008 14:56:43 -0000 On Monday 09 June 2008 21:24:42 Bjoern A. Zeeb wrote: > On Sun, 8 Jun 2008, Julian Elischer wrote: > > At the BSDCAn devsummit we discussed how to proceed with committing > > Vimage to -current. > > > > the Milestones included something like: > > > > June 8 (today) Headsup.... > > > > June 15 commit changes that add macros for vnet > > (network module) and vinet(inet virtualisation) > > with macros defined in such a way to make 0 actual > > differences. provable by md5 etc. > > Documentat > > s/hostname/g//V_hostname/ > > #define V_hostname hostname > > 2 weeks settle time, next step prepared, tested > > and reviewed. > > For which part were you talking about a sed/awk script to use? > Can we have a diff for just this part (once it is avail?) There's now a script in p4 projects/vimage/var_rename.tcl - that one is done in TCL, doing it in sed/awk was easier said than done... In projects/vimage/misc/ there's an machine-generated diff plus another small one manually forged that has to be applied afterwards for clean compile. > [schedule] > > * I am missing the BIG HEADS UP somewhere for all the people with > outstanding work so that they will not re-do any integration multiple > times. > > * I am missing the developers and users documentation in the > schedule. Users doc: vimage(8) is not entirely in sync with most recent code but not too misleading either, and there's a reasonably up-to-date cookbook here: http://imunes.net/virtnet/eurobsdcon07_tutorial.pdf Developers doc: yes I know it's a showstopper, it's in a pipeline... > > diffs can be found at: > > http://www.freebsd.org/~julian/vimage.diff and it are usually > > fairly up to date. > > I am just starting to skip through the patch, not doing a close > review atm (not checking functional changes, etc. at all), and even > this is hard at the end of the day... > > Are sys/ddb/db_command.c related in any way to this? No, the change there merely displays the list of possible command completions in a sorted order. This is nothing vimage-specific. > sys/kern/init_main.c has an extra whitespace before the LIST_FIRST. OK. Thanks for reporting this and others style violations bellow, will do a sweeping pass over the vimage branch today. > sys/kern/kern_linker.c Isizeof(lookup) should be 4 space indent not 2 > tabs. > > Do we need those changes like sys/kern/kern_switch.c ? Not really, they were / are experimental, primarily to check out how the virtualization framework can deal with other subsystems unrelated to networking. There's no plan to push for commiting these bits. > sys/kern/kern_sysctl.c has indentation problems in the > @@ -1322,7 +1421,17 @@ junk > > sys/kern/kern_timeout.c has an extra whitespace > > sys/kern/kern_vimage.c says "XXX RCS tag goes here" so add it. > sys/kern/kern_vimage.c has // comments no-no > - " - s,#define NAME,#define\tNAME,g > - " - vnet_mod_register,vnet_mod_register_multi,(more)... > declarations - " - adds a new suser() call. Yes priv() should be used here instead. > - " - in vi_symlookup() 2nsd line of for, remove a space > - " - thinks like this scare me: > /* A brute force check whether there's enough mem for a new vimage > */ especially if its freed again instantly This is a huge problem that needs much work. In general if any kernel subsystem fails to allocate resources at boot time it typically panics. With vimage at any point in time we might be calling those same per-subsystem initialization vectors at run time, but with a running system we need a more gratitious back-out mechanism in resource shortages than panics. So all of the existing initialization functions will have to be extended to potentially return an error, and we'll have to extend the virtualization framework to roll back partially initialized vnets in cases of such failures. > - " - near Detach / free per-module state instances remove > whitespace - " - vi_free() remove a \t before the break; > - " - db_show_vnets should probably check for db_pager_quit OK > sys/kern/kern_xxx.c the printf looks like debugging? yup > sys/kern/sys_socket.c has an unrelated whitespace change > > sys/kern/uipc_domain.c removes a comment I am entirely sure it can be > removed. - " - why do we need to change net_init_domain(?here?) just > to cast again? OK the original comment can be restored. Casting: net_init_domain() is now of vnet_attach_fn type which passes generic void *arg from the caller, this arg may be something other than struct domain * in other cases. The idea is that we can register a single initialization function multiple times with different arguments (different struct domain * in this case). When a new vnet gets instantiated, the vimage framework takes care to invoke each registration of net_init_domain() with proper struct domain *arg in the proper order. > sys/kern/uipc_socket.c junk @@ -1284,13 +1314,17 @@ s,\t, , > - " - if (how != SHUT_RD) { int error; add \n OK true > sys/kern/vfs_lookup.c adds something called IMUNES_SYMLINK_HACK which > should either be renamed or removed. Yes as the name implies this is a HACK not intended to be commited. > sys/modules/Makefile does not look like it belongs there Hm I had some issues compiling zfs module long time ago so disabled this, will see if zfs + VIMAGE can be compiled now. > sys/modules/netgraph/Makefile looks really strange, can we fix > that? This was my best shot hack at compiling ng_wormhole only if options VIMAGE is defined. ng_wormhole makes no sense and doesn't even compile on non-vimage kernels - it provides an explicit "tunnel" from one vnet to another at negraph layer. > sys/modules/netgraph/pipe/Makefile has an extra space > > sys/modules/netgraph/wormhole/Makefile has an extra space > > sys/net/bpf.c adds an IMUNES_BPF_HACK, and defines it - either > rename or remove; also has whitespace issues and debugging > printfs in there (that should not compile). A HACK -> not to be commited... > sys/net/if.c @@ -292,31 +317,73 @@ junk if (IS_DEFAULT_VNET(curvnet)) > { ... needs an extra \t, no? doesn't look nice; there are more of > those in this file; maybe not yet; not before the #ifdefs go. - " - > SYSINT .. if_attachdomain was a wrong ws change > - " - junk @@ -1842,6 +1971,24 @@ adds another suser() > - " - at the end there are two unrelated/wrong ws changes > > sys/net/if_ethersubr.c ether_reassign() has whitespace issues > - " - SYSCTL_V_INT for ether_ipfw 2nd line indent looks wrong > > sys/net/if_gif.c SYSCTL_V_INT 2nd line, parallel_tunnels indent > - " - gifmodevent() empty line wrongly removed > > sys/net/if_gif.h #define\tNAME > > sys/net/if_gre.c is there a reason to rename the local variables? ip_id is a global, and if_gre reuses the same name as local, hence renaming to gre_ip_id was done to reduce ambiguity in face of automated variable renaming scripts. Not sure about ip_tos -> gre_ip_tos to be honest... > sys/net/if_loop.c I cannot see a difference for vnet_loif_iattach > w/ or w/o the #ifdef. Should the outer one go? Hmm the difference is clearly visible to me, are we looking at the same code chunk? We could probably resolve IS_DEFAULT_VNET() to always true for non-vimage kernels though... +static int vnet_loif_iattach(unused) + const void *unused; +{ + INIT_VNET_NET(curvnet); + + LIST_INIT(&V_lo_list); +#ifdef VIMAGE + if (IS_DEFAULT_VNET(curvnet)) + if_clone_attach(&lo_cloner); + else + lo_cloner.ifc_attach(&lo_cloner); +#else + if_clone_attach(&lo_cloner); +#endif + return 0; +} > - " - is there a need to move the loif check up in > lo_clone_destroy? - " - junk @@ -190,7 +266,7 @@ use 4 spaces Hmm: static void lo_clone_destroy(struct ifnet *ifp) { struct lo_softc *sc; +#ifdef INVARIANTS + INIT_VNET_NET(ifp->if_vnet); +#endif sc = ifp->if_softc; /* XXX: destroying lo0 will lead to panics. */ KASSERT(V_loif != ifp, ("%s: destroying lo0", __func__)); + mtx_lock(&lo_mtx); + LIST_REMOVE(sc, sc_next); + mtx_unlock(&lo_mtx); bpfdetach(ifp); if_detach(ifp); if_free(ifp); What excatly do you see as problematic there? > sys/net/if_mib.c SYSCTL_V_INT fix ws > > sys/net/if_var.h do we need to move if_index? Yes each vnet should have a private if_index if that was the question? > sys/net/route.c static uma_zone_t rtzone; has an uneeded ws change > - " - rtable_init() ws wrong > - " - is that realted to more MRT changes or why are functions > split and shuffled? route_init() is called only once at boot time, and rtable_init() on each vnet instantiation. In nooptions VIMAGE configs route_init() hence directly calls rtable_init(). > - " - there were and still are more ws problems around > V_rt_tables - " - return 0; ws problem > - " - rtable_idetach() ws problem and more and the return > > sys/net/rtsock.c rnh =\n ... whitespace next line > > sys/net/vnet.h XXX RCS tag goes here do so > - " - struct vnet_net has ws issues with the _ether_ipfw line > - " - #define\tNAME > > > > I am running out of battery, so I am going to continue with the > next ~20%+- in sys/net80211/**, l 6556 tomorrow. > > > General: values in return statements should be enclosed in > parentheses. > > General: function declarations K&R vs. ANSI vs ... > > General: you are adding 92 lines with XXX, 18 say "locking", 2 say > WRONG, 10 say RCS, (other), ... can we get (most of) them fixed > before committing? (fixed, not removed) OK thanks for looking at the code, will do a sweeping round... Cheers, Marko From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 22:57:18 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0628A1065670 for ; Wed, 11 Jun 2008 22:57:18 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outB.internet-mail-service.net (outb.internet-mail-service.net [216.240.47.225]) by mx1.freebsd.org (Postfix) with ESMTP id DB8FA8FC15 for ; Wed, 11 Jun 2008 22:57:17 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 3377B24A6; Wed, 11 Jun 2008 15:57:18 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 1B4D72D6013; Wed, 11 Jun 2008 15:57:17 -0700 (PDT) Message-ID: <4850584C.1070109@elischer.org> Date: Wed, 11 Jun 2008 15:57:16 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: Marko Zec References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <200806111622.58194.zec@freebsd.org> In-Reply-To: <200806111622.58194.zec@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "Bjoern A. Zeeb" , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2008 22:57:18 -0000 Marko Zec wrote: > On Monday 09 June 2008 21:24:42 Bjoern A. Zeeb wrote: >> On Sun, 8 Jun 2008, Julian Elischer wrote: >>> At the BSDCAn devsummit we discussed how to proceed with committing >>> Vimage to -current. >>> >>> the Milestones included something like: >>> >>> June 8 (today) Headsup.... >>> >>> June 15 commit changes that add macros for vnet >>> (network module) and vinet(inet virtualisation) >>> with macros defined in such a way to make 0 actual >>> differences. provable by md5 etc. >>> Documentat >>> s/hostname/g//V_hostname/ >>> #define V_hostname hostname >>> 2 weeks settle time, next step prepared, tested >>> and reviewed. >> For which part were you talking about a sed/awk script to use? >> Can we have a diff for just this part (once it is avail?) > > There's now a script in p4 projects/vimage/var_rename.tcl - that one is > done in TCL, doing it in sed/awk was easier said than done... > In projects/vimage/misc/ there's an machine-generated diff plus another > small one manually forged that has to be applied afterwards for clean > compile. for compile of LINT some more were needed.. I have checked the output of that script and all fixups into: //repos/projects/vimage-commit2/... LINT compiles in that tree. It's probably what will be committed for the first step. > >> [schedule] >> >> * I am missing the BIG HEADS UP somewhere for all the people with >> outstanding work so that they will not re-do any integration multiple >> times. hmm you seem to be responding to it. so I don't see how you missed it.. >> >> * I am missing the developers and users documentation in the >> schedule. Docs are on the way. > > Users doc: vimage(8) is not entirely in sync with most recent code but > not too misleading either, and there's a reasonably up-to-date cookbook > here: http://imunes.net/virtnet/eurobsdcon07_tutorial.pdf > > Developers doc: yes I know it's a showstopper, it's in a pipeline... > >>> diffs can be found at: >>> http://www.freebsd.org/~julian/vimage.diff and it are usually >>> fairly up to date. >> I am just starting to skip through the patch, not doing a close >> review atm (not checking functional changes, etc. at all), and even >> this is hard at the end of the day... >> >> Are sys/ddb/db_command.c related in any way to this? > > No, the change there merely displays the list of possible command > completions in a sorted order. This is nothing vimage-specific. > >> sys/kern/init_main.c has an extra whitespace before the LIST_FIRST. > > OK. Thanks for reporting this and others style violations bellow, will > do a sweeping pass over the vimage branch today. > >> sys/kern/kern_linker.c Isizeof(lookup) should be 4 space indent not 2 >> tabs. >> >> Do we need those changes like sys/kern/kern_switch.c ? > > Not really, they were / are experimental, primarily to check out how the > virtualization framework can deal with other subsystems unrelated to > networking. There's no plan to push for commiting these bits. > >> sys/kern/kern_sysctl.c has indentation problems in the >> @@ -1322,7 +1421,17 @@ junk >> >> sys/kern/kern_timeout.c has an extra whitespace >> >> sys/kern/kern_vimage.c says "XXX RCS tag goes here" so add it. >> sys/kern/kern_vimage.c has // comments no-no >> - " - s,#define NAME,#define\tNAME,g >> - " - vnet_mod_register,vnet_mod_register_multi,(more)... >> declarations - " - adds a new suser() call. > > Yes priv() should be used here instead. > >> - " - in vi_symlookup() 2nsd line of for, remove a space >> - " - thinks like this scare me: >> /* A brute force check whether there's enough mem for a new vimage >> */ especially if its freed again instantly > > This is a huge problem that needs much work. In general if any kernel > subsystem fails to allocate resources at boot time it typically panics. > With vimage at any point in time we might be calling those same > per-subsystem initialization vectors at run time, but with a running > system we need a more gratitious back-out mechanism in resource > shortages than panics. So all of the existing initialization functions > will have to be extended to potentially return an error, and we'll have > to extend the virtualization framework to roll back partially > initialized vnets in cases of such failures. fixing all the init routines is a task that will take time and it will be something people will be fixing long after this commit is done. It is not however something to stop this work. If you want to not use vimage, things are as before. > >> - " - near Detach / free per-module state instances remove >> whitespace - " - vi_free() remove a \t before the break; >> - " - db_show_vnets should probably check for db_pager_quit > > OK > >> sys/kern/kern_xxx.c the printf looks like debugging? > > yup > >> sys/kern/sys_socket.c has an unrelated whitespace change >> >> sys/kern/uipc_domain.c removes a comment I am entirely sure it can be >> removed. - " - why do we need to change net_init_domain(?here?) just >> to cast again? > > OK the original comment can be restored. Casting: net_init_domain() is > now of vnet_attach_fn type which passes generic void *arg from the > caller, this arg may be something other than struct domain * in other > cases. > > The idea is that we can register a single initialization function > multiple times with different arguments (different struct domain * in > this case). When a new vnet gets instantiated, the vimage framework > takes care to invoke each registration of net_init_domain() with proper > struct domain *arg in the proper order. > >> sys/kern/uipc_socket.c junk @@ -1284,13 +1314,17 @@ s,\t, , >> - " - if (how != SHUT_RD) { int error; add \n > > OK true > >> sys/kern/vfs_lookup.c adds something called IMUNES_SYMLINK_HACK which >> should either be renamed or removed. > > Yes as the name implies this is a HACK not intended to be commited. > >> sys/modules/Makefile does not look like it belongs there > > Hm I had some issues compiling zfs module long time ago so disabled > this, will see if zfs + VIMAGE can be compiled now. > >> sys/modules/netgraph/Makefile looks really strange, can we fix >> that? > > This was my best shot hack at compiling ng_wormhole only if options > VIMAGE is defined. ng_wormhole makes no sense and doesn't even compile > on non-vimage kernels - it provides an explicit "tunnel" from one vnet > to another at netgraph layer. > >> sys/modules/netgraph/pipe/Makefile has an extra space >> >> sys/modules/netgraph/wormhole/Makefile has an extra space >> >> sys/net/bpf.c adds an IMUNES_BPF_HACK, and defines it - either >> rename or remove; also has whitespace issues and debugging >> printfs in there (that should not compile). > > A HACK -> not to be commited... the vimage branch is the 'head of development branch. From that we are extracting actual commit branches... the first commit will come from teh branch I mentioned at the top I think. the second commit will probably come from that same branch after some more changes have been moved to it.. At least that is how I'm planning it. Your comments are all however, valid.. > >> sys/net/if.c @@ -292,31 +317,73 @@ junk if (IS_DEFAULT_VNET(curvnet)) >> { ... needs an extra \t, no? doesn't look nice; there are more of >> those in this file; maybe not yet; not before the #ifdefs go. - " - >> SYSINT .. if_attachdomain was a wrong ws change >> - " - junk @@ -1842,6 +1971,24 @@ adds another suser() >> - " - at the end there are two unrelated/wrong ws changes >> >> sys/net/if_ethersubr.c ether_reassign() has whitespace issues >> - " - SYSCTL_V_INT for ether_ipfw 2nd line indent looks wrong >> >> sys/net/if_gif.c SYSCTL_V_INT 2nd line, parallel_tunnels indent >> - " - gifmodevent() empty line wrongly removed >> >> sys/net/if_gif.h #define\tNAME >> >> sys/net/if_gre.c is there a reason to rename the local variables? > > ip_id is a global, and if_gre reuses the same name as local, hence > renaming to gre_ip_id was done to reduce ambiguity in face of automated > variable renaming scripts. Not sure about ip_tos -> gre_ip_tos to be > honest... > >> sys/net/if_loop.c I cannot see a difference for vnet_loif_iattach >> w/ or w/o the #ifdef. Should the outer one go? > > Hmm the difference is clearly visible to me, are we looking at the same > code chunk? We could probably resolve IS_DEFAULT_VNET() to always true > for non-vimage kernels though... > > +static int vnet_loif_iattach(unused) > + const void *unused; > +{ > + INIT_VNET_NET(curvnet); > + > + LIST_INIT(&V_lo_list); > +#ifdef VIMAGE > + if (IS_DEFAULT_VNET(curvnet)) > + if_clone_attach(&lo_cloner); > + else > + lo_cloner.ifc_attach(&lo_cloner); > +#else > + if_clone_attach(&lo_cloner); > +#endif > + return 0; > +} > >> - " - is there a need to move the loif check up in >> lo_clone_destroy? - " - junk @@ -190,7 +266,7 @@ use 4 spaces > > Hmm: > > static void > lo_clone_destroy(struct ifnet *ifp) > { > struct lo_softc *sc; > +#ifdef INVARIANTS > + INIT_VNET_NET(ifp->if_vnet); > +#endif > > sc = ifp->if_softc; > > /* XXX: destroying lo0 will lead to panics. */ > KASSERT(V_loif != ifp, ("%s: destroying lo0", __func__)); > > + mtx_lock(&lo_mtx); > + LIST_REMOVE(sc, sc_next); > + mtx_unlock(&lo_mtx); > bpfdetach(ifp); > if_detach(ifp); > if_free(ifp); > > What excatly do you see as problematic there? > >> sys/net/if_mib.c SYSCTL_V_INT fix ws >> >> sys/net/if_var.h do we need to move if_index? > > Yes each vnet should have a private if_index if that was the question? > >> sys/net/route.c static uma_zone_t rtzone; has an uneeded ws change >> - " - rtable_init() ws wrong >> - " - is that realted to more MRT changes or why are functions >> split and shuffled? > > route_init() is called only once at boot time, and rtable_init() on each > vnet instantiation. In nooptions VIMAGE configs route_init() hence > directly calls rtable_init(). > >> - " - there were and still are more ws problems around >> V_rt_tables - " - return 0; ws problem >> - " - rtable_idetach() ws problem and more and the return >> >> sys/net/rtsock.c rnh =\n ... whitespace next line >> >> sys/net/vnet.h XXX RCS tag goes here do so >> - " - struct vnet_net has ws issues with the _ether_ipfw line >> - " - #define\tNAME >> >> >> >> I am running out of battery, so I am going to continue with the >> next ~20%+- in sys/net80211/**, l 6556 tomorrow. >> >> >> General: values in return statements should be enclosed in >> parentheses. >> >> General: function declarations K&R vs. ANSI vs ... >> >> General: you are adding 92 lines with XXX, 18 say "locking", 2 say >> WRONG, 10 say RCS, (other), ... can we get (most of) them fixed >> before committing? (fixed, not removed) > > OK thanks for looking at the code, will do a sweeping round... > > Cheers, > > Marko > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 12 16:49:20 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0775B1065688 for ; Thu, 12 Jun 2008 16:49:20 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outR.internet-mail-service.net (outr.internet-mail-service.net [216.240.47.241]) by mx1.freebsd.org (Postfix) with ESMTP id EA0388FC25 for ; Thu, 12 Jun 2008 16:49:19 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id F284524D0; Thu, 12 Jun 2008 09:49:34 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 452F62D6015; Thu, 12 Jun 2008 09:49:19 -0700 (PDT) Message-ID: <4851538E.7030004@elischer.org> Date: Thu, 12 Jun 2008 09:49:18 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: James Gritton References: <200806120757.m5C7vm90085356@repoman.freebsd.org> <48514C4B.4070709@gritton.org> In-Reply-To: <48514C4B.4070709@gritton.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org Subject: Re: PERFORCE change 143354 for review X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 16:49:20 -0000 James Gritton wrote: > Julian Elischer wrote: >> Use the real hostname for dumps >> it has been suggested that we define a R_hostname to mean Real >> hostname >> > > Another option, particularly for hostname, is just to leave the > "hostname" global variable. Right now both jail and vimage hostnames > are fixed arrays in their structures, but I'm considering going to > pointers instead (as a jail may or may not have a virtual hostname). > Then the "root jail" could just point to the static hostname[] array > which can continue to exist under its own name. > > It could be that hostname is the special case here, as far as having a > significant number of "global" references. Load average may have that > case too, but I'm not sure yet. Load average would be later on.. we are not planning on doing htat in this round. but you are right.. it is something where it makes sense in both global and local scope. Since vimage is hierarchical, I think however that it makes sense if the load average of a vimage includes the load average of children. thus the load average of the root vimage would be that of the whole machine. just an idea. > > - Jamie From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 12 21:43:10 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 664) id 82BEC1065670; Thu, 12 Jun 2008 21:43:10 +0000 (UTC) Date: Thu, 12 Jun 2008 21:43:10 +0000 From: David O'Brien To: Marko Zec Message-ID: <20080612214310.GA55195@hub.freebsd.org> References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <200806111622.58194.zec@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200806111622.58194.zec@freebsd.org> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 6.3-STABLE Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 X-Mailman-Approved-At: Thu, 12 Jun 2008 21:58:55 +0000 Cc: "Bjoern A. Zeeb" , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: obrien@freebsd.org List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 21:43:10 -0000 On Wed, Jun 11, 2008 at 04:22:58PM +0200, Marko Zec wrote: > There's now a script in p4 projects/vimage/var_rename.tcl - that one is > done in TCL, doing it in sed/awk was easier said than done... Please post a fetch(1)able URL to all the conversion files a 3rd party FreeBSD user needs to make the same changes on their source base. thanks, -- -- David (obrien@FreeBSD.org) From owner-freebsd-virtualization@FreeBSD.ORG Thu Jun 12 23:10:32 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1E6E106567A for ; Thu, 12 Jun 2008 23:10:32 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outX.internet-mail-service.net (outx.internet-mail-service.net [216.240.47.247]) by mx1.freebsd.org (Postfix) with ESMTP id 866AF8FC18 for ; Thu, 12 Jun 2008 23:10:32 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 7C9D224BE; Thu, 12 Jun 2008 16:10:41 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 61F2E2D6014; Thu, 12 Jun 2008 16:10:31 -0700 (PDT) Message-ID: <4851ACE7.2070505@elischer.org> Date: Thu, 12 Jun 2008 16:10:31 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: obrien@freebsd.org References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <200806111622.58194.zec@freebsd.org> <20080612214310.GA55195@hub.freebsd.org> In-Reply-To: <20080612214310.GA55195@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "Bjoern A. Zeeb" , Marko Zec , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 23:10:32 -0000 David O'Brien wrote: > On Wed, Jun 11, 2008 at 04:22:58PM +0200, Marko Zec wrote: >> There's now a script in p4 projects/vimage/var_rename.tcl - that one is >> done in TCL, doing it in sed/awk was easier said than done... > > Please post a fetch(1)able URL to all the conversion files a 3rd party > FreeBSD user needs to make the same changes on their source base. I sent you one but you apparently didn't believe the fact that I included a trace of fetch actually doing it.. the script is not perfect however you have to go over the output and check that it hasn't converted 1) local variables with the same name as the globals in question. 2) In one case I saw "hostname" become "V_hostname" so there is something not quite right there. > > thanks, From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 13 16:21:45 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C2C7106566C; Fri, 13 Jun 2008 16:21:45 +0000 (UTC) (envelope-from obrien@NUXI.org) Received: from dragon.nuxi.org (trang.nuxi.org [74.95.12.85]) by mx1.freebsd.org (Postfix) with ESMTP id 610EA8FC1B; Fri, 13 Jun 2008 16:21:45 +0000 (UTC) (envelope-from obrien@NUXI.org) Received: from dragon.nuxi.org (obrien@localhost [127.0.0.1]) by dragon.nuxi.org (8.14.2/8.14.2) with ESMTP id m5DGBZ5f086118; Fri, 13 Jun 2008 09:11:35 -0700 (PDT) (envelope-from obrien@dragon.nuxi.org) Received: (from obrien@localhost) by dragon.nuxi.org (8.14.2/8.14.2/Submit) id m5DGBYoA086117; Fri, 13 Jun 2008 09:11:34 -0700 (PDT) (envelope-from obrien) Date: Fri, 13 Jun 2008 09:11:34 -0700 From: "David O'Brien" To: Julian Elischer Message-ID: <20080613161134.GA86001@dragon.NUXI.org> References: <484CC690.9020303@elischer.org> <20080609174826.Q83875@maildrop.int.zabbadoz.net> <200806111622.58194.zec@freebsd.org> <20080612214310.GA55195@hub.freebsd.org> <4851ACE7.2070505@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4851ACE7.2070505@elischer.org> X-Operating-System: FreeBSD 8.0-CURRENT User-Agent: Mutt/1.5.16 (2007-06-09) X-Mailman-Approved-At: Fri, 13 Jun 2008 16:30:54 +0000 Cc: "Bjoern A. Zeeb" , Marko Zec , freebsd-virtualization@freebsd.org Subject: Re: kinda headsup.. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: obrien@freebsd.org List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2008 16:21:45 -0000 On Thu, Jun 12, 2008 at 04:10:31PM -0700, Julian Elischer wrote: > David O'Brien wrote: >> On Wed, Jun 11, 2008 at 04:22:58PM +0200, Marko Zec wrote: >>> There's now a script in p4 projects/vimage/var_rename.tcl - that one is >>> done in TCL, doing it in sed/awk was easier said than done... >> Please post a fetch(1)able URL to all the conversion files a 3rd party >> FreeBSD user needs to make the same changes on their source base. > > I sent you one but you apparently didn't believe the fact that I > included a trace of fetch actually doing it.. You mentioned a URL on IRC. That is very different from putting it in an email in a thread about reviewing this work. Also from the looks of http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects/vimage/... folks should get var_rename.tcl and vimage_globals. On IRC you only gave an example for one file. -- -- David (obrien@FreeBSD.org) From owner-freebsd-virtualization@FreeBSD.ORG Fri Jun 13 19:59:07 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 886AA1065673 for ; Fri, 13 Jun 2008 19:59:07 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from gritton.org (gritton.org [161.58.222.4]) by mx1.freebsd.org (Postfix) with ESMTP id 390CF8FC0C for ; Fri, 13 Jun 2008 19:59:06 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from guppy.corp.verio.net (fw.oremut02.us.wh.verio.net [198.65.168.24]) (authenticated bits=0) by gritton.org (8.13.6.20060614/8.13.6) with ESMTP id m5DJx6Zp022056 for ; Fri, 13 Jun 2008 13:59:06 -0600 (MDT) Message-ID: <4852D184.9060301@gritton.org> Date: Fri, 13 Jun 2008 13:59:00 -0600 From: James Gritton User-Agent: Thunderbird 2.0.0.9 (X11/20080228) MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.92.1/7427/Tue Jun 10 14:09:35 2008 on gritton.org X-Virus-Status: Clean Subject: V_rootvnode X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2008 19:59:07 -0000 In merging vimage into jails, there's one field prison has that vimage lacks: pr_root. In planning for the merge, it would make sense to treat rootvnode like the pr_root of the base prison, i.e. use a V_rootvnode macro. As with the hostname, there are many places in the kernel where there's no reasonable context for anything other than the real root vnode. So in addition to the V_rootvnode, we'd want a G_rootvnode like the recently introduced G_hostname. - Jamie From owner-freebsd-virtualization@FreeBSD.ORG Sat Jun 14 07:00:42 2008 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D34D1065670 for ; Sat, 14 Jun 2008 07:00:42 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outU.internet-mail-service.net (outu.internet-mail-service.net [216.240.47.244]) by mx1.freebsd.org (Postfix) with ESMTP id F0D6C8FC1B for ; Sat, 14 Jun 2008 07:00:41 +0000 (UTC) (envelope-from julian@elischer.org) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 8C8712439; Sat, 14 Jun 2008 00:00:41 -0700 (PDT) Received: from julian-mac.elischer.org (localhost [127.0.0.1]) by idiom.com (Postfix) with ESMTP id 19E672D6018; Sat, 14 Jun 2008 00:00:41 -0700 (PDT) Message-ID: <48536C9A.8020801@elischer.org> Date: Sat, 14 Jun 2008 00:00:42 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421) MIME-Version: 1.0 To: FreeBSD Current , freebsd-virtualization@freebsd.org, Marko Zec References: <484CC690.9020303@elischer.org> In-Reply-To: <484CC690.9020303@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Vimage headsup.. revised. X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Jun 2008 07:00:42 -0000 Julian Elischer wrote: > At the BSDCAn devsummit we discussed how to proceed with committing > Vimage to -current. > > the Milestones included something like: However I think we need to delay by at least a week. several issues have come up, none of them critical, but together, limiting the amount of testing and tweeking that we have been able to do. Also some interesting work has turned up that may be relevent at the later stages WRT cobining vimage and jails. SO, I think we should add 2 weeks to each of these dates. While I'd love to push ahead, commmon sense says that we will probably regret rushing in too fast. > > June 8 (today) Headsup.... > > June 15 commit changes that add macros for vnet > (network module) and vinet(inet virtualisation) > with macros defined in such a way to make 0 actual > differences. provable by md5 etc. > Documentat > s/hostname/g//V_hostname/ > #define V_hostname hostname > 2 weeks settle time, next step prepared, tested > and reviewed. > > June 29 Add changes to convert all globals to members of > per-module structures. Done in a reversible way > (i.e. compilable out). Macros defined so that > depending on compile options structures or globals > are used (one global structure). > Performance implications of using structures are > evaluated. Structures possibly tuned. > Initialisation routines added, checked and tuned. > example: > #if VIMAGE_USE_STRUCTS > #define V_hostname sys_globals.hostname > ... > #else > #define V_hostname hostname > ... > #endif > > > July 13 globals removed in vnet, vinet. > ifdefs and compile option removed or scaled back > to make code clean to read again. > Destructor routines added where needed. > Remaining "NULL Macros" (compile to nothing at this > point) committed to reduce the size of the > MEAT diffs. Review of Meat diffs formally under way > for final comment. > example: > #define INIT_VNET_INET(x) /* nothing */ > add "INIT_VNET_INET(curvnet);"(and similar) > where needed. > remove globals (e.g. 'hostname') > > > July 21 JAIL+Vimage framework committed. > e.g. add new syscall, program, etc. > (part one of meat diffs) structures still only > global instances. vimage inhansed jails can be created > but act jus tlike normal jails? > > July 28 Ability to created > 1 vimage enabled. > Vimage enhanced jails now have private network > stacks etc. > > > August start on converting more modules as needed and time > allows. > > Marko and I have been working towards splitting up the current diffs > (which do the whole thing) so allow this schedule to be followed. > > We may or may not be ready for the June 15 step by then, but if not > it may be a week there-after. So this should be considered the > heads-up. discussion will be on freebsd-virtualization@ > and the perforce branch that we have as a current working system > is branch 'vimage'. //depot/projects/vimage/... > > diffs can be found at: > http://www.freebsd.org/~julian/vimage.diff and it are usually > fairly up to date. > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org"