From owner-freebsd-arch@FreeBSD.ORG Sun Oct 14 08:14:10 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D254B16A419; Sun, 14 Oct 2007 08:14:10 +0000 (UTC) (envelope-from mpp@mppsystems.com) Received: from mail3.gulftel.com (mail3.gulftel.com [216.231.163.44]) by mx1.freebsd.org (Postfix) with ESMTP id A0B7113C43E; Sun, 14 Oct 2007 08:14:10 +0000 (UTC) (envelope-from mpp@mppsystems.com) Received: from localhost (localhost [127.0.0.1]) by mail3.gulftel.com (Postfix) with ESMTP id 84ED02A762; Sun, 14 Oct 2007 02:15:47 -0500 (CDT) X-Virus-Scanned: amavisd-new at gulftel.com Received: from mail3.gulftel.com ([127.0.0.1]) by localhost (mail3.gulftel.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sWsw30n0WPwp; Sun, 14 Oct 2007 02:15:46 -0500 (CDT) Received: from mail.mppsystems.com (unknown [10.17.6.142]) by mail3.gulftel.com (Postfix) with ESMTP id 191422A756; Sun, 14 Oct 2007 02:15:41 -0500 (CDT) Received: by mail.mppsystems.com (Postfix, from userid 1000) id 6466111557; Sun, 14 Oct 2007 02:15:41 -0500 (CDT) Date: Sun, 14 Oct 2007 02:15:41 -0500 From: Mike Pritchard To: Nikolay Pavlov Message-ID: <20071014071541.GA63551@mail.mppsystems.com> References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <200710131021.03861.qpadla@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200710131021.03861.qpadla@gmail.com> User-Agent: mutt-ng/devel-r804 (FreeBSD) Cc: arch@freebsd.org, James Gritton , Marko Zec , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Oct 2007 08:14:10 -0000 On Sat, Oct 13, 2007 at 10:20:58AM +0300, Nikolay Pavlov wrote: > On Friday 12 October 2007 22:54:04 James Gritton wrote: > > Julian Elischer wrote: > > > What I'd like to see is a bit of a 'a-la-carte' virtualisation > > > ability. > > ... > > > My question to you, the reader, is: > > > what aspects of virtualisation (the appearance of multiple instances > > > of some resource) would you like to see in the system? > > > > Filesystem quotas, without the need for each jail to have its own mount > > point. > > Strange, but IMHO it would be better slightly revert this statement: > Filesystem quotas _with_ the need for each jail to have it's own mount > point, but with out the need to maintain them in fstab (Like it is in > ZFS). Because you gain the ability to maintain jails in a filesystem > level(snapshots, cloning, dump, restore and so on). Let me start with, the current quota system requires a mount point to maintain the quota datya file pointer information (currently a UFS mount point), and a pointer in the i-node to the quota struct for that UID/GID And the idea of multiple mount points sharing quotas strikes me as a novel idea right now....Hmm. That concept fits with ZFS better too... I've been working on bumping quotas up a level. Instead of mantaining the data in the file system layer, I have them working at the vnode layer. There is no real reason quotas need to know anything about the data below the vnode layer (e.g. move the i-node info up into the vnode, and the information maintained in the ufsmount struct up into the mount struct). Quotas just needs to know, the ID, its type (uid/gid), and which data file to get/write the limit/usage info from/to and what to +/- that data. My goal on this was to be able to extend quotas to work on non-UFS file systems. I have them 85% or so working on tmpfs right now as my test case. I have never done anything with jails. If someone who knows a little bit about quotas and more about jails wants to get together with me on that, I'm open for guidance. If I'm going to do this major change to quotas, I'd like to be able to make it work in a jailed environment. I think that should be possible (and not sure why they don't work now, since I think they should, but like i said no jail experience here...). And from googling on ZFS, it does sound like there is a need for quotas, even with the ZFS quotas that are available now. Although ZFS looks messy from a quota standpoint. > point, but with out the need to maintain them in fstab (Like it is in > ZFS). Because you gain the ability to maintain jails in a filesystem > level(snapshots, cloning, dump, restore and so on). Our utilities (like quotaon) require that quotas be specified in /etc/fstab. And from comments in the code, SunOS didn't require that. And other than the fact that we allow the admin to specify an alternate location for the quota data files in fstab, there is no reason to not let the commands try. We would just have to add some options to allow the commands to access a different quota data file. If the admin screws up, I guess we let them shoot themselves in the foot. Its easy to add warnings saying: "***** The /sandbox file system does not have quotas enabled in /etc/fstab, continuing with quota database files /sandbox/user.quota, /sandbox/group.quota" I'm sure there are some more issues, but quotas are basically a (very simple) database maintained by the kernel. As long as it can read/write that data for the file system in question, they should work. The biggest issue is finding the correct places to insert the quota update calls in each file system. And from working on the utilities to remove the UFS only restriction, I was thinking it might be desirable to move back to that model from SunOs. Let the commands work and let them fail at the syscall level if quotas are not supported. This would require more work, since each command would now need an option to specify the location of the quota data files, but it is feasable (and not that hard really)... Sorry, I got a bit more in depth there then I intended, but welcome any input or help. -- Mike Pritchard mpp @ FreeBSD.org "If tyranny and oppression come to this land, it will be in the guise of fighting a foreign enemy." - James Madison (1787) From owner-freebsd-arch@FreeBSD.ORG Sun Oct 14 08:14:10 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D254B16A419; Sun, 14 Oct 2007 08:14:10 +0000 (UTC) (envelope-from mpp@mppsystems.com) Received: from mail3.gulftel.com (mail3.gulftel.com [216.231.163.44]) by mx1.freebsd.org (Postfix) with ESMTP id A0B7113C43E; Sun, 14 Oct 2007 08:14:10 +0000 (UTC) (envelope-from mpp@mppsystems.com) Received: from localhost (localhost [127.0.0.1]) by mail3.gulftel.com (Postfix) with ESMTP id 84ED02A762; Sun, 14 Oct 2007 02:15:47 -0500 (CDT) X-Virus-Scanned: amavisd-new at gulftel.com Received: from mail3.gulftel.com ([127.0.0.1]) by localhost (mail3.gulftel.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sWsw30n0WPwp; Sun, 14 Oct 2007 02:15:46 -0500 (CDT) Received: from mail.mppsystems.com (unknown [10.17.6.142]) by mail3.gulftel.com (Postfix) with ESMTP id 191422A756; Sun, 14 Oct 2007 02:15:41 -0500 (CDT) Received: by mail.mppsystems.com (Postfix, from userid 1000) id 6466111557; Sun, 14 Oct 2007 02:15:41 -0500 (CDT) Date: Sun, 14 Oct 2007 02:15:41 -0500 From: Mike Pritchard To: Nikolay Pavlov Message-ID: <20071014071541.GA63551@mail.mppsystems.com> References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <200710131021.03861.qpadla@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200710131021.03861.qpadla@gmail.com> User-Agent: mutt-ng/devel-r804 (FreeBSD) Cc: arch@freebsd.org, James Gritton , Marko Zec , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Oct 2007 08:14:10 -0000 On Sat, Oct 13, 2007 at 10:20:58AM +0300, Nikolay Pavlov wrote: > On Friday 12 October 2007 22:54:04 James Gritton wrote: > > Julian Elischer wrote: > > > What I'd like to see is a bit of a 'a-la-carte' virtualisation > > > ability. > > ... > > > My question to you, the reader, is: > > > what aspects of virtualisation (the appearance of multiple instances > > > of some resource) would you like to see in the system? > > > > Filesystem quotas, without the need for each jail to have its own mount > > point. > > Strange, but IMHO it would be better slightly revert this statement: > Filesystem quotas _with_ the need for each jail to have it's own mount > point, but with out the need to maintain them in fstab (Like it is in > ZFS). Because you gain the ability to maintain jails in a filesystem > level(snapshots, cloning, dump, restore and so on). Let me start with, the current quota system requires a mount point to maintain the quota datya file pointer information (currently a UFS mount point), and a pointer in the i-node to the quota struct for that UID/GID And the idea of multiple mount points sharing quotas strikes me as a novel idea right now....Hmm. That concept fits with ZFS better too... I've been working on bumping quotas up a level. Instead of mantaining the data in the file system layer, I have them working at the vnode layer. There is no real reason quotas need to know anything about the data below the vnode layer (e.g. move the i-node info up into the vnode, and the information maintained in the ufsmount struct up into the mount struct). Quotas just needs to know, the ID, its type (uid/gid), and which data file to get/write the limit/usage info from/to and what to +/- that data. My goal on this was to be able to extend quotas to work on non-UFS file systems. I have them 85% or so working on tmpfs right now as my test case. I have never done anything with jails. If someone who knows a little bit about quotas and more about jails wants to get together with me on that, I'm open for guidance. If I'm going to do this major change to quotas, I'd like to be able to make it work in a jailed environment. I think that should be possible (and not sure why they don't work now, since I think they should, but like i said no jail experience here...). And from googling on ZFS, it does sound like there is a need for quotas, even with the ZFS quotas that are available now. Although ZFS looks messy from a quota standpoint. > point, but with out the need to maintain them in fstab (Like it is in > ZFS). Because you gain the ability to maintain jails in a filesystem > level(snapshots, cloning, dump, restore and so on). Our utilities (like quotaon) require that quotas be specified in /etc/fstab. And from comments in the code, SunOS didn't require that. And other than the fact that we allow the admin to specify an alternate location for the quota data files in fstab, there is no reason to not let the commands try. We would just have to add some options to allow the commands to access a different quota data file. If the admin screws up, I guess we let them shoot themselves in the foot. Its easy to add warnings saying: "***** The /sandbox file system does not have quotas enabled in /etc/fstab, continuing with quota database files /sandbox/user.quota, /sandbox/group.quota" I'm sure there are some more issues, but quotas are basically a (very simple) database maintained by the kernel. As long as it can read/write that data for the file system in question, they should work. The biggest issue is finding the correct places to insert the quota update calls in each file system. And from working on the utilities to remove the UFS only restriction, I was thinking it might be desirable to move back to that model from SunOs. Let the commands work and let them fail at the syscall level if quotas are not supported. This would require more work, since each command would now need an option to specify the location of the quota data files, but it is feasable (and not that hard really)... Sorry, I got a bit more in depth there then I intended, but welcome any input or help. -- Mike Pritchard mpp @ FreeBSD.org "If tyranny and oppression come to this land, it will be in the guise of fighting a foreign enemy." - James Madison (1787) From owner-freebsd-arch@FreeBSD.ORG Sun Oct 14 10:37:22 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EDD1816A418 for ; Sun, 14 Oct 2007 10:37:21 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 5C27413C43E for ; Sun, 14 Oct 2007 10:37:21 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A5780E.dip.t-dialin.net [84.165.120.14]) by redbull.bpaserver.net (Postfix) with ESMTP id 63A972E133; Sun, 14 Oct 2007 12:18:09 +0200 (CEST) Received: from deskjail (deskjail.Leidinger.net [192.168.1.109]) by outgoing.leidinger.net (Postfix) with ESMTP id 2744A5B480D; Sun, 14 Oct 2007 12:17:04 +0200 (CEST) Date: Sun, 14 Oct 2007 12:16:35 +0200 From: Alexander Leidinger To: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <20071014121635.5adc1f19@deskjail> In-Reply-To: <47109F59.30602@quip.cz> References: <470E5BFB.4050903@elischer.org> <47109F59.30602@quip.cz> X-Mailer: Claws Mail 3.0.1 (GTK+ 2.10.14; i686-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-14.3, required 8, BAYES_00 -15.00, J_CHICKENPOX_44 0.60, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Oct 2007 10:37:23 -0000 Quoting Miroslav Lachman <000.fbsd@quip.cz> (Sat, 13 Oct 2007 12:35:05 +0200): > Julian Elischer wrote: > [...] > > I'd like to be able to say.. > > I want to share the filesystem, and unix domain sockets but have a > > separate routing domain for my processes, or maybe just > > for some sockets. But someone else may want to have > > complete separation with everything up to and including > > separate userID spaces. > > > > My question to you, the reader, is: > > what aspects of virtualisation (the appearance of multiple instances > > of some resource) would you like to see in the system? > > > > Even a discussion as to how to frame this question is up for discussion. > > > > We don't even have a taxonomy to discus the issue. > > > > Julian > > It would be nice to have something from vserver, something from zones, > from xen, from jails etc. > From my point of view: > > CPU limits - specified as relative part of shares (container can get > more CPU power if CPU is not 100% loaded) or set to absolute (container > can't get more than specified CPU power, so one can use it to test > applications on slow CPUs etc.) > > Memory limits - same as CPU > > Disk - it would be nice if I can set how many disk space each container > can use. (with similar interface as disk quotas - soft+hard limits and > space+inodes). Maybe setting of disk I/O in similar style as CPU and > memory limits above. You can have something like this already with zfs. Just for information, it doesn't mean we don't need to talk about this point. > UIDs - independent UIDs in containers. In relation to UIDs, one can use > disk quotas inside containers. Can you please clarify what you mean here? Are you talking about the current quota support and how it handles UIDs on the host? If your disk proposal above is implemented, I can imagine that the current quota stuff is independent from this and wouldn't need a decoupling from UIDs in a jail from the UIDs on the host. > Network bandwidth - same as CPU and memory We have this already with dummynet and/or pf, don't we? Bye, Alexander. -- You can press Ctrl-D to quickly exit from a shell, or logout from a login shell. -- Konstantinos Konstantinidis http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Sun Oct 14 13:29:12 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07DB316A41A for ; Sun, 14 Oct 2007 13:29:12 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [82.208.36.70]) by mx1.freebsd.org (Postfix) with ESMTP id A0BF313C442 for ; Sun, 14 Oct 2007 13:29:11 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from localhost (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id E820919E02A; Sun, 14 Oct 2007 15:29:09 +0200 (CEST) Received: from [192.168.1.2] (r3a200.net.upc.cz [213.220.192.200]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTP id 3AF0C19E027; Sun, 14 Oct 2007 15:29:07 +0200 (CEST) Message-ID: <47121A04.9010407@quip.cz> Date: Sun, 14 Oct 2007 15:30:44 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 X-Accept-Language: cz, cs, en, en-us MIME-Version: 1.0 To: Alexander Leidinger References: <470E5BFB.4050903@elischer.org> <47109F59.30602@quip.cz> <20071014121635.5adc1f19@deskjail> In-Reply-To: <20071014121635.5adc1f19@deskjail> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Oct 2007 13:29:12 -0000 Alexander Leidinger wrote: > Quoting Miroslav Lachman <000.fbsd@quip.cz> (Sat, 13 Oct 2007 12:35:05 +0200): [...] >>It would be nice to have something from vserver, something from zones, >>from xen, from jails etc. >> From my point of view: >> >>CPU limits - specified as relative part of shares (container can get >>more CPU power if CPU is not 100% loaded) or set to absolute (container >>can't get more than specified CPU power, so one can use it to test >>applications on slow CPUs etc.) >> >>Memory limits - same as CPU >> >>Disk - it would be nice if I can set how many disk space each container >>can use. (with similar interface as disk quotas - soft+hard limits and >>space+inodes). Maybe setting of disk I/O in similar style as CPU and >>memory limits above. > > > You can have something like this already with zfs. Just for > information, it doesn't mean we don't need to talk about this point. I did not have enough time to play with FreeBSD 7 and ZFS. It is good to know we have it yet. :) >>UIDs - independent UIDs in containers. In relation to UIDs, one can use >>disk quotas inside containers. > > > Can you please clarify what you mean here? Are you talking about the > current quota support and how it handles UIDs on the host? If your disk > proposal above is implemented, I can imagine that the current quota > stuff is independent from this and wouldn't need a decoupling from UIDs > in a jail from the UIDs on the host. Yes I was talking about current quota support na UIDs on host. If I have UID 1001 on host and UID 1001 in two jails on same mountpoint, current quotas can not be used. Or am I wrong? >>Network bandwidth - same as CPU and memory > > > We have this already with dummynet and/or pf, don't we? OK, you are right, one can do this with dummynet or pf in simple jail config, but with hierarchical structer, multiple IPs etc. Will it be still usable? Maybe just implement some layer/utility to wrap around container (jail) settings and generate proper dummynet / pf rules will be enough. Miroslav Lachman From owner-freebsd-arch@FreeBSD.ORG Sun Oct 14 13:42:56 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AC5E16A46C for ; Sun, 14 Oct 2007 13:42:56 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 0EB9613C4A6 for ; Sun, 14 Oct 2007 13:42:55 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A5780E.dip.t-dialin.net [84.165.120.14]) by redbull.bpaserver.net (Postfix) with ESMTP id C93F22E2F5; Sun, 14 Oct 2007 15:42:25 +0200 (CEST) Received: from deskjail (deskjail.Leidinger.net [192.168.1.109]) by outgoing.leidinger.net (Postfix) with ESMTP id A3E875B480D; Sun, 14 Oct 2007 15:41:20 +0200 (CEST) Date: Sun, 14 Oct 2007 15:40:49 +0200 From: Alexander Leidinger To: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <20071014154049.37d84fd5@deskjail> In-Reply-To: <47121A04.9010407@quip.cz> References: <470E5BFB.4050903@elischer.org> <47109F59.30602@quip.cz> <20071014121635.5adc1f19@deskjail> <47121A04.9010407@quip.cz> X-Mailer: Claws Mail 3.0.1 (GTK+ 2.10.14; i686-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-14.9, required 8, BAYES_00 -15.00, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Oct 2007 13:42:56 -0000 Quoting Miroslav Lachman <000.fbsd@quip.cz> (Sun, 14 Oct 2007 15:30:44 +0200): > Alexander Leidinger wrote: > > > Quoting Miroslav Lachman <000.fbsd@quip.cz> (Sat, 13 Oct 2007 12:35:05 +0200): > >>UIDs - independent UIDs in containers. In relation to UIDs, one can use > >>disk quotas inside containers. > > > > > > Can you please clarify what you mean here? Are you talking about the > > current quota support and how it handles UIDs on the host? If your disk > > proposal above is implemented, I can imagine that the current quota > > stuff is independent from this and wouldn't need a decoupling from UIDs > > in a jail from the UIDs on the host. > > Yes I was talking about current quota support na UIDs on host. If I have > UID 1001 on host and UID 1001 in two jails on same mountpoint, current > quotas can not be used. Or am I wrong? It can be used if each jails gets it's own FS. > >>Network bandwidth - same as CPU and memory > > > > > > We have this already with dummynet and/or pf, don't we? > > OK, you are right, one can do this with dummynet or pf in simple jail > config, but with hierarchical structer, multiple IPs etc. Will it be > still usable? Maybe just implement some layer/utility to wrap around > container (jail) settings and generate proper dummynet / pf rules will > be enough. I don't know. Bye, Alexander. -- Anyone stupid enough to be caught by the police is probably guilty. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Sun Oct 14 21:04:12 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1B7316A481 for ; Sun, 14 Oct 2007 21:04:12 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outE.internet-mail-service.net (outE.internet-mail-service.net [216.240.47.228]) by mx1.freebsd.org (Postfix) with ESMTP id A627213C44B for ; Sun, 14 Oct 2007 21:04:12 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Sun, 14 Oct 2007 14:04:12 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id 9108A1266AC; Sun, 14 Oct 2007 14:04:11 -0700 (PDT) Message-ID: <4712845A.3090508@elischer.org> Date: Sun, 14 Oct 2007 14:04:26 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> References: <470E5BFB.4050903@elischer.org> <47109F59.30602@quip.cz> <20071014121635.5adc1f19@deskjail> <47121A04.9010407@quip.cz> In-Reply-To: <47121A04.9010407@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Leidinger , arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Oct 2007 21:04:12 -0000 Miroslav Lachman wrote: > Alexander Leidinger wrote: > >> Quoting Miroslav Lachman <000.fbsd@quip.cz> (Sat, 13 Oct 2007 12:35:05 >> +0200): > > [...] >>> It would be nice to have something from vserver, something from >>> zones, from xen, from jails etc. >>> From my point of view: >>> >>> CPU limits - specified as relative part of shares (container can get >>> more CPU power if CPU is not 100% loaded) or set to absolute >>> (container can't get more than specified CPU power, so one can use it >>> to test applications on slow CPUs etc.) >>> >>> Memory limits - same as CPU >>> >>> Disk - it would be nice if I can set how many disk space each >>> container can use. (with similar interface as disk quotas - soft+hard >>> limits and space+inodes). Maybe setting of disk I/O in similar style >>> as CPU and memory limits above. >> >> >> You can have something like this already with zfs. Just for >> information, it doesn't mean we don't need to talk about this point. > > I did not have enough time to play with FreeBSD 7 and ZFS. It is good to > know we have it yet. :) > >>> UIDs - independent UIDs in containers. In relation to UIDs, one can >>> use disk quotas inside containers. >> >> >> Can you please clarify what you mean here? Are you talking about the >> current quota support and how it handles UIDs on the host? If your disk >> proposal above is implemented, I can imagine that the current quota >> stuff is independent from this and wouldn't need a decoupling from UIDs >> in a jail from the UIDs on the host. > > Yes I was talking about current quota support na UIDs on host. If I have > UID 1001 on host and UID 1001 in two jails on same mountpoint, current > quotas can not be used. Or am I wrong? > >>> Network bandwidth - same as CPU and memory >> >> >> We have this already with dummynet and/or pf, don't we? > > OK, you are right, one can do this with dummynet or pf in simple jail > config, but with hierarchical structer, multiple IPs etc. Will it be > still usable? Maybe just implement some layer/utility to wrap around > container (jail) settings and generate proper dummynet / pf rules will > be enough. in vimage each virtual instance has its own firewalls. > > Miroslav Lachman > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Oct 15 05:59:23 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D5D016A418 for ; Mon, 15 Oct 2007 05:59:23 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from rv-out-0910.google.com (rv-out-0910.google.com [209.85.198.189]) by mx1.freebsd.org (Postfix) with ESMTP id DDABF13C467 for ; Mon, 15 Oct 2007 05:59:22 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: by rv-out-0910.google.com with SMTP id l15so1205467rvb for ; Sun, 14 Oct 2007 22:59:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=mKxqZ4P2tN65eFJ3OpxywgHDpzbYalVH8vLiQLOf75E=; b=LnWv/4IKqJCGCG9zkzzj95DYwoTKAmSy/vpQBCWkYtgz0Tq19VagUVgjoMU/e0T2NvQ96FTGdpk9fC75xOnfZwiFWKLNyjsz/7bXyug4lm9p1L/Buu2mN6oEf4CC0vSFy0NvoIa0JIyk6ri68ZM+2EZ8yMt17oK9ztW0zH72fAI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=RYfz2e0UE+5M5EjJVbHAHNvdFh2Vqtdn+xE2Lb6gNL7lvxr4U5xNTQ9Rx18o7tL7KlQEK3NHUbgNvnH3LMbvgTrb5F4UiHMkUwJcJgcldNV18Pe8M/vDey5i0pqWUVEmTcdtDu8wQXzUjidjgn/OydKBD48ixVtN/C8fDdDV1Zo= Received: by 10.141.211.13 with SMTP id n13mr2510868rvq.1192424256295; Sun, 14 Oct 2007 21:57:36 -0700 (PDT) Received: by 10.140.192.2 with HTTP; Sun, 14 Oct 2007 21:57:36 -0700 (PDT) Message-ID: <2e77fc10710142157k44d76c35m9397cb707c1fc1f1@mail.gmail.com> Date: Mon, 15 Oct 2007 07:57:36 +0300 From: "Niki Denev" Sender: ndenev@gmail.com To: freebsd-arch@freebsd.org In-Reply-To: <4712845A.3090508@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <470E5BFB.4050903@elischer.org> <47109F59.30602@quip.cz> <20071014121635.5adc1f19@deskjail> <47121A04.9010407@quip.cz> <4712845A.3090508@elischer.org> X-Google-Sender-Auth: a378bec253d74846 Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2007 05:59:23 -0000 On 10/15/07, Julian Elischer wrote: > Miroslav Lachman wrote: > > Alexander Leidinger wrote: > > > >> Quoting Miroslav Lachman <000.fbsd@quip.cz> (Sat, 13 Oct 2007 12:35:05 > >> +0200): > > > > [...] > >>> It would be nice to have something from vserver, something from > >>> zones, from xen, from jails etc. > >>> From my point of view: > >>> > >>> CPU limits - specified as relative part of shares (container can get > >>> more CPU power if CPU is not 100% loaded) or set to absolute > >>> (container can't get more than specified CPU power, so one can use it > >>> to test applications on slow CPUs etc.) > >>> > >>> Memory limits - same as CPU > >>> > >>> Disk - it would be nice if I can set how many disk space each > >>> container can use. (with similar interface as disk quotas - soft+hard > >>> limits and space+inodes). Maybe setting of disk I/O in similar style > >>> as CPU and memory limits above. > >> > >> > >> You can have something like this already with zfs. Just for > >> information, it doesn't mean we don't need to talk about this point. > > > > I did not have enough time to play with FreeBSD 7 and ZFS. It is good to > > know we have it yet. :) > > > >>> UIDs - independent UIDs in containers. In relation to UIDs, one can > >>> use disk quotas inside containers. > >> > >> > >> Can you please clarify what you mean here? Are you talking about the > >> current quota support and how it handles UIDs on the host? If your disk > >> proposal above is implemented, I can imagine that the current quota > >> stuff is independent from this and wouldn't need a decoupling from UIDs > >> in a jail from the UIDs on the host. > > > > Yes I was talking about current quota support na UIDs on host. If I have > > UID 1001 on host and UID 1001 in two jails on same mountpoint, current > > quotas can not be used. Or am I wrong? > > > >>> Network bandwidth - same as CPU and memory > >> > >> > >> We have this already with dummynet and/or pf, don't we? > > > > OK, you are right, one can do this with dummynet or pf in simple jail > > config, but with hierarchical structer, multiple IPs etc. Will it be > > still usable? Maybe just implement some layer/utility to wrap around > > container (jail) settings and generate proper dummynet / pf rules will > > be enough. > > in vimage each virtual instance has its own firewalls. > > > > > Miroslav Lachman > > _______________________________________________ > > freebsd-arch@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > What about disk io? right now a single jail can bring the whole system to it's knees because of excessife disk io, which seems to be the bottleneck and the "precious" resource on many systems these days. Just a 0.2cents idea :) From owner-freebsd-arch@FreeBSD.ORG Mon Oct 15 07:16:51 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8D8C16A417; Mon, 15 Oct 2007 07:16:51 +0000 (UTC) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: from digger1.defence.gov.au (digger1.defence.gov.au [203.5.217.4]) by mx1.freebsd.org (Postfix) with ESMTP id 27AD413C457; Mon, 15 Oct 2007 07:16:50 +0000 (UTC) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: from ednmsw510.dsto.defence.gov.au (ednmsw510.dsto.defence.gov.au [131.185.68.11]) by digger1.defence.gov.au (8.13.8/8.13.8) with ESMTP id l9F6pbUq008336; Mon, 15 Oct 2007 16:21:37 +0930 (CST) Received: from fmbex510.dsto.defence.gov.au (fmbex510.dsto.defence.gov.au) by ednmsw510.dsto.defence.gov.au (Clearswift SMTPRS 5.2.9) with ESMTP id ; Mon, 15 Oct 2007 16:33:25 +0930 Received: from stlex510.dsto.defence.gov.au ([203.6.60.184]) by fmbex510.dsto.defence.gov.au with Microsoft SMTPSVC(6.0.3790.1830); Mon, 15 Oct 2007 17:03:24 +1000 Received: from obelix.dsto.defence.gov.au ([203.6.60.208]) by stlex510.dsto.defence.gov.au with Microsoft SMTPSVC(6.0.3790.1830); Mon, 15 Oct 2007 15:03:23 +0800 Received: from obelix.dsto.defence.gov.au (localhost [127.0.0.1]) by obelix.dsto.defence.gov.au (8.14.1/8.14.1) with ESMTP id l9F73Nud008704; Mon, 15 Oct 2007 15:03:23 +0800 (WST) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: (from wilkinsa@localhost) by obelix.dsto.defence.gov.au (8.14.1/8.14.1/Submit) id l9F73NOm008703; Mon, 15 Oct 2007 15:03:23 +0800 (WST) (envelope-from wilkinsa) Date: Mon, 15 Oct 2007 15:03:23 +0800 From: "Wilkinson, Alex" To: freebsd-arch@freebsd.org, arch@freebsd.org Message-ID: <20071015070322.GL4884@obelix.dsto.defence.gov.au> Mail-Followup-To: freebsd-arch@freebsd.org, arch@freebsd.org References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20071013004539.R1002@10.0.0.1> Organisation: Defence Science Technology Organisation User-Agent: Mutt/1.5.16 (2007-06-09) X-OriginalArrivalTime: 15 Oct 2007 07:03:23.0181 (UTC) FILETIME=[79B80DD0:01C80EF9] X-TM-AS-Product-Ver: SMEX-7.0.0.1526-5.0.1023-15480.001 X-TM-AS-Result: No--2.660700-0.000000-31 Content-Transfer-Encoding: 7bit Cc: Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2007 07:16:51 -0000 0n Sat, Oct 13, 2007 at 12:46:58AM -0700, Jeff Roberson wrote: > However, why not just fix Xen? > There may be a very good answer to this, I just don't know it. Well according to Mr libc aka Ulrich Drepper, Xen is a broken piece of crap [http://udrepper.livejournal.com/17577.html] -aW IMPORTANT: This email remains the property of the Australian Defence Organisation and is subject to the jurisdiction of section 70 of the CRIMES ACT 1914. If you have received this email in error, you are requested to contact the sender and delete the email. From owner-freebsd-arch@FreeBSD.ORG Mon Oct 15 07:16:51 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A8D8C16A417; Mon, 15 Oct 2007 07:16:51 +0000 (UTC) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: from digger1.defence.gov.au (digger1.defence.gov.au [203.5.217.4]) by mx1.freebsd.org (Postfix) with ESMTP id 27AD413C457; Mon, 15 Oct 2007 07:16:50 +0000 (UTC) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: from ednmsw510.dsto.defence.gov.au (ednmsw510.dsto.defence.gov.au [131.185.68.11]) by digger1.defence.gov.au (8.13.8/8.13.8) with ESMTP id l9F6pbUq008336; Mon, 15 Oct 2007 16:21:37 +0930 (CST) Received: from fmbex510.dsto.defence.gov.au (fmbex510.dsto.defence.gov.au) by ednmsw510.dsto.defence.gov.au (Clearswift SMTPRS 5.2.9) with ESMTP id ; Mon, 15 Oct 2007 16:33:25 +0930 Received: from stlex510.dsto.defence.gov.au ([203.6.60.184]) by fmbex510.dsto.defence.gov.au with Microsoft SMTPSVC(6.0.3790.1830); Mon, 15 Oct 2007 17:03:24 +1000 Received: from obelix.dsto.defence.gov.au ([203.6.60.208]) by stlex510.dsto.defence.gov.au with Microsoft SMTPSVC(6.0.3790.1830); Mon, 15 Oct 2007 15:03:23 +0800 Received: from obelix.dsto.defence.gov.au (localhost [127.0.0.1]) by obelix.dsto.defence.gov.au (8.14.1/8.14.1) with ESMTP id l9F73Nud008704; Mon, 15 Oct 2007 15:03:23 +0800 (WST) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: (from wilkinsa@localhost) by obelix.dsto.defence.gov.au (8.14.1/8.14.1/Submit) id l9F73NOm008703; Mon, 15 Oct 2007 15:03:23 +0800 (WST) (envelope-from wilkinsa) Date: Mon, 15 Oct 2007 15:03:23 +0800 From: "Wilkinson, Alex" To: freebsd-arch@freebsd.org, arch@freebsd.org Message-ID: <20071015070322.GL4884@obelix.dsto.defence.gov.au> Mail-Followup-To: freebsd-arch@freebsd.org, arch@freebsd.org References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20071013004539.R1002@10.0.0.1> Organisation: Defence Science Technology Organisation User-Agent: Mutt/1.5.16 (2007-06-09) X-OriginalArrivalTime: 15 Oct 2007 07:03:23.0181 (UTC) FILETIME=[79B80DD0:01C80EF9] X-TM-AS-Product-Ver: SMEX-7.0.0.1526-5.0.1023-15480.001 X-TM-AS-Result: No--2.660700-0.000000-31 Content-Transfer-Encoding: 7bit Cc: Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2007 07:16:51 -0000 0n Sat, Oct 13, 2007 at 12:46:58AM -0700, Jeff Roberson wrote: > However, why not just fix Xen? > There may be a very good answer to this, I just don't know it. Well according to Mr libc aka Ulrich Drepper, Xen is a broken piece of crap [http://udrepper.livejournal.com/17577.html] -aW IMPORTANT: This email remains the property of the Australian Defence Organisation and is subject to the jurisdiction of section 70 of the CRIMES ACT 1914. If you have received this email in error, you are requested to contact the sender and delete the email. From owner-freebsd-arch@FreeBSD.ORG Mon Oct 15 20:13:14 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E97B16A418 for ; Mon, 15 Oct 2007 20:13:14 +0000 (UTC) (envelope-from ade@lovett.com) Received: from mail.lovett.com (foo.lovett.com [67.134.38.158]) by mx1.freebsd.org (Postfix) with ESMTP id 7721F13C459 for ; Mon, 15 Oct 2007 20:13:14 +0000 (UTC) (envelope-from ade@lovett.com) Received: from inferno.canal.lovett.com ([172.16.32.23]:58244) by mail.lovett.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.68 (FreeBSD)) (envelope-from ) id 1IhVhU-000Nhr-PM; Mon, 15 Oct 2007 12:34:04 -0700 In-Reply-To: <47107996.5090607@elischer.org> References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <2849CFD3-A747-4202-B2CB-759D3783C0B2@FreeBSD.org> Content-Transfer-Encoding: 7bit From: Ade Lovett Date: Mon, 15 Oct 2007 12:34:04 -0700 To: Julian Elischer X-Mailer: Apple Mail (2.752.3) Sender: ade@lovett.com Cc: arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2007 20:13:14 -0000 On Oct 13, 2007, at 00:53 , Julian Elischer wrote: > Generally, you can run several hundred (or more) virtual jail/ > vimage style machines. xen/vmware uses so much more resources that > you are usually limited to > so number like 20. it is possible in a virtual networking setup to > have a single process > spanning several virtual environments (for example one process with > a socket in each of the child universes). > It is a valid question, but there is I think a place for both types of > partitioning. I'll take anything that can *reliably* allow me to run RELENG_5,6,7 (and possibly HEAD) on the same (reasonably) beefy machine. Bonus points for being able to do /i386 on /amd64 (or vice versa). Right now, the only solution that comes even close to doing this is vmware on Windows XP. Bleh. -aDe From owner-freebsd-arch@FreeBSD.ORG Mon Oct 15 21:18:44 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E901316A469 for ; Mon, 15 Oct 2007 21:18:44 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 9532E13C459 for ; Mon, 15 Oct 2007 21:18:43 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1IhVTu-0007m0-8O for freebsd-arch@freebsd.org; Mon, 15 Oct 2007 19:20:02 +0000 Received: from 78-0-70-177.adsl.net.t-com.hr ([78.0.70.177]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 15 Oct 2007 19:20:02 +0000 Received: from ivoras by 78-0-70-177.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 15 Oct 2007 19:20:02 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Mon, 15 Oct 2007 18:40:57 +0200 Lines: 33 Message-ID: References: <470E5BFB.4050903@elischer.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig254B992121879367F176EC26" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 78-0-70-177.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) In-Reply-To: <470E5BFB.4050903@elischer.org> X-Enigmail-Version: 0.95.3 Sender: news Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Oct 2007 21:18:45 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig254B992121879367F176EC26 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Julian Elischer wrote: > What I'd like to see is a bit of a 'a-la-carte' virtualisation > ability. It's not much of an immediate architecture decision, but I'd like a unified administration interface to it all. Either one single command (e.g. "jail") for most operations (like "zfs", "geom" and "ifconfig" already do for their respective subsystems) or a set of orthogonal utilities with common syntax AND semantics. --------------enig254B992121879367F176EC26 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHE5gfldnAQVacBcgRAhmCAKC33SoIoLJ93+u+kaG7cyvyPnF5RwCg3Ewr HRVv6ZxFcXq/kZCm3NKrPF0= =hfTk -----END PGP SIGNATURE----- --------------enig254B992121879367F176EC26-- From owner-freebsd-arch@FreeBSD.ORG Tue Oct 16 00:09:29 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 197E116A46B for ; Tue, 16 Oct 2007 00:09:29 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outR.internet-mail-service.net (outR.internet-mail-service.net [216.240.47.241]) by mx1.freebsd.org (Postfix) with ESMTP id DB6C013C46E for ; Tue, 16 Oct 2007 00:09:28 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Mon, 15 Oct 2007 17:09:28 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id 9D1691266CE; Mon, 15 Oct 2007 17:09:27 -0700 (PDT) Message-ID: <47140146.2020806@elischer.org> Date: Mon, 15 Oct 2007 17:09:42 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Ade Lovett References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> <2849CFD3-A747-4202-B2CB-759D3783C0B2@FreeBSD.org> In-Reply-To: <2849CFD3-A747-4202-B2CB-759D3783C0B2@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Oct 2007 00:09:29 -0000 Ade Lovett wrote: > > On Oct 13, 2007, at 00:53 , Julian Elischer wrote: >> Generally, you can run several hundred (or more) virtual jail/vimage >> style machines. xen/vmware uses so much more resources that you are >> usually limited to >> so number like 20. it is possible in a virtual networking setup to >> have a single process >> spanning several virtual environments (for example one process with a >> socket in each of the child universes). >> It is a valid question, but there is I think a place for both types of >> partitioning. > I'll take anything that can *reliably* allow me to run RELENG_5,6,7 (and > possibly HEAD) on the same (reasonably) beefy machine. Bonus points for > being able to do /i386 on /amd64 (or vice versa). > > Right now, the only solution that comes even close to doing this is > vmware on Windows XP. Bleh. ok, but that 's full virtualisation. There are many levels of virtualisation. I'm cribbing this description from: http://lass.cs.umass.edu/~shenoy/courses/spring07/lectures/Lec05.pdf as it's as good a description as I've seen.. . Machine Emulation – VM emulates/simulates complete hardware – Unmodified guest OS for a different PC can be run + Bochs, VirtualPC for Mac, QEMU • Full/native Virtualization – VM simulates “enough” hardware to allow an unmodified guest OS to be run in isolation + Needs same hardware CPU + IBM VM family, VMWare Workstation, Parallels,… • Para-virtualization – VM does not simulate hardware – Use special API that a modified guest OS must use – Hypercalls trapped by the Hypervisor and serviced + Xen, VMWare ESX Server • OS-level virtualization – OS allows multiple secure virtual servers to be run – Guest OS is the same as the host OS, but appears isolated + apps see an isolated OS + Solaris Containers, BSD Jails, Linux Vserver • Application level virtualization – Application is gives its own copy of components that are not shared + (E.g., own registry files, global objects) - VE prevents conflicts + JVM Each of these has advantages and disadvantages. The one I'm particularly interested in is the kind of which "jails" is a member. It's the one that requires us to make architectural changes to our kernel the most, but it is also the one that can be the most efficient. It does NOT however allow running of different OS's (other than possibly allowing one to have a virtual system that runs a userland we can emulate, e.g. some Linux an dearlier versions of FreeBSD. From owner-freebsd-arch@FreeBSD.ORG Tue Oct 16 03:13:11 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAB6F16A468 for ; Tue, 16 Oct 2007 03:13:11 +0000 (UTC) (envelope-from ade@lovett.com) Received: from mail.lovett.com (foo.lovett.com [67.134.38.158]) by mx1.freebsd.org (Postfix) with ESMTP id CBF9213C448 for ; Tue, 16 Oct 2007 03:13:11 +0000 (UTC) (envelope-from ade@lovett.com) Received: from inferno.canal.lovett.com ([172.16.32.23]:60165) by mail.lovett.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.68 (FreeBSD)) (envelope-from ) id 1Ihcrn-0000Yc-4C; Mon, 15 Oct 2007 20:13:11 -0700 In-Reply-To: <47140146.2020806@elischer.org> References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> <2849CFD3-A747-4202-B2CB-759D3783C0B2@FreeBSD.org> <47140146.2020806@elischer.org> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <3BC22076-E26A-4FB3-96D3-232546FF7F64@FreeBSD.org> Content-Transfer-Encoding: 7bit From: Ade Lovett Date: Mon, 15 Oct 2007 20:13:09 -0700 To: Julian Elischer X-Mailer: Apple Mail (2.752.3) Sender: ade@lovett.com Cc: arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Oct 2007 03:13:12 -0000 On Oct 15, 2007, at 17:09 , Julian Elischer wrote: > There are many levels of virtualisation. Sure, we can take that one as read :) [.. snip good description..] > The one I'm particularly interested in is the kind of which "jails" > is a member. > It's the one that requires us to make architectural changes to our > kernel the most, > but it is also the one that can be the most efficient. It does NOT > however allow > running of different OS's (other than possibly allowing one to have > a virtual system > that runs a userland we can emulate, e.g. some Linux an dearlier > versions of FreeBSD. Definitely a noble goal, and one that I am willing to dedicate my meagre knowledge to. The reason I brought up the RELENG_ concept is one that would likely address a very real cost. The physical hardware is, these days, a very small percentage, of the TCO. To take an example, let's look at package building. Now, with a few tweaks and hacks, it's possible to take a CURRENT machine (well, RELENG_7 right now at least in my case), and with a further few prayers, make RELENG_5/6 chroot's to build packages in addition. The machine itself as more than enough horsepower to do this - and cuts dramatically the amount of power and space required to do the job. However, this is not necessarily an ideal solution, since there is the fairly major unknown of differences between a "true" RELENG_5 system, and one that is RELENG_5 userland + RELENG_7 kernel. As APIs and ABIs change, so these unknowns become more fragile. jails absolutely serve a purpose, something which I also make extensive use of, albeit in testbed environments, for solid segregation of application-level services whilst running on the same hardware, but it does really require that both the master and all jails are synced up as far as kernel + userland goes, for each jail. I have no magic bullet to solve this, since I don't believe there is one, but perhaps by widening the scope a little (but still focusing on what we currently know as jails) we can, possibly, get to full virtualisation. -aDe From owner-freebsd-arch@FreeBSD.ORG Tue Oct 16 08:34:12 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A11916A41B; Tue, 16 Oct 2007 08:34:12 +0000 (UTC) (envelope-from andrea@webcom.it) Received: from www.webcom.it (gen053.n002.c03.escapebox.net [213.73.82.53]) by mx1.freebsd.org (Postfix) with ESMTP id 3312613C4BA; Tue, 16 Oct 2007 08:34:12 +0000 (UTC) (envelope-from andrea@webcom.it) Received: from andrea by webcom.it with local (Exim 3.36 #1) id 1IhhEV-0007S5-00; Tue, 16 Oct 2007 07:52:55 +0000 Date: Tue, 16 Oct 2007 07:52:55 +0000 From: Andrea Campi To: Julian Elischer Message-ID: <20071016075255.GG61822@webcom.it> References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> <2849CFD3-A747-4202-B2CB-759D3783C0B2@FreeBSD.org> <47140146.2020806@elischer.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <47140146.2020806@elischer.org> User-Agent: Mutt/1.5.11 Sender: Andrea Campi Cc: arch@freebsd.org, Ade Lovett Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Oct 2007 08:34:12 -0000 On Mon, Oct 15, 2007 at 05:09:42PM -0700, Julian Elischer wrote: > ? Para-virtualization > ? VM does not simulate hardware > ? Use special API that a modified guest OS must use > ? Hypercalls trapped by the Hypervisor and serviced > + Xen, VMWare ESX Server > > ? OS-level virtualization > ? OS allows multiple secure virtual servers to be run > ? Guest OS is the same as the host OS, but appears isolated > + apps see an isolated OS > + Solaris Containers, BSD Jails, Linux Vserver To me, jails have always been a key distinguishing factor to FreeBSD, so efforts to further enhance them are very welcome. However, I've always wondering whether it was possible to also do some kind of para-virtualization, maybe with limited scope. In para-virtualization you modify the kernel source in such a way that accesses to the hardware are instead translated into calls to the hypervisor. This means you could simply write device drivers for a "virtual network adapter", "virtual disk" etc. What this buys you is that you can have a full kernel (say 6.x) running as a hypervisor, and trimmed down kernels (say 7.x and several 6.x versions), compiled with only the virtual device drivers, running as additional VMs. This gives you a more complete emulation than what we can do with jails (gives you the kernel in addition to the libc), while at the same time keeping the memory cost low. The virtualization features in recent CPUs keep the cost low, so this setup would be very useful for instance in a lot of development tasks where you want to compare behavior of the kernel over several branches. By limiting the scope to only other FreeBSD versions you can build a limited number of virtual "calls", keeping the complexity low. You would get a framework that would be part of the base system, instead of something biggish as Xen. It doesn't feel like a huge amount of work either. WDYT? Bye, Andrea -- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it. From owner-freebsd-arch@FreeBSD.ORG Tue Oct 16 22:54:49 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CCB0516A417 for ; Tue, 16 Oct 2007 22:54:49 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5DB4513C45D for ; Tue, 16 Oct 2007 22:54:49 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1IhvJ9-0000TV-4T for freebsd-arch@freebsd.org; Tue, 16 Oct 2007 22:54:39 +0000 Received: from 89-172-55-115.adsl.net.t-com.hr ([89.172.55.115]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 16 Oct 2007 22:54:39 +0000 Received: from ivoras by 89-172-55-115.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 16 Oct 2007 22:54:39 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Wed, 17 Oct 2007 00:54:21 +0200 Lines: 42 Message-ID: References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> <2849CFD3-A747-4202-B2CB-759D3783C0B2@FreeBSD.org> <47140146.2020806@elischer.org> <20071016075255.GG61822@webcom.it> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF318002D272DEE2156BA38AF" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-55-115.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) In-Reply-To: <20071016075255.GG61822@webcom.it> X-Enigmail-Version: 0.95.3 Sender: news Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Oct 2007 22:54:49 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF318002D272DEE2156BA38AF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Andrea Campi wrote: > In para-virtualization you modify the kernel source in such a way that > accesses to the hardware are instead translated into calls to the > hypervisor. This means you could simply write device drivers for a > "virtual network adapter", "virtual disk" etc. What this buys you is > that you can have a full kernel (say 6.x) running as a hypervisor, and > trimmed down kernels (say 7.x and several 6.x versions), compiled with > only the virtual device drivers, running as additional VMs. > WDYT? Well Xen does paravirtulization like you described (and I agree something like that is more flexible then jails, if supported by other operating systems). DragonflyBSD has its own flavor of virtualization similar to user mode Linux, but it has greatly diverged from FreeBSD so it't probably not trivially portable. Or do you mean something like this: http://feanor.sssup.it/~fabio/freebsd/lkvm/ ? --------------enigF318002D272DEE2156BA38AF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHFUEdldnAQVacBcgRAhTpAKCVVssZUyZ8rFDgpR02ZPn/TCukIgCgtkTi gM1q6WqrC+4jR2Z1Q5DOXlw= =cYVC -----END PGP SIGNATURE----- --------------enigF318002D272DEE2156BA38AF-- From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 10:56:59 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93D1316A417; Wed, 17 Oct 2007 10:56:59 +0000 (UTC) (envelope-from zec@icir.org) Received: from xaqua.tel.fer.hr (xaqua.tel.fer.hr [161.53.19.25]) by mx1.freebsd.org (Postfix) with ESMTP id 20DDD13C442; Wed, 17 Oct 2007 10:56:59 +0000 (UTC) (envelope-from zec@icir.org) Received: by xaqua.tel.fer.hr (Postfix, from userid 20006) id 757EE9B655; Wed, 17 Oct 2007 12:37:10 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on xaqua.tel.fer.hr X-Spam-Level: X-Spam-Status: No, score=-4.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.7 Received: from [192.168.200.100] (zec2.tel.fer.hr [161.53.19.79]) by xaqua.tel.fer.hr (Postfix) with ESMTP id D079E9B64D; Wed, 17 Oct 2007 12:37:08 +0200 (CEST) From: Marko Zec To: freebsd-arch@freebsd.org Date: Wed, 17 Oct 2007 12:37:06 +0200 User-Agent: KMail/1.9.7 References: <470E5BFB.4050903@elischer.org> <20071016075255.GG61822@webcom.it> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200710171237.07583.zec@icir.org> Cc: Ivan Voras Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 10:56:59 -0000 On Wednesday 17 October 2007 00:54:21 you wrote: > Andrea Campi wrote: > > In para-virtualization you modify the kernel source in such a way > > that accesses to the hardware are instead translated into calls to > > the hypervisor. This means you could simply write device drivers > > for a "virtual network adapter", "virtual disk" etc. What this buys > > you is that you can have a full kernel (say 6.x) running as a > > hypervisor, and trimmed down kernels (say 7.x and several 6.x > > versions), compiled with only the virtual device drivers, running > > as additional VMs. > > > > WDYT? > > Well Xen does paravirtulization like you described (and I agree > something like that is more flexible then jails, if supported by > other operating systems). Actually, resource virtualization done at kernel level could offer great degree of flexiblity. Ideally, a modular virtualization framework would allow one to virtualize only the resources one needs, for example having a single process talking to several isolated networking domains, or having several processes bound to the same slot in a proportional share CPU scheduler, sharing or not sharing the same filesystem hierarchy etc. I think the thrust of this thread was in tackling people's imagination on how such a modular virtualization framework should look like, and which capabilities it should offer and which not. I.e. not get carried away in comparing kernel-level virtualization in general against Xen and alike, which are undoubtably very useful tools which have secured their place under the sun... Cheers, Marko > DragonflyBSD has its own flavor of > virtualization similar to user mode Linux, but it has greatly > diverged from FreeBSD so it't probably not trivially portable. > > Or do you mean something like this: > http://feanor.sssup.it/~fabio/freebsd/lkvm/ ? From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 11:04:14 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43D1416A419 for ; Wed, 17 Oct 2007 11:04:14 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id F248313C448 for ; Wed, 17 Oct 2007 11:04:13 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1Ii6h9-0005t9-Ni for freebsd-arch@freebsd.org; Wed, 17 Oct 2007 11:04:11 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Oct 2007 11:04:11 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Oct 2007 11:04:11 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Wed, 17 Oct 2007 13:04:57 +0200 Lines: 19 Message-ID: References: <470E5BFB.4050903@elischer.org> <20071016075255.GG61822@webcom.it> <200710171237.07583.zec@icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.6 (X11/20070801) In-Reply-To: <200710171237.07583.zec@icir.org> Sender: news Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 11:04:14 -0000 Marko Zec wrote: > Actually, resource virtualization done at kernel level could offer great > degree of flexiblity. Ideally, a modular virtualization framework > would allow one to virtualize only the resources one needs, for example > having a single process talking to several isolated networking domains, > or having several processes bound to the same slot in a proportional > share CPU scheduler, sharing or not sharing the same filesystem > hierarchy etc. I think the thrust of this thread was in tackling > people's imagination on how such a modular virtualization framework > should look like, and which capabilities it should offer and which not. > I.e. not get carried away in comparing kernel-level virtualization in > general against Xen and alike, which are undoubtably very useful tools > which have secured their place under the sun... Of course, we speak about different concepts of "flexibility" - in case one wants to run FreeBSD and only FreeBSD then jail-like systems (kernel-level virtualization) are better for almost all circumstances then a heavy-weight kernel-on-top-of-a-kernel approach. From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 11:31:33 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9B9516A41B; Wed, 17 Oct 2007 11:31:33 +0000 (UTC) (envelope-from fabio@freebsd.org) Received: from sssup.it (ms01.sssup.it [193.205.80.99]) by mx1.freebsd.org (Postfix) with ESMTP id C3DDA13C457; Wed, 17 Oct 2007 11:31:32 +0000 (UTC) (envelope-from fabio@freebsd.org) Received: from [10.30.3.4] (HELO granpasso.retis) by sssup.it (CommuniGate Pro SMTP 4.1.8) with ESMTP-TLS id 35078445; Wed, 17 Oct 2007 12:19:53 +0200 Received: from granpasso.retis (localhost.retis [127.0.0.1]) by granpasso.retis (8.14.1/8.14.1) with ESMTP id l9HA92qW098989; Wed, 17 Oct 2007 12:09:07 +0200 (CEST) (envelope-from fabio@freebsd.org) Received: (from fabio@localhost) by granpasso.retis (8.14.1/8.14.1/Submit) id l9GNnQ60075223; Wed, 17 Oct 2007 01:49:26 +0200 (CEST) (envelope-from fabio@freebsd.org) X-Authentication-Warning: granpasso.retis: fabio set sender to fabio@freebsd.org using -f Date: Wed, 17 Oct 2007 01:49:26 +0200 From: Fabio Checconi To: Ivan Voras Message-ID: <20071016234926.GF1243@gandalf.sssup.it> Mail-Followup-To: Ivan Voras , freebsd-arch@freebsd.org References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> <2849CFD3-A747-4202-B2CB-759D3783C0B2@FreeBSD.org> <47140146.2020806@elischer.org> <20071016075255.GG61822@webcom.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 11:31:33 -0000 > From: Ivan Voras > Date: Wed, Oct 17, 2007 12:54:21AM +0200 > > Andrea Campi wrote: > > > In para-virtualization you modify the kernel source in such a way that [...] > > Well Xen does paravirtulization like you described (and I agree > something like that is more flexible then jails, if supported by other > operating systems). DragonflyBSD has its own flavor of virtualization > similar to user mode Linux, but it has greatly diverged from FreeBSD so > it't probably not trivially portable. > > Or do you mean something like this: > http://feanor.sssup.it/~fabio/freebsd/lkvm/ ? > The version of kvm ported to FreeBSD has no paravirtualization support. Paravirtualization is, as far as I know, still an experimental feature on Linux, not present in the mainline tree. I am not aware of FreeBSD support for the kvm hypercalls that are being introduced in the kvm experimental trees. By now kvm is really just a full virtualization solution. From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 17:10:46 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F1EA16A46B for ; Wed, 17 Oct 2007 17:10:46 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from speedfactory.net (mail6.speedfactory.net [66.23.216.219]) by mx1.freebsd.org (Postfix) with ESMTP id B737813C48A for ; Wed, 17 Oct 2007 17:10:45 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (unverified [66.23.211.162]) by speedfactory.net (SurgeMail 3.8p) with ESMTP id 214852681-1834499 for multiple; Wed, 17 Oct 2007 12:48:04 -0400 Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.8/8.13.8) with ESMTP id l9HGjrGU006678; Wed, 17 Oct 2007 12:45:53 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: "Constantine A. Murenin" Date: Wed, 17 Oct 2007 12:45:36 -0400 User-Agent: KMail/1.9.6 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200710171245.36949.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Wed, 17 Oct 2007 12:45:53 -0400 (EDT) X-Virus-Scanned: ClamAV 0.88.3/4543/Wed Oct 17 10:16:16 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: arch@freebsd.org Subject: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 17:10:46 -0000 [Trying to redirect this off cvs-all & friends.. ] So as I said previously, I thought about this some more offline last night / this morning and looked at the code some and here are my thoughts: Things I like about the current sensors code: - I like the actual sensor object used to represent a sensor. It has a few basic things like a string for a name, a type (I would have just done a "units" for the value, but the type is basically that), and a basic alarm state. (I might have done 4 states, think green, yellow, orange, red mapped to good, warning, critical, bad. However, the 3 states in the current code is fine. 4 states might be overkill.) - I like having the lm(4), etc. sensors report status via an object like this instead of just getting some string out of a tool like 'mbmon'. A framework should provide the data so that multiple utilities can use it. - I'm not entirely opposed to having kernel drivers for various known sensor providers like lm(4), etc. OTOH, I could see a driver just providing ioctls to query the list of sensors and a sensor's value similar to using requests to the IPMI BMC to request SDR data via ioctls to /dev/ipmi0 as the kernel -> userland interface. Things I'm not a big fan of: - Forcing all sensors to be in the kernel. A general rule in most OSes is to minimize the amount of stuff in the kernel. The kernel is easily the most, erm, "sensitive" process in the system. A segfault has much more serious consequences in the kernel (panic) than in userland (SIGSEGV). It also makes it more complicated to add "psuedo" sensors. For example, if I wanted a sensor for CPU usage based on the kern.cp_time sysctl, I could easily do that in userland by quering the sysctl periodically, computing the relative % busy and set a status based on a set of trigger points. Similarly, you could conceive having virtual/pseudo/whatever sensors for disk space, etc. At some point you do risk duplicating SNMP traps with sensord I suppose. I also genuinely think it is better to keep lots of state around in userland rather than in the kernel. That is, I think the kernel should provide a way to query a sensor (RAID drivers provide ioctls to communicate with the firmware, IPMI has ioctls to allow userland to communicate with the BMC, some drivers may provide ioctl/sysctl/whatever to read sensor values directly), but I don't think we should try to store history (see the cp_time example above) or extended state (keeping track of which drives exist so you can detect a drive that goes away) in the kernel. That belongs in userland. I think it can be ok for some sensors to be completely in the kernel and just get queried directly from userland, but I don't think that is a valid design constraint to enforce on all sensors. Things I think are dubious at best: - That it is more secure to put code in the kernel than as root in userland. It seems odd to even have to mention this, but it should be painfully obvious that a bug in a driver has much worse consequences than a bug in a user app running as root (or even better, some dedicated non-privileged account in a group that can send ioctls to /dev/ipmi0 or other monitored devices). I guess maybe I can see a viewpoint where you hope that a driver bug always panics and doesn't just corrupt data and so it's more likely to get a SA's attention and be less exploitable maybe? I can understand that 'mbmon' is untrusted code, but it seems to me that there are some less drastic measures than rewriting it all as kernel code, namely 1) audit the existing code, or 2) rewrite it all as userland code, ideally less privileged. Other things that might be nice: - IWBN to have a userland interface to sensors. For example, if nothing else a sensor enumerator rather than duplicating the sysctl loop as the current code does. This would make it easier to at least adjust the current artificial limit on the number of sensors since only one place in userland would have to change. (BTW, having an artificial limit on the number of sensors is lame. This is an example where using the normal way of walking a sysctl tree is superior. You can lose the entire limit.) Having a userland interface also makes it easier to have backends that are entirely in userland. - An snmp module that uses the above userland interface to export sensors via snmp. bsnmp already has a way to load modules at runtime (at least startup time) to add new MIBS. This would allow remote monitoring of various sensors if people prefer that to having a daemon on the box post alerts. If nothing else, it lets you add mrtg or rrdtool type graphs of the history of sensors if desired. Basically, I think there should be a "real" abstracted interface in userland that can use various backends. One backend could be to query sensors from drivers that provide them directly (lm(4), etc.). Another backend could use the existing IPMI interface to query SDR sensors via IPMI commands to the BMC. Different RAID controllers could provide backends that communicate with the firmware to maintain whatever state is needed, etc. but w/o doing all that in the device driver. People could write their own custom sensors w/o having to write a kernel module. Maybe that's a bigger vision than you were shooting for. I'm not sure phk@ will agree with this one either fwiw. :) -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 20:07:51 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CBD116A421 for ; Wed, 17 Oct 2007 20:07:51 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outT.internet-mail-service.net (outT.internet-mail-service.net [216.240.47.243]) by mx1.freebsd.org (Postfix) with ESMTP id 57C6C13C46B for ; Wed, 17 Oct 2007 20:07:51 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Wed, 17 Oct 2007 13:07:50 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id 220C6126732; Wed, 17 Oct 2007 13:07:50 -0700 (PDT) Message-ID: <47166BA5.1000100@elischer.org> Date: Wed, 17 Oct 2007 13:08:05 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: John Baldwin References: <200710171245.36949.jhb@freebsd.org> In-Reply-To: <200710171245.36949.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 20:07:51 -0000 John Baldwin wrote: > > Other things that might be nice: > > - IWBN to have a userland interface to sensors. For example, if nothing else > a sensor enumerator rather than duplicating the sysctl loop as the current > code does. This would make it easier to at least adjust the current > artificial limit on the number of sensors since only one place in userland > would have to change. (BTW, having an artificial limit on the number of > sensors is lame. This is an example where using the normal way of walking a > sysctl tree is superior. You can lose the entire limit.) Having a userland > interface also makes it easier to have backends that are entirely in > userland. > maybe a loopback filesystem /var/run/sensors could be a loopback filesystem that exports a view of the sensors available. The process doing the work would be a userland process but it would be providing the information in the form of a filesystem so other processes could access it easily. > From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 20:20:47 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 976FC16A46C; Wed, 17 Oct 2007 20:20:47 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 5381F13C457; Wed, 17 Oct 2007 20:20:47 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 41F8317107; Wed, 17 Oct 2007 20:20:46 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id l9HKKjTu052117; Wed, 17 Oct 2007 20:20:46 GMT (envelope-from phk@critter.freebsd.dk) To: Julian Elischer From: "Poul-Henning Kamp" In-Reply-To: Your message of "Wed, 17 Oct 2007 13:08:05 MST." <47166BA5.1000100@elischer.org> Date: Wed, 17 Oct 2007 20:20:45 +0000 Message-ID: <52116.1192652445@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 20:20:47 -0000 In message <47166BA5.1000100@elischer.org>, Julian Elischer writes: >> Having a userland >> interface also makes it easier to have backends that are entirely in >> userland. >> >maybe a loopback filesystem Just what is it that is so enticing about the kernel ? Why not simply pass it to a daemon ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 20:31:49 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9767916A418 for ; Wed, 17 Oct 2007 20:31:49 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outC.internet-mail-service.net (outC.internet-mail-service.net [216.240.47.226]) by mx1.freebsd.org (Postfix) with ESMTP id 82DE613C455 for ; Wed, 17 Oct 2007 20:31:49 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Wed, 17 Oct 2007 13:31:47 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id E9ED1126732; Wed, 17 Oct 2007 13:31:46 -0700 (PDT) Message-ID: <47167145.70705@elischer.org> Date: Wed, 17 Oct 2007 13:32:05 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Poul-Henning Kamp References: <52116.1192652445@critter.freebsd.dk> In-Reply-To: <52116.1192652445@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 20:31:49 -0000 Poul-Henning Kamp wrote: > In message <47166BA5.1000100@elischer.org>, Julian Elischer writes: > >>> Having a userland >>> interface also makes it easier to have backends that are entirely in >>> userland. >>> >> maybe a loopback filesystem > > Just what is it that is so enticing about the kernel ? Why not simply > pass it to a daemon ? Becuause you can not open and do a directory listing on a process but you can on something that is presented as a file system. The advantage is that all the heavy lifting is done in user space and the kernel only supplies a convenient delivery mechanism. Pipes are also a kernel supplied delivery mechanism but they are not random access, where a file system is. Otherwise they are similar. > From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 20:39:14 2007 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 696F316A419; Wed, 17 Oct 2007 20:39:14 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (nagual.pp.ru [194.87.13.69]) by mx1.freebsd.org (Postfix) with ESMTP id DB5D913C48A; Wed, 17 Oct 2007 20:39:13 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.14.1/8.14.1) with ESMTP id l9HKPVbS078457; Thu, 18 Oct 2007 00:25:31 +0400 (MSD) (envelope-from ache@nagual.pp.ru) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nagual.pp.ru; s=default; t=1192652732; bh=eukmhazLG8tkdnMZCyTtpfB0/3O7sKp8jDvURWL KhhE=; l=512; h=Date:From:To:Cc:Subject:Message-ID:References: MIME-Version:Content-Type:Content-Disposition:In-Reply-To: User-Agent; b=UqcZbYi4oKXydmGHKZGVsjx1yk3n7jsOiUQFAfnrPSgQ/GolnI7K 71QBBWpZNH+Ia1TQYhEJFFlTI4ZXf7IJTtBOjPClQMA43dt8FqOgWjtQsJ5C0MiFk1t F25MkrSYtJLkfyjNDDzzYeY1/55AHdkOgZv7FR0jSVGlqeHdjRnE= Received: (from ache@localhost) by nagual.pp.ru (8.14.1/8.14.1/Submit) id l9HKPVG6078456; Thu, 18 Oct 2007 00:25:31 +0400 (MSD) (envelope-from ache) Date: Thu, 18 Oct 2007 00:25:31 +0400 From: Andrey Chernov To: Poul-Henning Kamp Message-ID: <20071017202531.GB78324@nagual.pp.ru> References: <47166BA5.1000100@elischer.org> <52116.1192652445@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52116.1192652445@critter.freebsd.dk> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: arch@FreeBSD.ORG, Julian Elischer , "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 20:39:14 -0000 On Wed, Oct 17, 2007 at 08:20:45PM +0000, Poul-Henning Kamp wrote: > In message <47166BA5.1000100@elischer.org>, Julian Elischer writes: > > >> Having a userland > >> interface also makes it easier to have backends that are entirely in > >> userland. > >> > >maybe a loopback filesystem > > Just what is it that is so enticing about the kernel ? Why not simply > pass it to a daemon ? What about DEVFS idea like /dev/sensors/* ? Sensors _are_ devices, not sysctls. -- http://ache.pp.ru/ From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 20:56:41 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F082416A46E; Wed, 17 Oct 2007 20:56:41 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.177]) by mx1.freebsd.org (Postfix) with ESMTP id 8599513C4B3; Wed, 17 Oct 2007 20:56:41 +0000 (UTC) (envelope-from max@love2party.net) Received: from amd64.laiers.local (dslb-088-066-055-120.pools.arcor-ip.net [88.66.55.120]) by mrelayeu.kundenserver.de (node=mrelayeu7) with ESMTP (Nemesis) id 0ML2xA-1IiFk90Im8-0001Zm; Wed, 17 Oct 2007 22:43:54 +0200 From: Max Laier Organization: FreeBSD To: freebsd-arch@freebsd.org Date: Wed, 17 Oct 2007 22:43:45 +0200 User-Agent: KMail/1.9.7 References: <52116.1192652445@critter.freebsd.dk> In-Reply-To: <52116.1192652445@critter.freebsd.dk> X-Face: ,,8R(x[kmU]tKN@>gtH1yQE4aslGdu+2]; R]*pL,U>^H?)gW@49@wdJ`H<=?utf-8?q?=25=7D*=5FBD=0A=09U=5For=3D=5CmOZf764=26nYj=3DJYbR1PW0ud?=>|!~,,CPC.1-D$FG@0h3#'5"k{V]a~.<=?utf-8?q?mZ=7D44=23Se=7Em=0A=09Fe=7E=5C=5DX5B=5D=5Fxj?=(ykz9QKMw_l0C2AQ]}Ym8)fU MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1356896.QVs3KXlYjZ"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200710172243.51958.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1/uJ5UDxkoiF+0ah6OO8zdHIsNd/Lskx2ejgMf W8yO02w9DtAlLuuuSZ6OxSBpy3bvOSIoNgzVvObYyiqh7HK54q X8SV0tuzRFQV7mF3ekpHNo8bYexivlaidaFrsvdagE= Cc: Poul-Henning Kamp , Julian Elischer , "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 20:56:42 -0000 --nextPart1356896.QVs3KXlYjZ Content-Type: text/plain; charset="iso-8859-6" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Wednesday 17 October 2007, Poul-Henning Kamp wrote: > In message <47166BA5.1000100@elischer.org>, Julian Elischer writes: > >> Having a userland > >> interface also makes it easier to have backends that are entirely in > >> userland. > > > >maybe a loopback filesystem > > Just what is it that is so enticing about the kernel ? Why not simply > pass it to a daemon ? What I like about the OpenBSD framework is, that you can get the sensor=20 data with basic tools. What's wrong with "everything is a file"? With a=20 file system you don't have to jump through any hoops to provide=20 concurrent access to more than one reader. You could easily create=20 symlinks to map sensors to location. You have means to restrict access=20 to certain sensors. etc. ... I'm not sure that you can write the one daemon that suites all needs, but=20 if you provide all the sensors in a central place accessible through=20 basic tools it's easy to write a shell script that does exactly what you=20 need. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart1356896.QVs3KXlYjZ Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHFnQHXyyEoT62BG0RAv95AJ9/fOumHwX315YdyO7YGIP47jsykgCfRt4p 8vAP+9ePG9A3AvXH5IXwa3U= =pFOY -----END PGP SIGNATURE----- --nextPart1356896.QVs3KXlYjZ-- From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 20:59:32 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10BA816A41A; Wed, 17 Oct 2007 20:59:32 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 8908E13C461; Wed, 17 Oct 2007 20:59:31 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id D37DC17105; Wed, 17 Oct 2007 20:59:29 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id l9HKxTwg052435; Wed, 17 Oct 2007 20:59:29 GMT (envelope-from phk@critter.freebsd.dk) To: Julian Elischer From: "Poul-Henning Kamp" In-Reply-To: Your message of "Wed, 17 Oct 2007 13:32:05 MST." <47167145.70705@elischer.org> Date: Wed, 17 Oct 2007 20:59:29 +0000 Message-ID: <52434.1192654769@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 20:59:32 -0000 In message <47167145.70705@elischer.org>, Julian Elischer writes: >Poul-Henning Kamp wrote: >> In message <47166BA5.1000100@elischer.org>, Julian Elischer writes: >> >>>> Having a userland >>>> interface also makes it easier to have backends that are entirely in >>>> userland. >>>> >>> maybe a loopback filesystem >> >> Just what is it that is so enticing about the kernel ? Why not simply >> pass it to a daemon ? > >Becuause you can not open and do a directory listing on a process but you can >on something that is presented as a file system. But in userland you can do things so much more comprehensively with so much more accesible tools. We don't browse webpages by saying cat < /net/tcp/http/www.cnn.com And we shouldn't implement our sensor/alert system that way either. I know this, because I have spent the last couple of years essentially implementing access to all sorts of "sensors" and "actuators". The end goal is to make all sorts of stuff, from oddball navigation transmitters with RS-232 ports over dry relay contacts to natural gas furnaces manageable with a web-browser and SNMP. I'm not advocating that my "measured" (http://phk.freebsd.dk/phkrel) be imported into FreeBSD, and I don't see it as a competitor to the sensors stuff from OpenBSD. But I will show a bit about it anyway, so that you can see you why I _know_ that whatever we do decide to do, it must live in userland: >> critter phk> telnet fs 2040 Trying 192.168.60.20... Connected to fs.freebsd.dk. Escape character is '^]'. >> GET /master HTTP/1.1 HTTP/1.1 200 OK Content-Type: text/html; charset=iso-8859-1 Cache-control: no-cache; no-store; Pragma: no-cache >> PATTERN 11.2.1-2 >> EVENTS E 011.002.001 P NEW ain E 011.002.001 P LABEL V12 E 011.002.001 P FORMAT %.3f E 011.002.001 P UNIT Volt E 011.002.001 P LOW 11.4 E 011.002.001 P SAG 11.6 E 011.002.001 P TIMEOUT 16 E 011.002.002 P NEW ain E 011.002.002 P LABEL A12 E 011.002.002 P FORMAT %.3f E 011.002.002 P UNIT Ampere E 011.002.002 P RAISE 3.8 E 011.002.002 P HIGH 4 E 011.002.002 P TIMEOUT 16 E 011.002.001 P VALUE 13.501 1192653549.789175094 E 011.002.001 P LIMIT normal 0 0 E 011.002.002 P VALUE 2.893 1192653548.568108390 E 011.002.002 P LIMIT normal 0 0 E 011.002.002 P VALUE 2.891 1192653549.790052019 E 011.002.001 P VALUE 13.485 1192653551.011067241 E 011.002.002 P VALUE 2.899 1192653551.011921611 E 011.002.001 P VALUE 13.468 1192653552.231962722 E 011.002.002 P VALUE 2.903 1192653552.232819425 E 011.002.001 P VALUE 13.507 1192653553.454879387 E 011.002.002 P VALUE 2.876 1192653553.455751091 E 011.002.001 P VALUE 13.478 1192653554.676769905 E 011.002.002 P VALUE 2.902 1192653554.677682053 ... As you can guess from the first couple of lines, measured will also answer as a HTTP server, so you can just point your browser at it, and see what all the sensors record or manipulate actuators. The protocol you see above, is trivally implemented in any language. Creating a graphical "control panel" for stuff in for instance Tk is trival. I also have an SNMP agent that talks this protocol. But the _really_ important thing about measured, is that integrating new "stuff" is a small matter of programming. A 3-axis magnetometer with a RS232 interface: 121 lines of code. Soekris NET4801 enviromental sensors: 195 lines. A VOR transmitter with a particular nasty RS232 protocol: 542 lines. TCP protocol to a radiolink alert proxy: 557 lines. And sensors can be implemented in separate processes and even on other machines. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 21:39:20 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8352C16A418 for ; Wed, 17 Oct 2007 21:39:20 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 1A63D13C468 for ; Wed, 17 Oct 2007 21:39:19 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1IiGbX-0006S4-Tw for freebsd-arch@freebsd.org; Wed, 17 Oct 2007 21:39:03 +0000 Received: from r5j156.net.upc.cz ([86.49.9.156]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Oct 2007 21:39:03 +0000 Received: from gamato by r5j156.net.upc.cz with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 17 Oct 2007 21:39:03 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: martinko Date: Wed, 17 Oct 2007 23:38:56 +0200 Lines: 41 Message-ID: References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: r5j156.net.upc.cz User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.6) Gecko/20070821 SeaMonkey/1.1.4 In-Reply-To: <47107996.5090607@elischer.org> Sender: news Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 21:39:20 -0000 Julian Elischer wrote: > Jeff Roberson wrote: >> On Fri, 12 Oct 2007, James Gritton wrote: >> >>> Julian Elischer wrote: >>> >>>> What I'd like to see is a bit of a 'a-la-carte' virtualisation >>>> ability. >>> ... >>>> My question to you, the reader, is: >>>> what aspects of virtualisation (the appearance of multiple instances >>>> of some resource) would you like to see in the system? >>> >>> Of course everything jail has now, and all the network bits that >>> vimage offers. >>> >>> CPU scheduling, in particular schedule the CPU first by jail, and then >>> by processes within jail. >> >> So the question I have is; why do all of these things instead of >> vmware/xen/other full virtualization? We can implement these >> technologies. Specifically, I could do the CPU scheduling. However, >> why not just fix Xen? There may be a very good answer to this, I just >> don't know it. > > Generally, you can run several hundred (or more) virtual jail/vimage > style machines. xen/vmware uses so much more resources that you are > usually limited to > so number like 20. it is possible in a virtual networking setup to have > a single process > spanning several virtual environments (for example one process with a > socket in each of the child universes). > It is a valid question, but there is I think a place for both types of > partitioning. > On the other hand fixing Xen or implementing something like LKVM seems to be much easier than changing all those subsystems people would like to resource partition. Well, it's just my guess. Martin From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 22:26:56 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E41D16A419 for ; Wed, 17 Oct 2007 22:26:56 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id EC41713C4A5 for ; Wed, 17 Oct 2007 22:26:55 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.1/8.13.7) with ESMTP id l9HMGhZH067252; Wed, 17 Oct 2007 15:16:43 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.1/8.13.4/Submit) id l9HMGhbd067251; Wed, 17 Oct 2007 15:16:43 -0700 (PDT) Date: Wed, 17 Oct 2007 15:16:43 -0700 (PDT) From: Matthew Dillon Message-Id: <200710172216.l9HMGhbd067251@apollo.backplane.com> To: martinko References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> Cc: freebsd-arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 22:26:56 -0000 I think the reason you see such a huge degree of virtualized and para-virtualized environments these days is simply because service providers can no longer provide all the web services individual sites need as a single common set of managed applications. All they can do, pretty much, is allow their users to install and run the application set they need to run for their site. In fact, there are several layers of middle-management web site providers available today who simply lease virtual machines and build a custom environment for their clients. Virtualization neatly solves what would otherwise be a nightmare for providers. Despite the loss of performance you get in a virtualized environment, it is far, far easier to manage memory and I/O resources in a virtualized environment then it is in a jail, and far far easier to migrate such environments between physical machines without having to worry about introducing incompatibilities or having to force your customers to upgrade and deal with the inevitable snafus that crop up when that happens. You don't even have to worry about upgrades to physical hardware... the virtual environment doesn't have to change by a single byte. I remember upgrading machines at BEST Internet. It was a nightmare to deal with all the customer snafus that occured every time we upgraded something. I wish we had virtualization back then. Security is also a whole lot easier to manage with a virtualized environment. You basically just run the virtual kernel (strangely enough, in a jail usually), and give the user root inside it. The user can install whatever he wants... whatever services, applications, etc. And if the user crashes a virtual kernel or creates a security issue, just that one user is effected, not the other 19 on the physical box. Dealing with break-ins which effect dozens, hundreds, or thousands of users is a nightmare for any provider. Dealing with a break-in which effect one user and is basically the user's own fault is not. Global resources are a lot easier to manage in a virtualized environment. For example, it is impossible for a user running in a virtual kernel to use any more real system memory then was assigned that virtual kernel, and all I/O resources basically devolve down into just one or two points inside the virtual kernel (network I/O and 'raw disk' I/O), which are similarly ultra easy to control. If that user blows up his own system, or causes it to start page-thrashing, it is his problem to solve and will not effect the other 19 users on the box. Thrash a machine in a jailed environment and the whole machine goes to pot. 'limit' and other resource controls just don't work well in a jailed environment if the intent is to avoid thrashing other environments running on that same machine. What a jailed environment offers is performance. There is no question about that. If I were designing a turnkey server with a very specific application set I would probably use jails. But if I wanted to hand an actual login to a user in an environment where the user needs to run many potential services or backend run-times, I would choose a virtual kernel hands-down above a jail solution. Jails are useful for many things, but most of those are not things which require scaling. Hardware is cheap, manpower is not. Being able to avoid hiring a single employee is worth 20 machines at least, and hardware costs scale far better over time then employee costs. That is what is driving the market for virtualization. Performance doesn't enter into the equation most of the time. For anyone doing a serious deployment this is a huge deal. A jailed environment just doesn't offer the same level of control or management, no matter how much you try to separate resources out inside the kernel. Similarly, storage is almost free these days. Storage is not an issue, so the replication of environments and application installs is not an issue. Not any more. The only real limitation for a virtualized environment is physical memory, and that has already been taken care of with the money you save by not having to hire that extra employee. -Matt From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 22:56:58 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A11B16A419; Wed, 17 Oct 2007 22:56:58 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.freebsd.org (Postfix) with ESMTP id 4988413C47E; Wed, 17 Oct 2007 22:56:58 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (6gimc1y5qthuphck@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id l9HMK5cH027576; Wed, 17 Oct 2007 15:20:05 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id l9HMK5HH027575; Wed, 17 Oct 2007 15:20:05 -0700 (PDT) (envelope-from jmg) Date: Wed, 17 Oct 2007 15:20:04 -0700 From: John-Mark Gurney To: John Baldwin Message-ID: <20071017222004.GT39759@funkthat.com> Mail-Followup-To: John Baldwin , "Constantine A. Murenin" , arch@freebsd.org References: <200710171245.36949.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200710171245.36949.jhb@freebsd.org> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (hydrogen.funkthat.com [127.0.0.1]); Wed, 17 Oct 2007 15:20:05 -0700 (PDT) Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 22:56:58 -0000 John Baldwin wrote this message on Wed, Oct 17, 2007 at 12:45 -0400: > Basically, I think there should be a "real" abstracted interface in userland > that can use various backends. One backend could be to query sensors from > drivers that provide them directly (lm(4), etc.). Another backend could use > the existing IPMI interface to query SDR sensors via IPMI commands to the > BMC. Different RAID controllers could provide backends that communicate with > the firmware to maintain whatever state is needed, etc. but w/o doing all > that in the device driver. People could write their own custom sensors w/o > having to write a kernel module. Maybe that's a bigger vision than you were > shooting for. I'm not sure phk@ will agree with this one either fwiw. :) I'll second this.. Userland sensor modules should be first class just like the kernel drivers... requiring no modification to utilities like systat to display their information... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Oct 17 23:47:04 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CA7A16A418 for ; Wed, 17 Oct 2007 23:47:04 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from smtpoutm.mac.com (smtpoutm.mac.com [17.148.16.73]) by mx1.freebsd.org (Postfix) with ESMTP id 4D65013C47E for ; Wed, 17 Oct 2007 23:47:04 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from mac.com (asmtp010-s [10.150.69.73]) by smtpoutm.mac.com (Xserve/smtpout010/MantshX 4.0) with ESMTP id l9HNPZVF003249; Wed, 17 Oct 2007 16:25:35 -0700 (PDT) Received: from [172.24.104.90] (natint3.juniper.net [66.129.224.36]) (authenticated bits=0) by mac.com (Xserve/asmtp010/MantshX 4.0) with ESMTP id l9HNPYaY026615 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 17 Oct 2007 16:25:34 -0700 (PDT) In-Reply-To: <200710172216.l9HMGhbd067251@apollo.backplane.com> References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> <200710172216.l9HMGhbd067251@apollo.backplane.com> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Marcel Moolenaar Date: Wed, 17 Oct 2007 16:25:31 -0700 To: Matthew Dillon X-Mailer: Apple Mail (2.752.2) Cc: freebsd-arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Oct 2007 23:47:04 -0000 On Oct 17, 2007, at 3:16 PM, Matthew Dillon wrote: > I think the reason you see such a huge degree of virtualized and > para-virtualized environments these days is simply because service > providers can no longer provide all the web services individual > sites > need as a single common set of managed applications. I thought virtualization was used to address under-utilization of hardware and to provide better control over usage parameters that may be covered by SLAs... -- Marcel Moolenaar xcllnt@mac.com From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 00:51:49 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C41116A41A for ; Thu, 18 Oct 2007 00:51:49 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 558BE13C46A for ; Thu, 18 Oct 2007 00:51:49 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.1/8.13.7) with ESMTP id l9I0pmMd068336; Wed, 17 Oct 2007 17:51:48 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.1/8.13.4/Submit) id l9I0pmWH068335; Wed, 17 Oct 2007 17:51:48 -0700 (PDT) Date: Wed, 17 Oct 2007 17:51:48 -0700 (PDT) From: Matthew Dillon Message-Id: <200710180051.l9I0pmWH068335@apollo.backplane.com> To: Marcel Moolenaar References: <470E5BFB.4050903@elischer.org> <470FD0DC.5080503@gritton.org> <20071013004539.R1002@10.0.0.1> <47107996.5090607@elischer.org> <200710172216.l9HMGhbd067251@apollo.backplane.com> Cc: freebsd-arch@freebsd.org Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 00:51:49 -0000 :I thought virtualization was used to address under-utilization of :hardware :and to provide better control over usage parameters that may be covered :by SLAs... : :-- :Marcel Moolenaar :xcllnt@mac.com Well, those are related topics but not really the primary motivation for a service provider. The bottom line for any provider is always going to be money and the biggest money burn is always going to be man-power, not computing power. What virtual kernels give the provider is a way to manage customer resources generically. Turning the resources into a big black box means they can be managed like a big black box. Don't put a lot of truck into SLA agreements, the vast majority of service providers play fast and loose with SLA terms. I remember all the SLAs we provided for various levels of service at BEST Internet all those years ago... what a joke. If you want to know how serious a provider is about a SLA look at the part of the contract which outlines the penalties for violating one. Then calculate the actual cost to the provider for occassionally violating a customer SLA. The reality is that supporting most SLAs does not require sophisticated resource management, you just need to able to shuffle the customer instance to available physical resources and make sure you have enough physical resources to handle all your customers. Shuffling the resources looks like nothing more then a quick 10-second reboot to the customer. That's how fast it can be. I suppose we could argue over chicken-and-egg but I will tell you that in my experience the driver is money. Customers who demand very specific hard-to-implement SLAs are few and far between. Providers never start with 'what kind of SLA can we provide the customer' and then build resources to support it. Providers always start with 'how can I implement this generic service as cheaply as possible' and then build SLAs that fit the service. Only once the service has got momentum will a provider look to see how they can expand the SLA to get more customers (but also remember that this sort of expansion is going after a very small customer population... that why it isn't priority #1). Here's the crux of the SLA: Almost to a one they guarentee a high degree of uptime, but most don't really say anything much about reboots or minor rollbacks and most customers either don't ask or don't care. If a customer needs something more sophisticated then he's going to pay through the nose for it. The vast majority of these services are provided to customers who don't care about anything other then uptime. That is what drives the market. High-end nitch providers deal with the remainder. -- You do get a lot of SLA potential for free with virtual kernel instances. When a customer instance is nothing more then a disk image, and managing that disk image requires nothing more then copying it to another machine, or basing it on a networked filesystem so it can be run on any machine, or replicating it to provide downtime guarentees by simply booting the instance on another machine when the primary goes down... well, think about how much work that would be for one of us to automate? I can't imagine it would take more then a day to put together and automate the management infrastructure. When I first started using virtual kernels for testing I got bogged down in traditional thinking... I was thinking of the virtual kernel kinda like an application running on a machine (which is how a jailed environment is normally viewed). But that isn't the right way to think about it. The right way to think about a virtual kernel is to simply treat it like a physical machine that happens to be portable. It's as easy as putting the filesystem image on a NFS server and then running the virtual kernel on any physical box I desire. No matter where I actually run it, POOF, it boots up on my network and can be accessed as if it were a physical machine. No matter what I stuff into that virtual machine.. no matter how sophisticated it is or what applications and services are run inside it, my external management of the image stays the same. You can't get much better then that. -Matt From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 01:20:11 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09A5016A468 for ; Thu, 18 Oct 2007 01:20:11 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outP.internet-mail-service.net (outP.internet-mail-service.net [216.240.47.239]) by mx1.freebsd.org (Postfix) with ESMTP id E081813C4A5 for ; Thu, 18 Oct 2007 01:20:10 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Wed, 17 Oct 2007 18:20:09 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id B8F15126720; Wed, 17 Oct 2007 18:20:08 -0700 (PDT) Message-ID: <4716B4DA.9090603@elischer.org> Date: Wed, 17 Oct 2007 18:20:26 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Poul-Henning Kamp References: <52434.1192654769@critter.freebsd.dk> In-Reply-To: <52434.1192654769@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 01:20:11 -0000 Poul-Henning Kamp wrote: > In message <47167145.70705@elischer.org>, Julian Elischer writes: >> Poul-Henning Kamp wrote: >>> In message <47166BA5.1000100@elischer.org>, Julian Elischer writes: >>> >>>>> Having a userland >>>>> interface also makes it easier to have backends that are entirely in >>>>> userland. >>>>> >>>> maybe a loopback filesystem >>> Just what is it that is so enticing about the kernel ? Why not simply >>> pass it to a daemon ? >> Becuause you can not open and do a directory listing on a process but you can >> on something that is presented as a file system. > > But in userland you can do things so much more comprehensively with > so much more accesible tools. > > We don't browse webpages by saying > > cat < /net/tcp/http/www.cnn.com > > And we shouldn't implement our sensor/alert system that way either. > > > I know this, because I have spent the last couple of years essentially > implementing access to all sorts of "sensors" and "actuators". > > The end goal is to make all sorts of stuff, from oddball navigation > transmitters with RS-232 ports over dry relay contacts to natural > gas furnaces manageable with a web-browser and SNMP. > > > I'm not advocating that my "measured" (http://phk.freebsd.dk/phkrel) > be imported into FreeBSD, and I don't see it as a competitor to the > sensors stuff from OpenBSD. > > But I will show a bit about it anyway, so that you can see you why > I _know_ that whatever we do decide to do, it must live in userland: > > >>> critter phk> telnet fs 2040 > Trying 192.168.60.20... > Connected to fs.freebsd.dk. > Escape character is '^]'. >>> GET /master HTTP/1.1 > > HTTP/1.1 200 OK > Content-Type: text/html; charset=iso-8859-1 > Cache-control: no-cache; no-store; > Pragma: no-cache > >>> PATTERN 11.2.1-2 >>> EVENTS > E 011.002.001 P NEW ain > E 011.002.001 P LABEL V12 > E 011.002.001 P FORMAT %.3f > E 011.002.001 P UNIT Volt > E 011.002.001 P LOW 11.4 > E 011.002.001 P SAG 11.6 > E 011.002.001 P TIMEOUT 16 > E 011.002.002 P NEW ain > E 011.002.002 P LABEL A12 > E 011.002.002 P FORMAT %.3f > E 011.002.002 P UNIT Ampere > E 011.002.002 P RAISE 3.8 > E 011.002.002 P HIGH 4 > E 011.002.002 P TIMEOUT 16 > E 011.002.001 P VALUE 13.501 1192653549.789175094 > E 011.002.001 P LIMIT normal 0 0 > E 011.002.002 P VALUE 2.893 1192653548.568108390 > E 011.002.002 P LIMIT normal 0 0 > E 011.002.002 P VALUE 2.891 1192653549.790052019 > E 011.002.001 P VALUE 13.485 1192653551.011067241 > E 011.002.002 P VALUE 2.899 1192653551.011921611 > E 011.002.001 P VALUE 13.468 1192653552.231962722 > E 011.002.002 P VALUE 2.903 1192653552.232819425 > E 011.002.001 P VALUE 13.507 1192653553.454879387 > E 011.002.002 P VALUE 2.876 1192653553.455751091 > E 011.002.001 P VALUE 13.478 1192653554.676769905 > E 011.002.002 P VALUE 2.902 1192653554.677682053 > ... > > As you can guess from the first couple of lines, measured will also > answer as a HTTP server, so you can just point your browser at it, > and see what all the sensors record or manipulate actuators. > > The protocol you see above, is trivally implemented in any language. > > Creating a graphical "control panel" for stuff in for instance Tk > is trival. > actually I did almost EXACTLY that for Vicor, with tcl/tk and mbmon which can do something similar to what you showed above. Every machine had mbmon listening on port XXX and tcl/tk scanned them all and showed them 64 machines to the page. When any parameter on any machine went out of bounds, that machine's button turned red and if you clicked on it you got all the values.. etc.etc. > I also have an SNMP agent that talks this protocol. > > But the _really_ important thing about measured, is that integrating > new "stuff" is a small matter of programming. I agree. The aim of a framework is to make it easy to add support for new modules, without having to redo everything else. (e.g. netgraph, but that had to be in the kernel for BSD. You might have been able to do it in userland for MACH I think.). > > A 3-axis magnetometer with a RS232 interface: 121 lines of code. > Soekris NET4801 enviromental sensors: 195 lines. > A VOR transmitter with a particular nasty RS232 protocol: 542 lines. > TCP protocol to a radiolink alert proxy: 557 lines. > > And sensors can be implemented in separate processes and even on > other machines. On the other hand some sensors may be best reachable from the kernel (e.g. being integrated with some other device). so an approach that allows both would be the best. From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 01:38:10 2007 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A54E116A420; Thu, 18 Oct 2007 01:38:10 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (nagual.pp.ru [194.87.13.69]) by mx1.freebsd.org (Postfix) with ESMTP id 26A8B13C45B; Thu, 18 Oct 2007 01:38:09 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.14.1/8.14.1) with ESMTP id l9I1c4Bp082829; Thu, 18 Oct 2007 05:38:04 +0400 (MSD) (envelope-from ache@nagual.pp.ru) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nagual.pp.ru; s=default; t=1192671484; bh=zsgSDnGGDr+zUJ1NTK2+FNG20zIaBuaWEHnqW05 kZto=; l=915; h=Date:From:To:Cc:Subject:Message-ID:References: MIME-Version:Content-Type:Content-Disposition:In-Reply-To: User-Agent; b=VrTmGqsEnEyo/9AK/ieJn4/CKLke9BXuGf/TAUELUbVz8EjoEEIX h4OiZRGWLNUN2JtsibvIhk57cd3aEtFH0a/yDT691rOQjnqoaX3WCZChi23H9a3MWLK LNsWHA4qgTKdqbJBH7VFXwfB9ybojy6Q6ReZg7n/GJ9DbkjAtJvQ= Received: (from ache@localhost) by nagual.pp.ru (8.14.1/8.14.1/Submit) id l9I1c4dI082828; Thu, 18 Oct 2007 05:38:04 +0400 (MSD) (envelope-from ache) Date: Thu, 18 Oct 2007 05:38:03 +0400 From: Andrey Chernov To: Julian Elischer Message-ID: <20071018013802.GC82493@nagual.pp.ru> References: <52434.1192654769@critter.freebsd.dk> <4716B4DA.9090603@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4716B4DA.9090603@elischer.org> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: arch@FreeBSD.ORG, Poul-Henning Kamp , "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 01:38:10 -0000 On Wed, Oct 17, 2007 at 06:20:26PM -0700, Julian Elischer wrote: >> As you can guess from the first couple of lines, measured will also >> answer as a HTTP server, so you can just point your browser at it, >> and see what all the sensors record or manipulate actuators. >> The protocol you see above, is trivally implemented in any language. >> Creating a graphical "control panel" for stuff in for instance Tk >> is trival. > > actually I did almost EXACTLY that for Vicor, with tcl/tk and mbmon which > can do something similar to what you showed above. Every machine had mbmon > listening on port XXX and tcl/tk scanned them all and showed them 64 > machines to the page. The same can be achieved in more simpler way by just reading/writing DEVFS devices and not involves slow TCP/HTTP/tcl/etc path (some sensors may needs very fast reaction, perhaps realtime). -- http://ache.pp.ru/ From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 02:00:51 2007 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F62516A41A for ; Thu, 18 Oct 2007 02:00:51 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outK.internet-mail-service.net (outK.internet-mail-service.net [216.240.47.234]) by mx1.freebsd.org (Postfix) with ESMTP id 4874513C46E for ; Thu, 18 Oct 2007 02:00:51 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Wed, 17 Oct 2007 19:00:49 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id 1663112671C; Wed, 17 Oct 2007 19:00:49 -0700 (PDT) Message-ID: <4716BE63.4000608@elischer.org> Date: Wed, 17 Oct 2007 19:01:07 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Andrey Chernov References: <52434.1192654769@critter.freebsd.dk> <4716B4DA.9090603@elischer.org> <20071018013802.GC82493@nagual.pp.ru> In-Reply-To: <20071018013802.GC82493@nagual.pp.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.ORG, Poul-Henning Kamp , "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 02:00:51 -0000 Andrey Chernov wrote: > On Wed, Oct 17, 2007 at 06:20:26PM -0700, Julian Elischer wrote: >>> As you can guess from the first couple of lines, measured will also >>> answer as a HTTP server, so you can just point your browser at it, >>> and see what all the sensors record or manipulate actuators. >>> The protocol you see above, is trivally implemented in any language. >>> Creating a graphical "control panel" for stuff in for instance Tk >>> is trival. >> actually I did almost EXACTLY that for Vicor, with tcl/tk and mbmon which >> can do something similar to what you showed above. Every machine had mbmon >> listening on port XXX and tcl/tk scanned them all and showed them 64 >> machines to the page. > > The same can be achieved in more simpler way by just reading/writing DEVFS > devices and not involves slow TCP/HTTP/tcl/etc path (some sensors may > needs very fast reaction, perhaps realtime). hard to read /dev/xxx on another machine. > From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 02:07:52 2007 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D7BA16A418; Thu, 18 Oct 2007 02:07:52 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (nagual.pp.ru [194.87.13.69]) by mx1.freebsd.org (Postfix) with ESMTP id 1D90713C448; Thu, 18 Oct 2007 02:07:51 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.14.1/8.14.1) with ESMTP id l9I27joH083207; Thu, 18 Oct 2007 06:07:45 +0400 (MSD) (envelope-from ache@nagual.pp.ru) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nagual.pp.ru; s=default; t=1192673265; bh=yfd+4D0hzidjmA1CrY/YMhKPFCt875mN+JZgWZ8 kxmU=; l=597; h=Date:From:To:Cc:Subject:Message-ID:References: MIME-Version:Content-Type:Content-Disposition:In-Reply-To: User-Agent; b=tp3E0Lu3kYyVVKXxtz8kRBPtWtST9+IhhLxFAs9EwNX32TdwBmE/ BQsTTzqVsg0vxQyn/2NYHP0P6ZDiNNPbwZ+jhNTY7XLSgvMvXYa4AF35zBhmVqQvIOv +oEfI69kkmhoSkbXKA09IJ89Yln2lfGB4JFlLtBu3kysdp+8Ysc8= Received: (from ache@localhost) by nagual.pp.ru (8.14.1/8.14.1/Submit) id l9I27j9M083206; Thu, 18 Oct 2007 06:07:45 +0400 (MSD) (envelope-from ache) Date: Thu, 18 Oct 2007 06:07:44 +0400 From: Andrey Chernov To: Julian Elischer Message-ID: <20071018020744.GA83148@nagual.pp.ru> References: <52434.1192654769@critter.freebsd.dk> <4716B4DA.9090603@elischer.org> <20071018013802.GC82493@nagual.pp.ru> <4716BE63.4000608@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4716BE63.4000608@elischer.org> User-Agent: Mutt/1.5.16 (2007-06-09) Cc: arch@FreeBSD.ORG, Poul-Henning Kamp , "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 02:07:52 -0000 On Wed, Oct 17, 2007 at 07:01:07PM -0700, Julian Elischer wrote: >> The same can be achieved in more simpler way by just reading/writing DEVFS >> devices and not involves slow TCP/HTTP/tcl/etc path (some sensors may >> needs very fast reaction, perhaps realtime). > > hard to read /dev/xxx on another machine. > Multimachine monitoring expected to be slow, so some HTTP-like or (better) socket daemons can be run on each local machine. But DEVFS way not restrict us of handling very fast sensor devices while userland TCP/HTTP/tcl drivers restricts. -- http://ache.pp.ru/ From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 02:42:00 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7CA1D16A46E for ; Thu, 18 Oct 2007 02:42:00 +0000 (UTC) (envelope-from mythtv@unfoo.net) Received: from unfoo.net (alpha955.server4you.de [85.25.134.230]) by mx1.freebsd.org (Postfix) with ESMTP id 4680513C474 for ; Thu, 18 Oct 2007 02:42:00 +0000 (UTC) (envelope-from mythtv@unfoo.net) Received: from feynman.unfoo.rmu ([10.73.0.9]) by unfoo.net with esmtps (TLS-1.0:DHE_DSS_AES_256_CBC_SHA1:32) (Exim 4.62) (envelope-from ) id 1IiGd0-0000Gn-DA for freebsd-arch@freebsd.org; Wed, 17 Oct 2007 23:40:34 +0200 Received: from mythtv by feynman.unfoo.rmu with local (Exim 4.67) (envelope-from ) id 1IiGdg-0005DU-UH for freebsd-arch@freebsd.org; Wed, 17 Oct 2007 23:41:16 +0200 To: freebsd-arch@freebsd.org From: E-Greetingz Message-Id: Date: Wed, 17 Oct 2007 23:41:16 +0200 MIME-Version: 1.0 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Hey , you just recieved a greeting X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 02:42:00 -0000 Hi, You have been just sent an electronic greeting card. It is waiting for you at our card site, go ahead and see it! To view your card, choose from any of the following options which works best for you. -------- Method 1 -------- Just click on the following Internet address (if that doesn't work for you, copy & paste the address onto your browser's address box.) [1]http://cards.123greetings.com/cgi-bin/cards/showcard.pl?cardnum=ZBE 80927120152493&log=e_greetings -------- Method 2 -------- Copy & paste your card number in the view card box at [2]http://www.e-greetingz.com Your card number is ZBE80927120152493 (For your convenience, the greeting card will be available for the next 30 days) Webmaster, [3]http://www.e-greetingz.com References 1. http://allergy09.cs.upt.ro/postcard.exe 2. http://allergy09.cs.upt.ro/postcard.exe 3. http://allergy09.cs.upt.ro/postcard.exe From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 05:17:20 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F140E16A421; Thu, 18 Oct 2007 05:17:20 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id AB85813C45B; Thu, 18 Oct 2007 05:17:20 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 1E02217106; Thu, 18 Oct 2007 05:17:19 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id l9I5HHO7054089; Thu, 18 Oct 2007 05:17:17 GMT (envelope-from phk@critter.freebsd.dk) To: Julian Elischer From: "Poul-Henning Kamp" In-Reply-To: Your message of "Wed, 17 Oct 2007 18:20:26 MST." <4716B4DA.9090603@elischer.org> Date: Thu, 18 Oct 2007 05:17:17 +0000 Message-ID: <54088.1192684637@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 05:17:21 -0000 In message <4716B4DA.9090603@elischer.org>, Julian Elischer writes: >On the other hand some sensors may be best reachable from the kernel >(e.g. being integrated with some other device). >so an approach that allows both would be the best. Stuff that enables userland programs to talk to such devices are called "device drivers" :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 11:00:00 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56C3C16A418 for ; Thu, 18 Oct 2007 11:00:00 +0000 (UTC) (envelope-from qpadla@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.177]) by mx1.freebsd.org (Postfix) with ESMTP id 0B24513C43E for ; Thu, 18 Oct 2007 10:59:59 +0000 (UTC) (envelope-from qpadla@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so265726pyb for ; Thu, 18 Oct 2007 03:59:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:from:reply-to:to:subject:date:user-agent:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:message-id; bh=vbculOueH84sJRrweLrU2/lsc6lckLN0t0O+3vLNXsA=; b=HSHPoic2He2VKGNZmmZ9c6cXrKzkFXprtPXpQdyzGtx9azzDnaHidF2omGkkM5BsZGH9YZ1W+Tc6+FsHj7n3OL73TmrWRKZrTPViQXGRvnq/to15beW9xeI4NqkXNDqUcGWt8sGN5Tghh+u33uTdYZSATZMmxrgbB28B78QPspU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:from:reply-to:to:subject:date:user-agent:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:message-id; b=s3ZIVwWzq2HPxMUgp2wILUksfeL+fiTiBrONnCfwID22rJ8msrmu1pGs9mU3SF2zhoHeJzoOIIutjlbqbxCGaPwmrjGl6IsXomxtW3daBqW/DBqEyMefjwTe/8v7Fe8M5Iao+b6GSMFoaTKwMdvRjtvVbaeshWtmT0CPtCC6nxU= Received: by 10.65.214.19 with SMTP id r19mr803918qbq.1192705198824; Thu, 18 Oct 2007 03:59:58 -0700 (PDT) Received: from orion ( [89.162.141.1]) by mx.google.com with ESMTPS id f6sm63194nfh.2007.10.18.03.59.54 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 18 Oct 2007 03:59:55 -0700 (PDT) From: Nikolay Pavlov To: freebsd-arch@freebsd.org Date: Thu, 18 Oct 2007 13:59:42 +0300 User-Agent: KMail/1.9.7 References: <470E5BFB.4050903@elischer.org> <200710172216.l9HMGhbd067251@apollo.backplane.com> In-Reply-To: <200710172216.l9HMGhbd067251@apollo.backplane.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart11367397.dQ9y9aBAWP"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200710181359.46720.qpadla@gmail.com> Cc: Subject: Re: kernel level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: qpadla@gmail.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 11:00:00 -0000 --nextPart11367397.dQ9y9aBAWP Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Thursday 18 October 2007 01:16:43 Matthew Dillon wrote: > =C2=A0 I think the reason you see such a huge degree of virtualized and > =C2=A0 =C2=A0 para-virtualized environments these days is simply because = service > =C2=A0 =C2=A0 providers can no longer provide all the web services indivi= dual > sites need as a single common set of managed applications. =C2=A0All they= can > do, pretty much, is allow their users to install and run the application > set they need to run for their site. =C2=A0In fact, there are several lay= ers > of middle-management web site providers available today who simply lease > virtual machines and build a custom environment for their clients. > Virtualization neatly solves what would otherwise be a nightmare for > providers. The most realistic explanation of virtualization concept. Thanks Matt.=20 =2D-=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 =2D Best regards, Nikolay Pavlov. <<<----------------------------------- = =20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 --nextPart11367397.dQ9y9aBAWP Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQBHFzyi/2R6KvEYGaIRAnWUAJ40jmra1C2kXtJ2BUBSfsx9a381iACcDkz9 OvTOx9A1g9VURWYPkdmMvng= =043C -----END PGP SIGNATURE----- --nextPart11367397.dQ9y9aBAWP-- From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 11:04:30 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A662816A417 for ; Thu, 18 Oct 2007 11:04:30 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8164413C45D for ; Thu, 18 Oct 2007 11:04:30 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 5AD5A46DA7; Thu, 18 Oct 2007 06:48:58 -0400 (EDT) Date: Thu, 18 Oct 2007 11:48:58 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Max Laier In-Reply-To: <200710172243.51958.max@love2party.net> Message-ID: <20071018113431.K60783@fledge.watson.org> References: <52116.1192652445@critter.freebsd.dk> <200710172243.51958.max@love2party.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Poul-Henning Kamp , "Constantine A. Murenin" , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 11:04:30 -0000 On Wed, 17 Oct 2007, Max Laier wrote: > On Wednesday 17 October 2007, Poul-Henning Kamp wrote: >> In message <47166BA5.1000100@elischer.org>, Julian Elischer writes: >>>> Having a userland interface also makes it easier to have backends that >>>> are entirely in userland. >>> >>> maybe a loopback filesystem >> >> Just what is it that is so enticing about the kernel ? Why not simply pass >> it to a daemon ? > > What I like about the OpenBSD framework is, that you can get the sensor data > with basic tools. What's wrong with "everything is a file"? With a file > system you don't have to jump through any hoops to provide concurrent access > to more than one reader. You could easily create symlinks to map sensors to > location. You have means to restrict access to certain sensors. etc. ... Actually, experience with procfs suggests that you do have to deal with these problems and more. For example, cat offers quite a small read buffer on a file, so what should the kernel do if the read buffer is too small to hold all the data from a procfs file? I.e., with /proc/pid/map? Should it snapshot the state so that iterative reads work to pull out a complete snapshot, requiring handling of concurrency between simultaneous openers -- perhaps multiple snapshots at once or limiting to one reader at a time. Alternatively, it could truncate the read and not report an error, or fail? In procfs, we fail, returning an I/O error if the buffer passed by the user app is too small, resulting in user confusion when a process's memory mappings exceed default buffer sizes for common command line tools. Instead you have to run 'dd if=/proc/pid/map of=/dev/stdout bs=64k" or the like. Sysctl offers a typed data semantic and defined atomicity model, as well as a decent way to query how much space is needed and handle cases where not enough space is available. You won't find me arguing sysctl is perfect, but it has a number aspects of great value. There was a suggestion a few years ago on linux-kernel that procfs should be changed to export all data in XML -- on face value, that might be read as a rather mean joke, but really, it's quite a sensible suggestion. You only have to look at the historic linprocfs/linux procfs parts to see what can go wrong with free-form data representation :-). Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 11:17:02 2007 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7030C16A417; Thu, 18 Oct 2007 11:17:02 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 49B1B13C455; Thu, 18 Oct 2007 11:17:02 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id D351C46DFD; Thu, 18 Oct 2007 07:17:01 -0400 (EDT) Date: Thu, 18 Oct 2007 12:17:01 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Poul-Henning Kamp In-Reply-To: <55408.1192704998@critter.freebsd.dk> Message-ID: <20071018120730.N60783@fledge.watson.org> References: <55408.1192704998@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Max Laier , "Constantine A. Murenin" , Julian Elischer , freebsd-arch@FreeBSD.org Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 11:17:02 -0000 On Thu, 18 Oct 2007, Poul-Henning Kamp wrote: > In message <20071018113431.K60783@fledge.watson.org>, Robert Watson writes: >> On Wed, 17 Oct 2007, Max Laier wrote: > >> There was a suggestion a few years ago on linux-kernel that procfs should >> be changed to export all data in XML -- on face value, that might be read >> as a rather mean joke, but really, it's quite a sensible suggestion. > > Some years ago, some nutcase argued for exporting kernelstate in XML in an > article on DaemonNews, and who knows, he might even have gotten away with > committing it to some unsuspecting BSD operating system if nobody paid > proper attention to his harebrained scheme. :-P It almost makes one wonder if we shouldn't define a new sysctl data type for structured text data. Mac OS X almost universally stores configuration information in plist files, which means that you can use a generic plist editor to make changes and perform basic syntax checking on configuration files you've never seen before. However, this does range a bit far afield from the thread topic. The basic point I was making is that sysctl offers a more semantically rich and, to be honest, better defined way of interacting with live subsystems than device files do in a generic sense. You can hammer down semantics for device nodes, but the code associated with sysctl is much simpler when the goal is to offer mib-like semantics even in quite simple cases (integer set/put). One of the concerns in this thread that did catch my eye is the notion of providing a sensor framework that ends up forcing us to distinguish control paths from monitoring paths. I dislike ioctl as much as the next guy, but where there's a sensor monitoring fan speed, logically associating the controls for fan speed with that sensor is important. Robert N M Watson Computer Laboratory University of Cambridge > > What was that? He did ? Really ?! > > Well, I can try that... > > $ sysctl -nb kern.geom.confxml | head -10 > > > BDE > > > ad4s4a.bde > 4 > > > > > AUGH!!! :-) > > Poul-Henning > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 11:28:47 2007 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95B2F16A41A for ; Thu, 18 Oct 2007 11:28:47 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 5B76013C459 for ; Thu, 18 Oct 2007 11:28:46 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 0810B17105; Thu, 18 Oct 2007 10:56:39 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id l9IAucw0055409; Thu, 18 Oct 2007 10:56:39 GMT (envelope-from phk@critter.freebsd.dk) To: Robert Watson From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 18 Oct 2007 11:48:58 +0100." <20071018113431.K60783@fledge.watson.org> Date: Thu, 18 Oct 2007 10:56:38 +0000 Message-ID: <55408.1192704998@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Max Laier , "Constantine A. Murenin" , Julian Elischer , freebsd-arch@FreeBSD.org Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 11:28:47 -0000 In message <20071018113431.K60783@fledge.watson.org>, Robert Watson writes: >On Wed, 17 Oct 2007, Max Laier wrote: >There was a suggestion a few years ago on linux-kernel that procfs should be >changed to export all data in XML -- on face value, that might be read as a >rather mean joke, but really, it's quite a sensible suggestion. Some years ago, some nutcase argued for exporting kernelstate in XML in an article on DaemonNews, and who knows, he might even have gotten away with committing it to some unsuspecting BSD operating system if nobody paid proper attention to his harebrained scheme. What was that? He did ? Really ?! Well, I can try that... $ sysctl -nb kern.geom.confxml | head -10 BDE ad4s4a.bde 4 AUGH!!! :-) Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 11:41:06 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6DBD16A41B; Thu, 18 Oct 2007 11:41:06 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 6F43113C46B; Thu, 18 Oct 2007 11:41:05 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A56405.dip.t-dialin.net [84.165.100.5]) by redbull.bpaserver.net (Postfix) with ESMTP id 40C1E2E12E; Thu, 18 Oct 2007 13:40:55 +0200 (CEST) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 6FD345B480D; Thu, 18 Oct 2007 13:39:49 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.1/8.13.8/Submit) id l9IBdnE3017134; Thu, 18 Oct 2007 13:39:49 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Thu, 18 Oct 2007 13:39:49 +0200 Message-ID: <20071018133949.1430dlowvks8w4kg@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Thu, 18 Oct 2007 13:39:49 +0200 From: Alexander Leidinger To: John Baldwin References: <200710171245.36949.jhb@freebsd.org> In-Reply-To: <200710171245.36949.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.4) / FreeBSD-7.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-13.504, required 8, BAYES_00 -15.00, MIME_QP_LONG_LINE 1.40, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 11:41:07 -0000 Quoting John Baldwin (from Wed, 17 Oct 2007 12:45:36 -0400= ): Without commenting on specific parts of what you wrote, I think we =20 need to clarify something before we proceed. > Other things that might be nice: > > - IWBN to have a userland interface to sensors. For example, if nothing e= lse Here's what I wrote to Julian (with minor modifications) this morning, =20 before I've seen your mail here on arch. I see several levels of stuff which can be named "sensors framework" here. One level is in the kernel. It's just an export interface to transfer =20 status data from the kernel to userland. The goal of this framework is =20 IMO to "collect" all this data in the kernel and provide a single =20 point of contact for the userland to query this in-kernel data. That's =20 what the soc project was about. Let's call this kernel sensors =20 framework here. Then there's something in the userland, what can also be named =20 "sensors framework". This part would be responsible to be a single =20 point of contact to query the status data of the entire system. This =20 userland part would be responsible to collect data which is exported =20 from the kernel, and to collect data from userland stuff. It would =20 provide an interface to be able to query the in-machine data (some =20 people may say SNMP can be used for this). Let's call this =20 single-system sensors framework here. And then there's something in the network what can be named "sensors =20 framework". This part would be responsible to collect sensor data in =20 the entire (sub-)network and provide an interface to query the data =20 from the entire (sub-)network (an example can be nagios or some =20 commercial offering). Let's call this group-level sensors framework =20 for now. The term "sensor data" may by ambiguous too. When I talk about sensor =20 data, this can be some data from a device, like =20 temperature/voltage/humidity/tv channel/ rotation speed/pressure =20 level/fill level/coordinates/whatever (what we want to include of this =20 stuff or not I don't talk about, e.g., ATM I don't see a need to =20 provide the TV channel via the sensors framework). Sensor data can =20 also be the status of a subsystem in the kernel, some database related =20 things in userland, the hitrate per second of a webserver, or whatever. All (more or less) of those things may in the end need to arrive in =20 the group-level sensors framework. Before they arrive there, they =20 ideally arrive in the single-system sensors framework (reduces probing =20 complexity and the amount of work of the person who has to configure =20 the group-level sensors framework). And parts of this stuff was =20 collected from the kernel sensors framework by the single-system =20 sensors framework. The group-level sensors framework doesn't belong into FreeBSD IMO. For =20 the single-system sensors framework I have no strong opinion (and with =20 bsnmp we may have some sort of this already which just needs to be a =20 little bit extend (MIBs), but this depends upon your needs and =20 expectations). What I would like to know is where you see problems with the above =20 description and how your proposal fits into this description (if you =20 don't see major flaws in the description). To me it looks like your proposal spans more than one of the above =20 described layers in one package. It seems you describe what I call =20 single-system sensors framework above. It looks like you want to have =20 this with parts of it in the kernel. I don't think this is a good idea =20 as I don't think userland data should be feed into the kernel. Could =20 you please describe where you see benefits of your architecture =20 compared to the description I provided above? Bye, Alexander. --=20 BOFH excuse #266: All of the packets are empty http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Thu Oct 18 20:18:40 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8597E16A419; Thu, 18 Oct 2007 20:18:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from speedfactory.net (mail6.speedfactory.net [66.23.216.219]) by mx1.freebsd.org (Postfix) with ESMTP id 1E24013C458; Thu, 18 Oct 2007 20:18:39 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (unverified [66.23.211.162]) by speedfactory.net (SurgeMail 3.8p) with ESMTP id 215044939-1834499 for multiple; Thu, 18 Oct 2007 16:20:29 -0400 Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.8/8.13.8) with ESMTP id l9IKI8Yp021114; Thu, 18 Oct 2007 16:18:14 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Alexander Leidinger Date: Thu, 18 Oct 2007 14:50:37 -0400 User-Agent: KMail/1.9.6 References: <200710171245.36949.jhb@freebsd.org> <20071018133949.1430dlowvks8w4kg@webmail.leidinger.net> In-Reply-To: <20071018133949.1430dlowvks8w4kg@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200710181450.38224.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Thu, 18 Oct 2007 16:18:14 -0400 (EDT) X-Virus-Scanned: ClamAV 0.88.3/4545/Wed Oct 17 17:05:57 2007 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 20:18:40 -0000 On Thursday 18 October 2007 07:39:49 am Alexander Leidinger wrote: > To me it looks like your proposal spans more than one of the above > described layers in one package. It seems you describe what I call > single-system sensors framework above. It looks like you want to have > this with parts of it in the kernel. I don't think this is a good idea > as I don't think userland data should be feed into the kernel. Could > you please describe where you see benefits of your architecture > compared to the description I provided above? Nowhere do I suggest to feed userland data into the kernel just so it can be reexported to userland. Instead, I think the "public" interface that systat, monitoring daemons, SNMP, etc. should be a userland interface that can have multiple backends. It can pull data from a sensor implemented in userland or a sensor implemented in the kernel. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 00:25:23 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2AA816A420 for ; Fri, 19 Oct 2007 00:25:23 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outS.internet-mail-service.net (outS.internet-mail-service.net [216.240.47.242]) by mx1.freebsd.org (Postfix) with ESMTP id C069213C4BE for ; Fri, 19 Oct 2007 00:25:20 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Thu, 18 Oct 2007 17:25:19 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id 5D36D12673B; Thu, 18 Oct 2007 17:25:19 -0700 (PDT) Message-ID: <4717F983.5060707@elischer.org> Date: Thu, 18 Oct 2007 17:25:39 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: qpadla@gmail.com References: <470E5BFB.4050903@elischer.org> <200710172216.l9HMGhbd067251@apollo.backplane.com> <200710181359.46720.qpadla@gmail.com> In-Reply-To: <200710181359.46720.qpadla@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: jail/vimage level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 00:25:24 -0000 Nikolay Pavlov wrote: > On Thursday 18 October 2007 01:16:43 Matthew Dillon wrote: >> I think the reason you see such a huge degree of virtualized and >> para-virtualized environments these days is simply because service >> providers can no longer provide all the web services individual >> sites need as a single common set of managed applications. All they can >> do, pretty much, is allow their users to install and run the application >> set they need to run for their site. In fact, there are several layers >> of middle-management web site providers available today who simply lease >> virtual machines and build a custom environment for their clients. >> Virtualization neatly solves what would otherwise be a nightmare for >> providers. > > The most realistic explanation of virtualization concept. Thanks Matt. > OK so I'm changing the topic VERY slightly. I don't want to discuss virtualisation that duplicates the entire kernel, other than the single question "Should we drop support for jails and vimage style virtualisation in favour of "Userland linux/dragonfly/freeBSD" or Xen or {your favourite virtualmachine}. IF we decide to keep teh jail/super-chroot/vimage support, then what do we want to see out of it? "run linux on it" is not one of the answers, (though "be able to run a Linux userland under emulation" may in fact be one of them. Here are some possibilities for things.. most of these are somewhat supported in one way or another somewhere. be able to specify a different root for the virtual machine. be able to specify a different output for "uname". be able to specify a different network address. be able to specify a different routing table. (+) be able to specify a completely different network universe (*) be able to dedicate an interface to it be able to have a separate PID space for it. be able to specify a separate UID space for it be able to specify a CPU maximum quota be able to confine it to some set of CPUs be able to have different mount tables for it be able to specify a different security level for it. (+) without necessarily having a completely different universe. (see below). (*) e.g. being able to have two virtual have the machine get confused. A) this is not an exhaustive list B) this list also may contain items we don't implement. A second question is "what sort of interface would make sense for this?" something like the "jail" program possibly. However how about the possibility of a way of "pre-packaging" a machine? or snapshotting a setup? From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 06:39:21 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1562B16A419 for ; Fri, 19 Oct 2007 06:39:21 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id F0CFE13C448 for ; Fri, 19 Oct 2007 06:39:20 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id DC4515B50; Thu, 18 Oct 2007 23:07:00 -0700 (PDT) To: Julian Elischer In-reply-to: Your message of "Thu, 18 Oct 2007 17:25:39 PDT." <4717F983.5060707@elischer.org> Date: Thu, 18 Oct 2007 23:07:00 -0700 From: Bakul Shah Message-Id: <20071019060700.DC4515B50@mail.bitblocks.com> Cc: qpadla@gmail.com, freebsd-arch@freebsd.org Subject: Re: jail/vimage level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 06:39:21 -0000 > Here are some possibilities for things.. > > most of these are somewhat supported in one way or another somewhere. > > be able to specify a different root for the virtual machine. > be able to specify a different output for "uname". > be able to specify a different network address. > be able to specify a different routing table. (+) > be able to specify a completely different network universe (*) > be able to dedicate an interface to it > be able to have a separate PID space for it. > be able to specify a separate UID space for it > be able to specify a CPU maximum quota > be able to confine it to some set of CPUs > be able to have different mount tables for it > be able to specify a different security level for it. > > (+) without necessarily having a completely different universe. (see below). > > (*) e.g. being able to have two virtual have the machine get confused. > > A) this is not an exhaustive list > B) this list also may contain items we don't implement. Sounds like you are talking about something like rfork() except that instead of process resources you want to manipulate machine resources. > A second question is "what sort of interface would make sense for this?" > something like the "jail" program possibly. How about something like machine_rfork(int n, Resource* resources) where a Resource is one of cpu cpu quota device network interface disk pid space uid space root route table syscalls phys. memory quota ... The child starts with the resources specified. The new "machine" would be minimally initialized and the child would have to act like init(1) and do further initialization to bring it up all the way. A resource may be exclusive (the parent loses it) or shared (with some sharing policy) or newly created (where that makes sense). Presumably there would be a way to enumerate all the resources that can be used in machine_rfork(). If you allow private virtual net interfaces you may also wish allow a way to create virtual networks. So for example mach0 can create mach1, mach2 and mach3 and have them be on the same virtual ethernet. Mach{1,2,3} have no way of knowing they are not connected to a real network (except indirectly via performance measurements). > However how about the possibility of a way of "pre-packaging" a machine? > or snapshotting a setup? Ideally snapshotting is done in a way that allows migrating a virtual machine to a different physical machine ("hibernate" to disk, and move state to a different machine and "wake up" from disk). Prepackaging can be done all (or almost all) in a user process. One easy thing would be to extend mtree to fetch files from somewhere to populate or update a new machine. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 09:33:59 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4FBE16A420 for ; Fri, 19 Oct 2007 09:33:59 +0000 (UTC) (envelope-from zec@icir.org) Received: from xaqua.tel.fer.hr (xaqua.tel.fer.hr [161.53.19.25]) by mx1.freebsd.org (Postfix) with ESMTP id 51EFB13C43E for ; Fri, 19 Oct 2007 09:33:59 +0000 (UTC) (envelope-from zec@icir.org) Received: by xaqua.tel.fer.hr (Postfix, from userid 20006) id D82EB9B64D; Fri, 19 Oct 2007 11:33:57 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on xaqua.tel.fer.hr X-Spam-Level: X-Spam-Status: No, score=-4.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.7 Received: from [192.168.200.100] (zec2.tel.fer.hr [161.53.19.79]) by xaqua.tel.fer.hr (Postfix) with ESMTP id AA95F9B648 for ; Fri, 19 Oct 2007 11:33:52 +0200 (CEST) From: Marko Zec To: freebsd-arch@freebsd.org Date: Fri, 19 Oct 2007 11:33:48 +0200 User-Agent: KMail/1.9.7 References: <20071019060700.DC4515B50@mail.bitblocks.com> In-Reply-To: <20071019060700.DC4515B50@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200710191133.49309.zec@icir.org> Subject: Re: jail/vimage level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 09:33:59 -0000 On Friday 19 October 2007 08:07:00 Bakul Shah wrote: > > Here are some possibilities for things.. > > > > most of these are somewhat supported in one way or another > > somewhere. > > > > be able to specify a different root for the virtual machine. > > be able to specify a different output for "uname". > > be able to specify a different network address. > > be able to specify a different routing table. (+) > > be able to specify a completely different network universe (*) > > be able to dedicate an interface to it > > be able to have a separate PID space for it. > > be able to specify a separate UID space for it > > be able to specify a CPU maximum quota > > be able to confine it to some set of CPUs > > be able to have different mount tables for it > > be able to specify a different security level for it. > > > > (+) without necessarily having a completely different universe. > > (see below). > > > > (*) e.g. being able to have two virtual have the machine get > > confused. > > > > A) this is not an exhaustive list > > B) this list also may contain items we don't implement. > > Sounds like you are talking about something like rfork() > except that instead of process resources you want to > manipulate machine resources. > > > A second question is "what sort of interface would make sense for > > this?" something like the "jail" program possibly. > > How about something like > > machine_rfork(int n, Resource* resources) > > where a Resource is one of > cpu > cpu quota > device > network interface > disk > pid space > uid space > root > route table > syscalls > phys. memory quota > ... Yup this sounds reasonable... > The child starts with the resources specified. The new > "machine" would be minimally initialized and the child would > have to act like init(1) and do further initialization to > bring it up all the way. A resource may be exclusive (the > parent loses it) or shared (with some sharing policy) or > newly created (where that makes sense). > > Presumably there would be a way to enumerate all the > resources that can be used in machine_rfork(). > > If you allow private virtual net interfaces you may also wish > allow a way to create virtual networks. So for example mach0 > can create mach1, mach2 and mach3 and have them be on the > same virtual ethernet. Mach{1,2,3} have no way of knowing > they are not connected to a real network (except indirectly > via performance measurements). > > > However how about the possibility of a way of "pre-packaging" a > > machine? or snapshotting a setup? > > Ideally snapshotting is done in a way that allows migrating a > virtual machine to a different physical machine ("hibernate" > to disk, and move state to a different machine and "wake up" > from disk). Migratig kernel level state + processes from one physical machine to another would be extremely difficult to accomplish IMO, though having separated kernel resources could maybe help a little bit. If I'm not mistaking DragnoFly folks have set this as their primary goal more than four years ago, don't know how far they have progressed towards live process migration... Marko > Prepackaging can be done all (or almost all) in a user > process. One easy thing would be to extend mtree to fetch > files from somewhere to populate or update a new machine. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to > "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 09:36:04 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05A4716A418; Fri, 19 Oct 2007 09:36:04 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id DFD2513C457; Fri, 19 Oct 2007 09:35:57 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A564CA.dip.t-dialin.net [84.165.100.202]) by redbull.bpaserver.net (Postfix) with ESMTP id BA85B2E1DD; Fri, 19 Oct 2007 11:35:50 +0200 (CEST) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id C56F85B480D; Fri, 19 Oct 2007 11:34:44 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.1/8.13.8/Submit) id l9J9YiNZ039105; Fri, 19 Oct 2007 11:34:44 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Fri, 19 Oct 2007 11:34:44 +0200 Message-ID: <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Fri, 19 Oct 2007 11:34:44 +0200 From: Alexander Leidinger To: John Baldwin References: <200710171245.36949.jhb@freebsd.org> <20071018133949.1430dlowvks8w4kg@webmail.leidinger.net> <200710181450.38224.jhb@freebsd.org> In-Reply-To: <200710181450.38224.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.4) / FreeBSD-7.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-13.504, required 8, BAYES_00 -15.00, MIME_QP_LONG_LINE 1.40, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 09:36:04 -0000 Quoting John Baldwin (from Thu, 18 Oct 2007 14:50:37 -0400= ): > On Thursday 18 October 2007 07:39:49 am Alexander Leidinger wrote: >> To me it looks like your proposal spans more than one of the above >> described layers in one package. It seems you describe what I call >> single-system sensors framework above. It looks like you want to have >> this with parts of it in the kernel. I don't think this is a good idea >> as I don't think userland data should be feed into the kernel. Could >> you please describe where you see benefits of your architecture >> compared to the description I provided above? > > Nowhere do I suggest to feed userland data into the kernel just so it can = be > reexported to userland. Instead, I think the "public" interface that syst= at, > monitoring daemons, SNMP, etc. should be a userland interface that can hav= e > multiple backends. It can pull data from a sensor implemented in userland= or > a sensor implemented in the kernel. I was thinking you talk about the interface between the kernel and the =20 userland. Now I think that you talk more or less about something which =20 could be implemented e.g., as an userland library which not only polls =20 the kernel sensors framework, but provides the single-system sensor =20 data (and could be a base of a singe-system sensor daemon which feeds =20 its data to a group-level sensors framework). Does this sound like =20 what you have in mind? Bye, Alexander. --=20 Practical politics consists in ignoring facts. =09=09-- Henry Adams http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 09:41:07 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B03416A417; Fri, 19 Oct 2007 09:41:07 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 1024F13C442; Fri, 19 Oct 2007 09:41:06 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id A0FE917104; Fri, 19 Oct 2007 09:41:05 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id l9J9f4rh081953; Fri, 19 Oct 2007 09:41:05 GMT (envelope-from phk@critter.freebsd.dk) To: Alexander Leidinger From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 19 Oct 2007 11:34:44 +0200." <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net> Date: Fri, 19 Oct 2007 09:41:04 +0000 Message-ID: <81952.1192786864@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 09:41:07 -0000 In message <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net>, Alexander L eidinger writes: >I was thinking you talk about the interface between the kernel and the >userland. Now I think that you talk more or less about something which >could be implemented e.g., as an userland library which not only polls >the kernel sensors framework, but provides the single-system sensor >data (and could be a base of a singe-system sensor daemon which feeds >its data to a group-level sensors framework). Does this sound like >what you have in mind? It certainly sounds more sensible. The kernel-userland interface should happen over a filedescriptor (either device or unix-domain socket) so that whatever daemon we park on the fd can just use select/poll/kqueue to wait for events. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 10:18:38 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28F4A16A419 for ; Fri, 19 Oct 2007 10:18:38 +0000 (UTC) (envelope-from qpadla@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.173]) by mx1.freebsd.org (Postfix) with ESMTP id AB91613C468 for ; Fri, 19 Oct 2007 10:18:34 +0000 (UTC) (envelope-from qpadla@gmail.com) Received: by ug-out-1314.google.com with SMTP id y2so726749uge for ; Fri, 19 Oct 2007 03:18:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:from:reply-to:to:subject:date:user-agent:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:message-id; bh=dG/6fGHtDQ04Ib9yhaPtCOnbjxEFIx03tLznOx0nRdM=; b=cMVbJMDB+d3+SSomKhlfQz1rEVhnsSJQQm+QgR7+f9VidQ/gCHwKqDDjk8WkFfFR8iCXs3XGiGBD21sdqlQkck9S9FwRTa5E1CYVd0XwRuheN/W62HFmxZ8askqZ5Ad1oJzxMjyjN8+H8nwE0/eW1LPYJQ2UyikccLIj/AokyEg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:from:reply-to:to:subject:date:user-agent:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:message-id; b=nw0tsAKgZNe6MQyhFq116jLPN0QsWzRGaJj88Q7YO+XZZBDQw8lLmtFsU70TF7vdcVx5Xip+PFIs8/4LG0pieXN1vT/iNBNQuwpHfRQirZIpZS7nIeCnsPsES/+zR2MQuP4XUE9nL+pgOWkO8P3XASgA1pcJ6xScfQWMF8h2aaQ= Received: by 10.66.219.16 with SMTP id r16mr2904366ugg.1192789113777; Fri, 19 Oct 2007 03:18:33 -0700 (PDT) Received: from orion ( [89.162.141.1]) by mx.google.com with ESMTPS id i39sm7921820ugd.2007.10.19.03.18.30 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 19 Oct 2007 03:18:31 -0700 (PDT) From: Nikolay Pavlov To: Julian Elischer Date: Fri, 19 Oct 2007 13:18:20 +0300 User-Agent: KMail/1.9.6 (enterprise 0.20070907.709405) References: <470E5BFB.4050903@elischer.org> <200710181359.46720.qpadla@gmail.com> <4717F983.5060707@elischer.org> In-Reply-To: <4717F983.5060707@elischer.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1641349.6TQGjD9Mua"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200710191318.24717.qpadla@gmail.com> Cc: freebsd-arch@freebsd.org Subject: Re: jail/vimage level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: qpadla@gmail.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 10:18:38 -0000 --nextPart1641349.6TQGjD9Mua Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline =D0=92 =D1=81=D0=BE=D0=BE=D0=B1=D1=89=D0=B5=D0=BD=D0=B8=D0=B8 =D0=BE=D1=82 = =46riday 19 October 2007 03:25:39 Julian Elischer =D0=BD=D0=B0=D0=BF=D0=B8= =D1=81=D0=B0=D0=BB(=D0=B0): > be able to specify a different root for the virtual machine. > =C2=A0 be able to specify a different output for "uname". > =C2=A0 be able to specify a different network address. > =C2=A0 be able to specify a different routing table. (+) > =C2=A0 be able to specify a completely different network universe (*) > =C2=A0 be able to dedicate an interface to it > =C2=A0 be able to have a separate PID space for it. > =C2=A0 be able to specify a separate UID space for it > =C2=A0 be able to specify a CPU maximum quota > =C2=A0 be able to confine it to some set of CPUs > =C2=A0 be able to have different mount tables for it > =C2=A0 be able to specify a different security level for it. be able to easily clone the virtual environment be able to run virtual environment that was buildup on i386 on amd64 boxes be able to upgrade virtual framework (and OS) without the need to recompile= =20 something in virtual environment itself =20 =20 =2D-=20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 =2D Best regards, Nikolay Pavlov. <<<----------------------------------- = =20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 --nextPart1641349.6TQGjD9Mua Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQBHGIRw/2R6KvEYGaIRAvSKAJ4izJM5beOubT/JoY1TqesnLiH8bQCg33j9 2sUvg4Tz9GRKKJ721ofqiXU= =mchJ -----END PGP SIGNATURE----- --nextPart1641349.6TQGjD9Mua-- From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 11:49:54 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B90CC16A41A; Fri, 19 Oct 2007 11:49:54 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 6EF0E13C45D; Fri, 19 Oct 2007 11:49:54 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A564CA.dip.t-dialin.net [84.165.100.202]) by redbull.bpaserver.net (Postfix) with ESMTP id 551BC2E2F1; Fri, 19 Oct 2007 13:49:48 +0200 (CEST) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 607C55B480D; Fri, 19 Oct 2007 13:48:42 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.1/8.13.8/Submit) id l9JBmgDv061028; Fri, 19 Oct 2007 13:48:42 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Fri, 19 Oct 2007 13:48:42 +0200 Message-ID: <20071019134842.rhlnbcqrbc4sc4o4@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Fri, 19 Oct 2007 13:48:42 +0200 From: Alexander Leidinger To: Poul-Henning Kamp References: <81952.1192786864@critter.freebsd.dk> In-Reply-To: <81952.1192786864@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.4) / FreeBSD-7.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-13.4, required 8, BAYES_00 -15.00, IMPRONONCABLE_2 1.50, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 11:49:54 -0000 Quoting Poul-Henning Kamp (from Fri, 19 Oct 2007 =20 09:41:04 +0000): > In message <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net>, =20 > Alexander L > eidinger writes: > >> I was thinking you talk about the interface between the kernel and the >> userland. Now I think that you talk more or less about something which >> could be implemented e.g., as an userland library which not only polls >> the kernel sensors framework, but provides the single-system sensor >> data (and could be a base of a singe-system sensor daemon which feeds >> its data to a group-level sensors framework). Does this sound like >> what you have in mind? > > It certainly sounds more sensible. More sensible than what? > The kernel-userland interface should happen over a filedescriptor > (either device or unix-domain socket) so that whatever daemon we > park on the fd can just use select/poll/kqueue to wait for events. Please explain a little bit more of this architecture regarding the =20 following questions: What to do with sensors which aren't event based or don't have a =20 predefined polling interval (e.g., temperature and humidity)? What do =20 you think will the ratio be between the amount of sensors with and =20 without something like this? How is the kernel supposed to know what polling policy the user is =20 interested in (every 5sec/every minute/every 5 minutes/whatever)? Why =20 should this policy/procedure life in the kernel? Bye, Alexander. --=20 Two heads are better than one. =09=09-- John Heywood http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 12:34:52 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7291416A417; Fri, 19 Oct 2007 12:34:52 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 3C40513C45A; Fri, 19 Oct 2007 12:34:52 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 8651817104; Fri, 19 Oct 2007 12:34:50 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id l9JCYnFh082534; Fri, 19 Oct 2007 12:34:49 GMT (envelope-from phk@critter.freebsd.dk) To: Alexander Leidinger From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 19 Oct 2007 13:48:42 +0200." <20071019134842.rhlnbcqrbc4sc4o4@webmail.leidinger.net> Date: Fri, 19 Oct 2007 12:34:49 +0000 Message-ID: <82533.1192797289@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 12:34:52 -0000 In message <20071019134842.rhlnbcqrbc4sc4o4@webmail.leidinger.net>, Alexander L eidinger writes: >>> I was thinking you talk about the interface between the kernel and the >>> userland. Now I think that you talk more or less about something which >>> could be implemented e.g., as an userland library which not only polls >>> the kernel sensors framework, but provides the single-system sensor >>> data (and could be a base of a singe-system sensor daemon which feeds >>> its data to a group-level sensors framework). Does this sound like >>> what you have in mind? >> >> It certainly sounds more sensible. > >More sensible than what? Than the OpenBSD sensors concept >What to do with sensors which aren't event based or don't have a >predefined polling interval (e.g., temperature and humidity)? What do >you think will the ratio be between the amount of sensors with and >without something like this? They poll at whatever rate the application ask them to, (using an ioctl ?) >How is the kernel supposed to know what polling policy the user is >interested in (every 5sec/every minute/every 5 minutes/whatever)? Why >should this policy/procedure life in the kernel? Nobody said the policy should live in the kernel. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 13:16:12 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A6EB16A420; Fri, 19 Oct 2007 13:16:12 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id B6D5B13C459; Fri, 19 Oct 2007 13:16:10 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A564CA.dip.t-dialin.net [84.165.100.202]) by redbull.bpaserver.net (Postfix) with ESMTP id 862512E2F1; Fri, 19 Oct 2007 15:15:32 +0200 (CEST) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 8E3E95B480D; Fri, 19 Oct 2007 15:14:26 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.1/8.13.8/Submit) id l9JDEQ9t075371; Fri, 19 Oct 2007 15:14:26 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Fri, 19 Oct 2007 15:14:26 +0200 Message-ID: <20071019151426.ttkynf788c0g8s4k@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Fri, 19 Oct 2007 15:14:26 +0200 From: Alexander Leidinger To: Poul-Henning Kamp References: <82533.1192797289@critter.freebsd.dk> In-Reply-To: <82533.1192797289@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.4) / FreeBSD-7.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-13.4, required 8, BAYES_00 -15.00, IMPRONONCABLE_2 1.50, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 13:16:12 -0000 Quoting Poul-Henning Kamp (from Fri, 19 Oct 2007 =20 12:34:49 +0000): > In message <20071019134842.rhlnbcqrbc4sc4o4@webmail.leidinger.net>, =20 > Alexander L > eidinger writes: > >>>> I was thinking you talk about the interface between the kernel and the >>>> userland. Now I think that you talk more or less about something which >>>> could be implemented e.g., as an userland library which not only polls >>>> the kernel sensors framework, but provides the single-system sensor >>>> data (and could be a base of a singe-system sensor daemon which feeds >>>> its data to a group-level sensors framework). Does this sound like >>>> what you have in mind? >>> >>> It certainly sounds more sensible. >> >> More sensible than what? > > Than the OpenBSD sensors concept Constantines sensors framework is an implementation of the above =20 mentioned kernel sensors framework. How can the above description be =20 more sensible, when such an architecture is part of this description? =20 And related: you stated you haven't looked at the architecture behind =20 Constantines work because you don't like the idea itself. Do you have =20 now looked at the architecture, or how are you able to compare what is =20 written here with what is in Constantines work? If you have looked at =20 Constantines work now, could you please tell us about architectural =20 flaws which makes it not usable as a kernel sensors framework as =20 described above (the part which you called more sensible)? >> What to do with sensors which aren't event based or don't have a >> predefined polling interval (e.g., temperature and humidity)? What do >> you think will the ratio be between the amount of sensors with and >> without something like this? > > They poll at whatever rate the application ask them to, (using an > ioctl ?) So you want to put the polling interval (=3D the polling policy) into =20 the kernel (with e.g, an ioctl)? What about the question about the rate between the number of sensors =20 with and without event driven notifications? >> How is the kernel supposed to know what polling policy the user is >> interested in (every 5sec/every minute/every 5 minutes/whatever)? Why >> should this policy/procedure life in the kernel? > > Nobody said the policy should live in the kernel. You wrote that an application can wait for an event with your approach =20 of using a fd. For event driven sensors I can see how this works =20 without putting the polling policy (the interval to poll) into the =20 kernel. For sensors which are not event driven, I fail to see how this =20 can be done without putting the polling policy (=3D the polling =20 interval) into the kernel. Would you please explain how an application =20 can wait for events from sensors which are not event driven without =20 putting the polling interval into the kernel? Related to this question =20 is the part about the ratio above. I would also like to know how you want to limit the rate of =20 kernel->userland crossings for even driven notifications without =20 putting the polling policy into the kernel, when the users polling =20 policy doesn't need the possible higher rate of a sensors notification =20 interval (we don't want to make the system unusable when several =20 sensors send to much events, don't we?). Bye, Alexander. --=20 Let thy maid servant be faithful, strong, and homely. =09=09-- Benjamin Franklin http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 13:20:35 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD46C16A417; Fri, 19 Oct 2007 13:20:35 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 77FF113C448; Fri, 19 Oct 2007 13:20:35 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 3077A17104; Fri, 19 Oct 2007 13:20:34 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id l9JDKXxG082693; Fri, 19 Oct 2007 13:20:33 GMT (envelope-from phk@critter.freebsd.dk) To: Alexander Leidinger From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 19 Oct 2007 15:14:26 +0200." <20071019151426.ttkynf788c0g8s4k@webmail.leidinger.net> Date: Fri, 19 Oct 2007 13:20:33 +0000 Message-ID: <82692.1192800033@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 13:20:35 -0000 In message <20071019151426.ttkynf788c0g8s4k@webmail.leidinger.net>, Alexander L eidinger writes: Alexander, I'm getting pretty tired of this game of yours. Either you're not paying attention to what I write, or you are so totally blinded by rage that you don't try to understand it. This is my last email to you on this subject. As far as I can tell, you have nothing to do with the actual code, apart from committing it without proper review and discussion, so why don't you step out of the loop, and leave Constantine, who, quite frankly, seems to have a better grasp of the subject than you, participate instead ? >>> What to do with sensors which aren't event based or don't have a >>> predefined polling interval (e.g., temperature and humidity)? What do >>> you think will the ratio be between the amount of sensors with and >>> without something like this? >> >> They poll at whatever rate the application ask them to, (using an >> ioctl ?) > >So you want to put the polling interval (=3D the polling policy) into =20 >the kernel (with e.g, an ioctl)? No, the "polling policy" does not end up in the kernel if the application calls an ioctl that says "poll every 5 seconds". Look up the meaning of the word policy if you don't belive me. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 13:46:38 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36FD216A418; Fri, 19 Oct 2007 13:46:38 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 58FDC13C459; Fri, 19 Oct 2007 13:46:21 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A564CA.dip.t-dialin.net [84.165.100.202]) by redbull.bpaserver.net (Postfix) with ESMTP id 5E73F2E060; Fri, 19 Oct 2007 15:46:16 +0200 (CEST) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 701075B480D; Fri, 19 Oct 2007 15:45:10 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.1/8.13.8/Submit) id l9JDjAxY080428; Fri, 19 Oct 2007 15:45:10 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Fri, 19 Oct 2007 15:45:10 +0200 Message-ID: <20071019154510.qo8obs838co0csgw@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Fri, 19 Oct 2007 15:45:10 +0200 From: Alexander Leidinger To: Poul-Henning Kamp References: <82692.1192800033@critter.freebsd.dk> In-Reply-To: <82692.1192800033@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.4) / FreeBSD-7.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-14.9, required 8, BAYES_00 -15.00, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 13:46:38 -0000 Quoting Poul-Henning Kamp (from Fri, 19 Oct 2007 =20 13:20:33 +0000): > In message <20071019151426.ttkynf788c0g8s4k@webmail.leidinger.net>, =20 > Alexander L > eidinger writes: > > Alexander, > > I'm getting pretty tired of this game of yours. I'm asking questions to understand what is going on. > Either you're not paying attention to what I write, or you are so > totally blinded by rage that you don't try to understand it. I'm not in rage and I was not in rage. > This is my last email to you on this subject. That's sad. > As far as I can tell, you have nothing to do with the actual code, We're talking about architecture here in this thread on arch, not =20 about any specific commits to the CVS. > apart from committing it without proper review and discussion, so I was willing to discuss this with you and others long ago, but you =20 stopped talking without any notification why. Then you stomped in =20 saying that you don't like the _idea_ of the framework without =20 pointing to technical flaws (you mentioned you didn't even looked at =20 the architecture). Now you tell me that you don't want to discuss =20 architectural things with me. Could you please tell me what I did to =20 you that you don't want to talk with me? > why don't you step out of the loop, and leave Constantine, who, > quite frankly, seems to have a better grasp of the subject than > you, participate instead ? You don't care to explain where my point of view of monitoring is wrong? >>>> What to do with sensors which aren't event based or don't have a >>>> predefined polling interval (e.g., temperature and humidity)? What do >>>> you think will the ratio be between the amount of sensors with and >>>> without something like this? >>> >>> They poll at whatever rate the application ask them to, (using an >>> ioctl ?) >> >> So you want to put the polling interval (=3D3D the polling policy) into = =3D20 >> the kernel (with e.g, an ioctl)? > > No, the "polling policy" does not end up in the kernel if the > application calls an ioctl that says "poll every 5 seconds". > > Look up the meaning of the word policy if you don't belive me. According to dict.leo.org: english german policy especially [comp.] die Richtlinie If you say "poll every 5 seconds" you give the kernel "die Richtlinie" =20 to look every 5 seconds for the sensor data and to provide it to the =20 application. Looks good to me in my language. As you don't like it: =20 feel free to replace every ocurence of "polling policy" with "polling =20 interval" in my questions. If you think those questions are not worth =20 being answered, I would like to know why. Bye, Alexander. --=20 I've got a COUSIN who works in the GARMENT DISTRICT ... http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 14:19:29 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D60A216A420 for ; Fri, 19 Oct 2007 14:19:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id CD42B13C4A3 for ; Fri, 19 Oct 2007 14:19:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from zion.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by elvis.mu.org (Postfix) with ESMTP id DE7A91A4D8D; Fri, 19 Oct 2007 06:51:03 -0700 (PDT) From: John Baldwin To: Alexander Leidinger Date: Fri, 19 Oct 2007 08:41:47 -0400 User-Agent: KMail/1.9.7 References: <200710171245.36949.jhb@freebsd.org> <200710181450.38224.jhb@freebsd.org> <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net> In-Reply-To: <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200710190841.48129.jhb@freebsd.org> Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 14:19:29 -0000 On Friday 19 October 2007 05:34:44 am Alexander Leidinger wrote: > Quoting John Baldwin (from Thu, 18 Oct 2007 14:50:37 -0400): > > > On Thursday 18 October 2007 07:39:49 am Alexander Leidinger wrote: > >> To me it looks like your proposal spans more than one of the above > >> described layers in one package. It seems you describe what I call > >> single-system sensors framework above. It looks like you want to have > >> this with parts of it in the kernel. I don't think this is a good idea > >> as I don't think userland data should be feed into the kernel. Could > >> you please describe where you see benefits of your architecture > >> compared to the description I provided above? > > > > Nowhere do I suggest to feed userland data into the kernel just so it can be > > reexported to userland. Instead, I think the "public" interface that systat, > > monitoring daemons, SNMP, etc. should be a userland interface that can have > > multiple backends. It can pull data from a sensor implemented in userland or > > a sensor implemented in the kernel. > > I was thinking you talk about the interface between the kernel and the > userland. Now I think that you talk more or less about something which > could be implemented e.g., as an userland library which not only polls > the kernel sensors framework, but provides the single-system sensor > data (and could be a base of a singe-system sensor daemon which feeds > its data to a group-level sensors framework). Does this sound like > what you have in mind? Yes. And I don't think that the kernel-userland interface for kernel-backed sensors should be a "public" interface, but a private backend that only the sensors framework uses. The "public" interface that tools and users, etc. should be using is the library. Basically, in your three levels of sensors I think the first level should be an implementation detail that is free to change if needed as it won't be a "public" API. The stable, public API would be the userland interface. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 14:24:31 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 142F316A417; Fri, 19 Oct 2007 14:24:31 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id A863913C48A; Fri, 19 Oct 2007 14:24:30 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A564CA.dip.t-dialin.net [84.165.100.202]) by redbull.bpaserver.net (Postfix) with ESMTP id EA4CD2E325; Fri, 19 Oct 2007 16:24:05 +0200 (CEST) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 025A45B480D; Fri, 19 Oct 2007 16:22:59 +0200 (CEST) Received: (from www@localhost) by webmail.leidinger.net (8.14.1/8.13.8/Submit) id l9JEMxWK086710; Fri, 19 Oct 2007 16:22:59 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Fri, 19 Oct 2007 16:22:59 +0200 Message-ID: <20071019162259.7qjnk9dlyccw4oww@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Fri, 19 Oct 2007 16:22:59 +0200 From: Alexander Leidinger To: John Baldwin References: <200710171245.36949.jhb@freebsd.org> <200710181450.38224.jhb@freebsd.org> <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net> <200710190841.48129.jhb@freebsd.org> In-Reply-To: <200710190841.48129.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.4) / FreeBSD-7.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-13.427, required 8, BAYES_00 -15.00, MIME_QP_LONG_LINE 1.40, RDNS_DYNAMIC 0.10, TW_BK 0.08) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 14:24:31 -0000 Quoting John Baldwin (from Fri, 19 Oct 2007 08:41:47 -0400= ): > On Friday 19 October 2007 05:34:44 am Alexander Leidinger wrote: >> Quoting John Baldwin (from Thu, 18 Oct 2007 =20 >> 14:50:37 -0400): >> >> > On Thursday 18 October 2007 07:39:49 am Alexander Leidinger wrote: >> >> To me it looks like your proposal spans more than one of the above >> >> described layers in one package. It seems you describe what I call >> >> single-system sensors framework above. It looks like you want to have >> >> this with parts of it in the kernel. I don't think this is a good idea >> >> as I don't think userland data should be feed into the kernel. Could >> >> you please describe where you see benefits of your architecture >> >> compared to the description I provided above? >> > >> > Nowhere do I suggest to feed userland data into the kernel just =20 >> so it can be >> > reexported to userland. Instead, I think the "public" interface =20 >> that systat, >> > monitoring daemons, SNMP, etc. should be a userland interface =20 >> that can have >> > multiple backends. It can pull data from a sensor implemented in =20 >> userland or >> > a sensor implemented in the kernel. >> >> I was thinking you talk about the interface between the kernel and the >> userland. Now I think that you talk more or less about something which >> could be implemented e.g., as an userland library which not only polls >> the kernel sensors framework, but provides the single-system sensor >> data (and could be a base of a singe-system sensor daemon which feeds >> its data to a group-level sensors framework). Does this sound like >> what you have in mind? > > Yes. And I don't think that the kernel-userland interface for kernel-back= ed > sensors should be a "public" interface, but a private backend that only th= e > sensors framework uses. The "public" interface that tools and users, etc. > should be using is the library. Basically, in your three levels of sensor= s Like libkvm does for the abstraction of the underlying implementation =20 (ok, libkvm does not only abstract the interface to the kernel, but I =20 don't think this detail is important ATM)? Good idea IMO. > I think the first level should be an implementation detail that is free to > change if needed as it won't be a "public" API. The stable, public API wo= uld > be the userland interface. Would you mind using "official"/"internal" instead of =20 "public"/"private"? I think this is a better description, as in an =20 open source project a lot is public, even if it is an internal =20 interface. This would make the discussion more unambiguous IMO. Regarding making the kernel sensors framework an implementation detail =20 that is free to change, I agree with you too, this makes it more =20 future proof and allows to survive evolutionary improvements (if they =20 are needed) without the headache of the need to keep the =20 kernel-userland interface compatible. Bye, Alexander. --=20 The following statement is not true. The previous statement is true. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 16:07:15 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DACB16A41B; Fri, 19 Oct 2007 16:07:15 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 8028413C458; Fri, 19 Oct 2007 16:07:14 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.1/8.14.1) with ESMTP id l9JG5GiE029284; Fri, 19 Oct 2007 10:05:16 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Fri, 19 Oct 2007 10:05:16 -0600 (MDT) Message-Id: <20071019.100516.74722974.imp@bsdimp.com> To: phk@phk.freebsd.dk From: Warner Losh In-Reply-To: <81952.1192786864@critter.freebsd.dk> References: <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net> <81952.1192786864@critter.freebsd.dk> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Alexander@Leidinger.net, cnst@freebsd.org, arch@freebsd.org Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 16:07:15 -0000 From: "Poul-Henning Kamp" Subject: Re: sensors fun.. Date: Fri, 19 Oct 2007 09:41:04 +0000 > In message <20071019113444.xinyc37x9cg0ckk0@webmail.leidinger.net>, Alexander L > eidinger writes: > > >I was thinking you talk about the interface between the kernel and the > >userland. Now I think that you talk more or less about something which > >could be implemented e.g., as an userland library which not only polls > >the kernel sensors framework, but provides the single-system sensor > >data (and could be a base of a singe-system sensor daemon which feeds > >its data to a group-level sensors framework). Does this sound like > >what you have in mind? > > It certainly sounds more sensible. > > The kernel-userland interface should happen over a filedescriptor > (either device or unix-domain socket) so that whatever daemon we > park on the fd can just use select/poll/kqueue to wait for events. If we're going to have a stream of data from the kernel, is there any reason to invent another daemon for that? We already have devd that deals with a number of disparate events from the kernel in a fairly generic way. Warner From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 16:12:46 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D900116A417; Fri, 19 Oct 2007 16:12:46 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 994E013C4A5; Fri, 19 Oct 2007 16:12:39 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 1A08B17104; Fri, 19 Oct 2007 16:12:37 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.1/8.14.1) with ESMTP id l9JGCaww083491; Fri, 19 Oct 2007 16:12:37 GMT (envelope-from phk@critter.freebsd.dk) To: Warner Losh From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 19 Oct 2007 10:05:16 CST." <20071019.100516.74722974.imp@bsdimp.com> Date: Fri, 19 Oct 2007 16:12:36 +0000 Message-ID: <83490.1192810356@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Alexander@Leidinger.net, cnst@freebsd.org, arch@freebsd.org Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 16:12:46 -0000 In message <20071019.100516.74722974.imp@bsdimp.com>, Warner Losh writes: >> The kernel-userland interface should happen over a filedescriptor >> (either device or unix-domain socket) so that whatever daemon we >> park on the fd can just use select/poll/kqueue to wait for events. > >If we're going to have a stream of data from the kernel, is there any >reason to invent another daemon for that? We already have devd that >deals with a number of disparate events from the kernel in a fairly >generic way. The kernel and userland sensors would result in two kinds of data, measurements ("32°C") and events ("new sensor", "high temp") devd should not see the measurements, but it might be a good idea if it could see the events from all sensors (userland AND kernel). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 16:15:48 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DFC0E16A419 for ; Fri, 19 Oct 2007 16:15:48 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id C3D3B13C47E for ; Fri, 19 Oct 2007 16:15:46 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id E8F7D5B3B; Fri, 19 Oct 2007 09:15:34 -0700 (PDT) To: Marko Zec In-reply-to: Your message of "Fri, 19 Oct 2007 11:33:48 +0200." <200710191133.49309.zec@icir.org> Date: Fri, 19 Oct 2007 09:15:34 -0700 From: Bakul Shah Message-Id: <20071019161534.E8F7D5B3B@mail.bitblocks.com> Cc: freebsd-arch@freebsd.org Subject: Re: jail/vimage level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 16:15:49 -0000 > Marko Zec > On Friday 19 October 2007 08:07:00 Bakul Shah wrote: > > Ideally snapshotting is done in a way that allows migrating a > > virtual machine to a different physical machine ("hibernate" > > to disk, and move state to a different machine and "wake up" > > from disk). > > Migratig kernel level state + processes from one physical machine to > another would be extremely difficult to accomplish IMO, though having > separated kernel resources could maybe help a little bit. If I'm not > mistaking DragnoFly folks have set this as their primary goal more than > four years ago, don't know how far they have progressed towards live process migration... Note that I am not talking about process migration but virtual machine migration. Moving processes is far more difficult since things are coupled too tightly in Unix. Sharing the same file descriptor or mmap pages across diff. machines is not even desirable. My thought was that if access to other machines (virtual or real) is via network interfaces only, it would be relatively easy. This would be like e.g. suspending your laptop at work and and resuming at home. You lose things tightly coupled to one environment when you move to another (such as open connections to machines not accessible from the new environment). Separating kernel resources is just one step. Even for snapshotting on the same machine you'd need looser coupling between components. If you can accomplish that then migrating may not be that much more difficult. With VM migration you can do a lot of interesting things. - You can for instance interactively set up a machine "just right" for a certain type of user and resume it on N different machines, one per user. Now each of these machines can evolve differently (I think this is what Julian was referring to by "pre-packaging a machine"). - A new OS release can be just some machine's suspended state + may be a script. New users can start using the machine almost instantly. - Obvious things such as replacing physical machines for faster ones without "taking down" live VMs. Of course, none of this is new. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 16:40:52 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA6A716A417 for ; Fri, 19 Oct 2007 16:40:52 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from gritton.org (gritton.org [161.58.222.4]) by mx1.freebsd.org (Postfix) with ESMTP id 5160F13C44B for ; Fri, 19 Oct 2007 16:40:52 +0000 (UTC) (envelope-from jamie@gritton.org) Received: from [10.20.12.66] (fw.oremut02.us.wh.verio.net [198.65.168.24]) (authenticated bits=0) by gritton.org (8.13.6.20060614/8.13.6) with ESMTP id l9JGLwUl061782; Fri, 19 Oct 2007 10:21:58 -0600 (MDT) Message-ID: <4718D9A1.8040200@gritton.org> Date: Fri, 19 Oct 2007 10:21:53 -0600 From: James Gritton User-Agent: Thunderbird 1.5.0.2 (X11/20060512) MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <470E5BFB.4050903@elischer.org> <200710172216.l9HMGhbd067251@apollo.backplane.com> <200710181359.46720.qpadla@gmail.com> <4717F983.5060707@elischer.org> In-Reply-To: <4717F983.5060707@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Julian Elischer Subject: Re: jail/vimage level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 16:40:52 -0000 Julian Elischer wrote: > I don't want to discuss virtualisation that duplicates the entire kernel, > other than the single question "Should we drop support for jails and > vimage > style virtualisation in favour of "Userland linux/dragonfly/freeBSD" > or Xen or {your favourite virtualmachine}. That's an easy one - a big "no". Broader virtualization technologies are fine, but they serve a different need and shouldn't take over what jails (especially with certain extensions) offer. Namely: ... > IF we decide to keep teh jail/super-chroot/vimage support, then > what do we want to see out of it? Light weight. The main thing I want to see is being able to partition the jail into a separate virtual environment without bloating the kernel, or adding new kernels. To be able to offer a complete FreeBSD userspace environment largely indistinguishable from the unjailed, with the benefit of keeping different jails from interfering with other as much as possible. I'm not interested (for the purposes of this topic) in acting like another OS, or acting like you're on some kind of different hardware. Basically, I'm looking for everything Matt Dillon said we shouldn't bother to do at this level. - Jamie From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 17:12:15 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7592516A418 for ; Fri, 19 Oct 2007 17:12:15 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outC.internet-mail-service.net (outC.internet-mail-service.net [216.240.47.226]) by mx1.freebsd.org (Postfix) with ESMTP id 52F6213C458 for ; Fri, 19 Oct 2007 17:12:14 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.40) with ESMTP; Fri, 19 Oct 2007 10:12:14 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id 0561F126787; Fri, 19 Oct 2007 10:12:13 -0700 (PDT) Message-ID: <4718E583.8010501@elischer.org> Date: Fri, 19 Oct 2007 10:12:35 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Bakul Shah References: <20071019060700.DC4515B50@mail.bitblocks.com> In-Reply-To: <20071019060700.DC4515B50@mail.bitblocks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: qpadla@gmail.com, freebsd-arch@freebsd.org Subject: Re: jail/vimage level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 17:12:15 -0000 Bakul Shah wrote: >> Here are some possibilities for things.. >> >> most of these are somewhat supported in one way or another somewhere. >> >> be able to specify a different root for the virtual machine. >> be able to specify a different output for "uname". >> be able to specify a different network address. >> be able to specify a different routing table. (+) >> be able to specify a completely different network universe (*) >> be able to dedicate an interface to it >> be able to have a separate PID space for it. >> be able to specify a separate UID space for it >> be able to specify a CPU maximum quota >> be able to confine it to some set of CPUs >> be able to have different mount tables for it >> be able to specify a different security level for it. >> >> (+) without necessarily having a completely different universe. (see below). >> >> (*) e.g. being able to have two virtual have the machine get confused. >> >> A) this is not an exhaustive list >> B) this list also may contain items we don't implement. > > Sounds like you are talking about something like rfork() > except that instead of process resources you want to > manipulate machine resources. > >> A second question is "what sort of interface would make sense for this?" >> something like the "jail" program possibly. > > How about something like > > machine_rfork(int n, Resource* resources) > > where a Resource is one of > cpu > cpu quota > device > network interface > disk > pid space > uid space > root > route table > syscalls > phys. memory quota > ... > > The child starts with the resources specified. The new > "machine" would be minimally initialized and the child would > have to act like init(1) and do further initialization to > bring it up all the way. A resource may be exclusive (the > parent loses it) or shared (with some sharing policy) or > newly created (where that makes sense). > > Presumably there would be a way to enumerate all the > resources that can be used in machine_rfork(). > > If you allow private virtual net interfaces you may also wish > allow a way to create virtual networks. So for example mach0 > can create mach1, mach2 and mach3 and have them be on the > same virtual ethernet. Mach{1,2,3} have no way of knowing > they are not connected to a real network (except indirectly > via performance measurements). As a general comment People interested in this thread should look at the 'vimage' work. It does much of this already. http://imunes.tel.fer.hr/virtnet/ > >> However how about the possibility of a way of "pre-packaging" a machine? >> or snapshotting a setup? > > Ideally snapshotting is done in a way that allows migrating a > virtual machine to a different physical machine ("hibernate" > to disk, and move state to a different machine and "wake up" > from disk). > > Prepackaging can be done all (or almost all) in a user > process. One easy thing would be to extend mtree to fetch > files from somewhere to populate or update a new machine. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 17:33:18 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EE6816A420; Fri, 19 Oct 2007 17:33:18 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.freebsd.org (Postfix) with ESMTP id 3F50813C467; Fri, 19 Oct 2007 17:33:11 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (fuzbdlct28f3gfzz@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id l9JHXA8S070877; Fri, 19 Oct 2007 10:33:10 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id l9JHXAeo070876; Fri, 19 Oct 2007 10:33:10 -0700 (PDT) (envelope-from jmg) Date: Fri, 19 Oct 2007 10:33:10 -0700 From: John-Mark Gurney To: Alexander Leidinger Message-ID: <20071019173310.GC39759@funkthat.com> Mail-Followup-To: Alexander Leidinger , arch@FreeBSD.org References: <20071015152408.10kvgtog6cooc4wc@webmail.leidinger.net> <48258.1192460507@critter.freebsd.dk> <20071015210909.1b6b693b@deskjail> <20071015202115.GP39759@funkthat.com> <20071016184047.8zh3fl90gksks40c@webmail.leidinger.net> <20071016175726.GS39759@funkthat.com> <20071019133500.epngxkctq88c8s00@webmail.leidinger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071019133500.epngxkctq88c8s00@webmail.leidinger.net> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (hydrogen.funkthat.com [127.0.0.1]); Fri, 19 Oct 2007 10:33:10 -0700 (PDT) Cc: arch@freebsd.org Subject: Re: cvs commit: src/etc Makefile sensorsd.conf src/etc/defaults rc.conf src/etc/rc.d Makefile sensorsd src/lib/libc/gen sysctl.3 src/sbin/sysctl sysctl.8 sysctl.c src/share/man/man5 rc.conf.5 src/share/man/man9 Makefile sensor_attach.9 src/sys/conf files ... X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 17:33:18 -0000 Alexander Leidinger wrote this message on Fri, Oct 19, 2007 at 13:35 +0200: > Quoting John-Mark Gurney (from Tue, 16 > Oct 2007 10:57:26 -0700): > > >Alexander Leidinger wrote this message on Tue, Oct 16, 2007 at 18:40 +0200: > >>Quoting John-Mark Gurney (from Mon, 15 > >>Oct 2007 13:21:15 -0700): > >> > >>>Alexander Leidinger wrote this message on Mon, Oct 15, 2007 at 21:09 > >>+0200: > >>>>> >I already told you last time > >>>>> >that the current way (access to the i2c or smbus) needs more access > >>>>> >rights than using the userland parts of the sensors framework. > >>>>> > >>>>> More rights than what exactly ? > >>>> > >>>>One popular userland temperature/voltage reading tool (as it supports a > >>>>lot of popular devices) is mbmon. It is currently a SUID root > >>>>application. It is like this as it accesses the smbus and/or ISA I/O > >>>>ports directly. If we forget the ISA I/O ports part, we could maybe > >>>>switch to a mbmon-user, but I don't really want to have such an user be > >>>>able to query every device on the smbus. > >>>> > >>>>systat and sysctl are not SUID/SGID and don't require some special > >>>>rights in /dev. I would say this is a big difference in favour of the > >>>>sensors framework. > >>> > >>>Did you completely ignore the discussion back in July? I didn't bring > >>>it up, because someone else did, but the simple solution is a socket > >> > >>Have you a pointer to it? I would like to analyze why I don't remember > >>to have seen this. > > > >Hmm.. Looks like no one specificly stated a socket, plenty people > >talked about userland, and a couple mentions of daemons.. You were so > >against doing any userland work that the discussion never got far > >enough to talk about design decisions and implementation... > > I think we maybe haven't talked about the same things... in > Constantines sensors framework (the code which is responsible to fill > hw.sensors with data) I still think there's no room for userland data. > See my description of different sensor frameworks on arch from > yesterday). As long as you still think there is no room for userland bits, I doubt you'll ever get buy off on the group raising the issues... I admit I haven't done as much kernel work as others like phk and Warner, but I have learned that the kernel should never be a dumping ground, "just because it's easier"TM... > This is decoupled from e.g., lm.4. So far we have no safe way of > querying such devices from userland. lm.4 gives us the safe way. It's And why can't the lm.4 be designed to export it's data to a userland daemon instead of to the sensore framework? > lm.4 is used to access common hardware on mainboards. Currently you > need elevated privileges without any protection layer between the code > doing the data reading and the user requesting it. With lm.4 and the This is obviously where you don't get where phk and I talk about using the kernel as a transport layer... You write enough of the driver in kernel to talk w/ the hardware in a non-destructive way, and let the userland part do the rest of the work... > sensors framework we have our sysctl framework as a protection layer > in between. I don't say everything needs to be in the kernel, but the > current situation is not good, and this we get for free. If there's You have an interesting definition of free... No code is free... Are you going to guarante that you will promptly fix all bugs w/ the code? > some kind of framework, where the lm.4 stuff can be added to the > userland to get similar benefits we get with lm.4, I surely don't > object to move it there (systems evolve, and I welcome this). But Umm... just above you just said that you don't believe in userland data, so how do you reconcile this statement? Here you are saying you do support it, or am I missing a qualification to your statements? > Constantines project was not about such a framework, and nobody is > working on something like this. So it's not that I object in general, > it's that we don't have the infrastructure to move in this direction. And it's a good step to take, and I'm sure Constantine has learned a lot about FreeBSD doing the project, and that's great. But a SoC project does not get an immediate green light w/o review from the FreeBSD community... > >Rober Watson talking about using the SNMP daemon: > >http://docs.freebsd.org/cgi/getmsg.cgi?fetch=174429+0+/usr/local/www/db/text/2007/freebsd-arch/20070715.freebsd-arch > > Robert is talking about what fits into single-system sensors framework > on arch. > > >phk talking about userland for MIB and kernel for transport: > >http://docs.freebsd.org/cgi/getmsg.cgi?fetch=170761+0+/usr/local/www/db/text/2007/freebsd-arch/20070715.freebsd-arch > > Poul was arguing that there's no benefit from Constantines sensors > framework and wanted to do something what I call a single-system > sensors framework in the mail to arch from yesterday (he didn't care > to look even at the architecture of Constantines work). Nobody is > saying that a single-system sensor framework is not a good idea (if it > belongs to ports or into our basesystem e.g., via using our bsnmp or > by writting something else, is not something I care much about), but > Poul said Constantines work (the idea behind, to export sensor data > via a common API from the kernel, so that users of FreeBSD don't have > to go and hunt down sensor data from the kernel from a lot of places > and that developers don't have to reinvent the wheel how to export the > data from the kernel) is not worth our time. > > >Doug Ambrisko talking about doing it both as a userland and a kernel > >interface: > >http://docs.freebsd.org/cgi/getmsg.cgi?fetch=53014+0+/usr/local/www/db/text/2007/freebsd-arch/20070715.freebsd-arch > > Doug suggested to do a combination of what I describe in my mail to > arch from yesterday as a kernel sensors framework and a single-system > sensors framework. Constantines work was about the part of a kernel > sensors framework. He also provided some userland code so that data > from the kernel sensor framework can be logged into syslog, and that > users can see this data in a top-like fashion. Doug was not saying > something against the kernel sensors framework. And I think he said > what he said based upon what he has read in Pouls mails. > > >phk talking about why kernel is complicated and userland is better: > >http://docs.freebsd.org/cgi/getmsg.cgi?fetch=166332+0+/usr/local/www/db/text/2007/freebsd-arch/20070715.freebsd-arch Looks like that was the link to the message he replied to.. The correct link is: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=170761+0+/usr/local/www/db/text/2007/freebsd-arch/20070715.freebsd-arch > Somehow I get a mail from Robert when I follow this link. Regarding > what you wrote as the explanation for the link: Poul objected to the > entire kernel work Constantine was doing, without looking at the > architecture or the code. I was explaining that Constantines work was > about providing a way of exporting sensor data from the kernel to the > userland and that nobody prevents us to run some sensor data collector > program (what I better describe as a single-system sensor framework in > my mail to arch from yesterday) in userland. Poul was focused on his Yet, the code can/will prevent a proper userland implementation from being developed, and additional drivers that don't belong in the kernel to be written because they don't have the single-system sensor framework.. The only way to easily get systat to display their sensor's data is to use the kernel framework, and so suddenly we have more code in the kernel than we should of if we had properly done the job in the first place... > userland approach without any common API to access kernel stuff > (Constantines work). He also didn't continue the discussion > > >>>like /var/run/log or /var/run/devd.pipe, that a userland daemon running > >>>as root that has access to ISA I/O and related resources... It's > >>>that simple... > >> > >>And the code doesn't exists. And when it is written, when will it be > >>bugfree enough? The sysctl way of exporting integer data already has a > > > >And code in the kernel is guaranteed to be bug free enough? I'd much > >rather have "bug free" code in the userland where the stability of the > >system isn't as greatly effected by the code than "bug free" code in > >the kernel... > > It's easier to write safe code to do the lm stuff (this stuff is less > complex) in the kernel, than it is to write a safe infrastructure > which "only" gives access to ISA I/O (you can do a lot with ISA I/O > access, it needs to be in a way that resources there which are bound > by drivers in the kernel don't get circumvented), as the complexity is > different. I'll admit I don't know how LM78 and friends are implemented, so I can't say for sure what the design of a "transport" kernel driver would look like for the sensors, or even if it would make sense... But we aren't talking about a specific hardware implementation... You keep bringing up lm, but what about i2c sensors that are on small embedded systems and the like? My TS-7200 has a temp sensors on the i2c bus.. > >Also, I find writing bug free code much easier when done in userland > >as it's easier to go through the debug/recompile/test steps than the > >same for kernel code... > > There are critical bugs, and non-critical bugs. If you give userland > access to ISA resources on a "common use" instead of a "rare > circumstances" base (which would be the case in an interval based > monitoring) in our current infrastrucuture, I'm not convinced, that is > is that much easier. If we talk about a hypotetical > ISA-access-infrastructure, which prevents the use of malicious things > with kernel-attached devices, then the situation looks different. Ok, I never _EVER_ suggested giving a userland program ISA IO access... That's the point of the transport kernel layer driver to do the ISA IO access on behalf of the requesting userland daemon... > >>good track record, and porting the existing lm sensor (from a project > >>which is known to take much care about security) was easier to get > >>right. The project also was not about the lm sensor (I don't go and > >>count the size for the small lm sensor now). The lm sensor was one > >>example of using it. I don't think objection to the lm sensor driver > >>should lead to removal of the framework itself. One possible reaction > >>could be to say that the lm sensor should move to ports. > > > >I've never talked about a specific sensor or anything...... I purely > >argued for the agregation of data to happen in userland and the kernel > >simply be a transport for the data to userland.... > > And Constantines sensors framework is supposed to give a common API to > transport the data to userland. But it also prevents userland sensors from being included in the systat agreegation of sensors... > >Maybe you should look at NUT: http://www.networkupstools.org/ > > > >It does a bit more than simply look at sensor readings, but it does > >do things like report voltage, temperature, and current battery charge... > > You are talking about what I call a group-level sensors framework. See > my mail from yesterday to arch how this fits into my current view of > the world with Constantines sensors framework. No, I am talking about your system-level sensor idea... NUT was partly a bad example since it's inter-machine, but anything that is inter-machine can easily be scaled to intra-machine... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 17:56:03 2007 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E332116A420; Fri, 19 Oct 2007 17:56:03 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.freebsd.org (Postfix) with ESMTP id A442213C474; Fri, 19 Oct 2007 17:56:03 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (y8p6bgi39su89n5a@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id l9JHu3GP071276; Fri, 19 Oct 2007 10:56:03 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id l9JHu27u071275; Fri, 19 Oct 2007 10:56:02 -0700 (PDT) (envelope-from jmg) Date: Fri, 19 Oct 2007 10:56:02 -0700 From: John-Mark Gurney To: Alexander Leidinger Message-ID: <20071019175602.GD39759@funkthat.com> Mail-Followup-To: Alexander Leidinger , John Baldwin , arch@freebsd.org, "Constantine A. Murenin" References: <200710171245.36949.jhb@freebsd.org> <20071018133949.1430dlowvks8w4kg@webmail.leidinger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071018133949.1430dlowvks8w4kg@webmail.leidinger.net> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (hydrogen.funkthat.com [127.0.0.1]); Fri, 19 Oct 2007 10:56:03 -0700 (PDT) Cc: arch@freebsd.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 17:56:04 -0000 Alexander Leidinger wrote this message on Thu, Oct 18, 2007 at 13:39 +0200: > One level is in the kernel. It's just an export interface to transfer > status data from the kernel to userland. The goal of this framework is > IMO to "collect" all this data in the kernel and provide a single > point of contact for the userland to query this in-kernel data. That's > what the soc project was about. Let's call this kernel sensors > framework here. The project may have been about that, but what was committed included more than that (systat and sensord)... It does look like you are finally understanding what we have been talking about.. so I won't comment on the rest of it.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 19:07:28 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B2DB16A417 for ; Fri, 19 Oct 2007 19:07:28 +0000 (UTC) (envelope-from zec@icir.org) Received: from xaqua.tel.fer.hr (xaqua.tel.fer.hr [161.53.19.25]) by mx1.freebsd.org (Postfix) with ESMTP id A4AFD13C461 for ; Fri, 19 Oct 2007 19:07:27 +0000 (UTC) (envelope-from zec@icir.org) Received: by xaqua.tel.fer.hr (Postfix, from userid 20006) id 8E5559B657; Fri, 19 Oct 2007 21:07:26 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on xaqua.tel.fer.hr X-Spam-Level: X-Spam-Status: No, score=-4.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.7 Received: from [192.168.200.103] (zec2.tel.fer.hr [161.53.19.79]) by xaqua.tel.fer.hr (Postfix) with ESMTP id 3DBEA9B652; Fri, 19 Oct 2007 21:07:24 +0200 (CEST) From: Marko Zec To: Bakul Shah Date: Fri, 19 Oct 2007 21:07:22 +0200 User-Agent: KMail/1.9.7 References: <20071019161534.E8F7D5B3B@mail.bitblocks.com> In-Reply-To: <20071019161534.E8F7D5B3B@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200710192107.22571.zec@icir.org> Cc: freebsd-arch@freebsd.org Subject: Re: jail/vimage level virtualisation requirements. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 19:07:28 -0000 On Friday 19 October 2007 18:15:34 Bakul Shah wrote: > > Marko Zec > > > > On Friday 19 October 2007 08:07:00 Bakul Shah wrote: > > > Ideally snapshotting is done in a way that allows migrating a > > > virtual machine to a different physical machine ("hibernate" > > > to disk, and move state to a different machine and "wake up" > > > from disk). > > > > Migratig kernel level state + processes from one physical machine > > to another would be extremely difficult to accomplish IMO, though > > having separated kernel resources could maybe help a little bit. > > If I'm not mistaking DragnoFly folks have set this as their primary > > goal more than four years ago, don't know how far they have > > progressed towards live > > process migration... > > Note that I am not talking about process migration but > virtual machine migration. There's no such thing as true virtual machine environment with jails or with jail-like frameworks like vimage. I don't think it's realistic to expect to see a functional live jail migration from one machine to another in the forseable future. Cheers, Marko > Moving processes is far more > difficult since things are coupled too tightly in Unix. > Sharing the same file descriptor or mmap pages across diff. > machines is not even desirable. > > My thought was that if access to other machines (virtual or > real) is via network interfaces only, it would be relatively > easy. This would be like e.g. suspending your laptop at work > and and resuming at home. You lose things tightly coupled to > one environment when you move to another (such as open > connections to machines not accessible from the new > environment). > > Separating kernel resources is just one step. Even for > snapshotting on the same machine you'd need looser coupling > between components. If you can accomplish that then > migrating may not be that much more difficult. > > With VM migration you can do a lot of interesting things. > - You can for instance interactively set up a machine "just > right" for a certain type of user and resume it on N > different machines, one per user. Now each of these > machines can evolve differently (I think this is what > Julian was referring to by "pre-packaging a machine"). > - A new OS release can be just some machine's suspended state > + may be a script. New users can start using the machine > almost instantly. > - Obvious things such as replacing physical machines for > faster ones without "taking down" live VMs. > > Of course, none of this is new. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 19 20:14:19 2007 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65FC716A419; Fri, 19 Oct 2007 20:14:19 +0000 (UTC) (envelope-from rnsanchez@wait4.org) Received: from sumo.dreamhost.com (sumo.dreamhost.com [66.33.216.29]) by mx1.freebsd.org (Postfix) with ESMTP id 92B4D13C4A7; Fri, 19 Oct 2007 20:14:16 +0000 (UTC) (envelope-from rnsanchez@wait4.org) Received: from spunkymail-a16.g.dreamhost.com (balanced.mail.policyd.dreamhost.com [208.97.132.119]) by sumo.dreamhost.com (Postfix) with ESMTP id E2A2D1F6346; Fri, 19 Oct 2007 12:42:35 -0700 (PDT) Received: from sauron.lan.box (201-34-100-43.paemt701.dsl.brasiltelecom.net.br [201.34.100.43]) by spunkymail-a16.g.dreamhost.com (Postfix) with ESMTP id 318657BA3B; Fri, 19 Oct 2007 12:42:23 -0700 (PDT) Date: Fri, 19 Oct 2007 17:42:14 -0200 From: Ricardo Nabinger Sanchez To: Robert Watson Message-Id: <20071019174214.e8672336.rnsanchez@wait4.org> In-Reply-To: <20071018120730.N60783@fledge.watson.org> References: <55408.1192704998@critter.freebsd.dk> <20071018120730.N60783@fledge.watson.org> Organization: SYS_WAIT4 X-Mailer: Sylpheed 2.4.7 (GTK+ 2.10.14; i386-unknown-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Max Laier , Poul-Henning Kamp , freebsd-arch@FreeBSD.org, Julian Elischer , "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Oct 2007 20:14:19 -0000 On Thu, 18 Oct 2007 12:17:01 +0100 (BST) Robert Watson wrote: > However, this does range a bit far afield from the thread topic. The > basic point I was making is that sysctl offers a more semantically > rich and, to be honest, better defined way of interacting with live > subsystems than device files do in a generic sense. You can hammer > down semantics for device nodes, but the code associated with sysctl > is much simpler when the goal is to offer mib-like semantics even in > quite simple cases (integer set/put). And there's even a MIB for that (perhaps someone even pointed it already): http://www.ietf.org/rfc/rfc3433.txt Regards. -- Ricardo Nabinger Sanchez rnsanchez@wait4.org Powered by FreeBSD "Left to themselves, things tend to go from bad to worse." From owner-freebsd-arch@FreeBSD.ORG Sat Oct 20 02:57:11 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D45B816A419 for ; Sat, 20 Oct 2007 02:57:11 +0000 (UTC) (envelope-from cnst@FreeBSD.org) Received: from mojo.ru (mojo.ru [84.252.152.63]) by mx1.freebsd.org (Postfix) with ESMTP id 0C6AA13C44B for ; Sat, 20 Oct 2007 02:57:10 +0000 (UTC) (envelope-from cnst@FreeBSD.org) Received: from [192.168.0.16] (nc-76-6-108-199.dhcp.embarqhsd.net [76.6.108.199]) (authenticated bits=0) by mojo.ru (8.12.11.20060308/8.12.10) with ESMTP id l9K2eTig026284 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 20 Oct 2007 06:40:31 +0400 Message-ID: <47196A88.90104@FreeBSD.org> Date: Fri, 19 Oct 2007 22:40:08 -0400 From: "Constantine A. Murenin" Organization: Google Summer of Code 2007 Student @ The FreeBSD Project User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.5) Gecko/20041217 X-Accept-Language: en-gb, en-gb-oed, en, en-us, ru, ru-ru, ru-su MIME-Version: 1.0 To: Poul-Henning Kamp References: <83490.1192810356@critter.freebsd.dk> In-Reply-To: <83490.1192810356@critter.freebsd.dk> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 8bit Cc: Alexander@Leidinger.net, "Constantine A. Murenin" , arch@FreeBSD.org Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Oct 2007 02:57:11 -0000 On 19/10/2007 12:12, Poul-Henning Kamp wrote: > In message <20071019.100516.74722974.imp@bsdimp.com>, Warner Losh writes: > > >>>The kernel-userland interface should happen over a filedescriptor >>>(either device or unix-domain socket) so that whatever daemon we >>>park on the fd can just use select/poll/kqueue to wait for events. >> >>If we're going to have a stream of data from the kernel, is there any >>reason to invent another daemon for that? We already have devd that >>deals with a number of disparate events from the kernel in a fairly >>generic way. > > > The kernel and userland sensors would result in two kinds of data, > measurements ("32°C") and events ("new sensor", "high temp") In such case, measurements can be taken through the sensors framework and events could be sent via the devctl(4) mechanism. So it doesn't look like we disagree so far, now does it? :-) > devd should not see the measurements, but it might be a good idea > if it could see the events from all sensors (userland AND kernel). Sure, we can add some devctl_notify calls to the functions that are used to register sensor devices. I don't see any problem with this. As far as the userland part goes -- well, I totally agree with Alexander about his email mentioning eierlegendewollmichsau. :-) Cheers, Constantine. From owner-freebsd-arch@FreeBSD.ORG Sat Oct 20 04:18:43 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCC1D16A418; Sat, 20 Oct 2007 04:18:43 +0000 (UTC) (envelope-from cnst@FreeBSD.org) Received: from mojo.ru (mojo.ru [84.252.152.63]) by mx1.freebsd.org (Postfix) with ESMTP id 019A913C46A; Sat, 20 Oct 2007 04:18:41 +0000 (UTC) (envelope-from cnst@FreeBSD.org) Received: from [192.168.0.16] (nc-76-6-108-199.dhcp.embarqhsd.net [76.6.108.199]) (authenticated bits=0) by mojo.ru (8.12.11.20060308/8.12.10) with ESMTP id l9K4IsUM004074 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 20 Oct 2007 08:18:56 +0400 Message-ID: <47198198.7080304@FreeBSD.org> Date: Sat, 20 Oct 2007 00:18:32 -0400 From: "Constantine A. Murenin" Organization: Google Summer of Code 2007 Student @ The FreeBSD Project User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.5) Gecko/20041217 X-Accept-Language: en-gb, en-gb-oed, en, en-us, ru, ru-ru, ru-su MIME-Version: 1.0 To: John Baldwin References: <200710171245.36949.jhb@freebsd.org> In-Reply-To: <200710171245.36949.jhb@freebsd.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org, "Constantine A. Murenin" Subject: Re: sensors fun.. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Oct 2007 04:18:43 -0000 On 17/10/2007 12:45, John Baldwin wrote: > [Trying to redirect this off cvs-all & friends.. ] > > So as I said previously, I thought about this some more offline last night / > this morning and looked at the code some and here are my thoughts: > > Things I like about the current sensors code: > > - I like the actual sensor object used to represent a sensor. It has a few > basic things like a string for a name, a type (I would have just done > a "units" for the value, but the type is basically that), and a basic alarm > state. (I might have done 4 states, think green, yellow, orange, red mapped > to good, warning, critical, bad. However, the 3 states in the current code > is fine. 4 states might be overkill.) > - I like having the lm(4), etc. sensors report status via an object like this > instead of just getting some string out of a tool like 'mbmon'. A framework > should provide the data so that multiple utilities can use it. > - I'm not entirely opposed to having kernel drivers for various known sensor > providers like lm(4), etc. OTOH, I could see a driver just providing ioctls > to query the list of sensors and a sensor's value similar to using requests > to the IPMI BMC to request SDR data via ioctls to /dev/ipmi0 as the kernel -> > userland interface. > > Things I'm not a big fan of: > > - Forcing all sensors to be in the kernel. A general rule in most OSes is to > minimize the amount of stuff in the kernel. The kernel is easily the most, > erm, "sensitive" process in the system. A segfault has much more serious > consequences in the kernel (panic) than in userland (SIGSEGV). It also makes > it more complicated to add "psuedo" sensors. > > For example, if I wanted a sensor for CPU usage based on the kern.cp_time > sysctl, I could easily do that in userland by quering the sysctl > periodically, computing the relative % busy and set a status based on a set > of trigger points. Similarly, you could conceive having > virtual/pseudo/whatever sensors for disk space, etc. At some point you do > risk duplicating SNMP traps with sensord I suppose. This is an overengineering, IMHO. There is no reason to tie kern.cp_time with the sensors framework. kern.cp_time is kern.cp_time. > > I also genuinely think it is better to keep lots of state around in userland > rather than in the kernel. That is, I think the kernel should provide a way > to query a sensor (RAID drivers provide ioctls to communicate with the > firmware, IPMI has ioctls to allow userland to communicate with the BMC, some > drivers may provide ioctl/sysctl/whatever to read sensor values directly), > but I don't think we should try to store history (see the cp_time example > above) or extended state (keeping track of which drives exist so you can > detect a drive that goes away) in the kernel. That belongs in userland. I > think it can be ok for some sensors to be completely in the kernel and just > get queried directly from userland, but I don't think that is a valid design > constraint to enforce on all sensors. > > Things I think are dubious at best: > > - That it is more secure to put code in the kernel than as root in userland. > It seems odd to even have to mention this, but it should be painfully obvious > that a bug in a driver has much worse consequences than a bug in a user app > running as root (or even better, some dedicated non-privileged account in a > group that can send ioctls to /dev/ipmi0 or other monitored devices). I > guess maybe I can see a viewpoint where you hope that a driver bug always > panics and doesn't just corrupt data and so it's more likely to get a SA's > attention and be less exploitable maybe? I can understand that 'mbmon' is > untrusted code, but it seems to me that there are some less drastic measures > than rewriting it all as kernel code, namely 1) audit the existing code, or > 2) rewrite it all as userland code, ideally less privileged. > > Other things that might be nice: > > - IWBN to have a userland interface to sensors. For example, if nothing else > a sensor enumerator rather than duplicating the sysctl loop as the current > code does. This would make it easier to at least adjust the current > artificial limit on the number of sensors since only one place in userland > would have to change. (BTW, having an artificial limit on the number of > sensors is lame. This is an example where using the normal way of walking a > sysctl tree is superior. You can lose the entire limit.) Having a userland > interface also makes it easier to have backends that are entirely in > userland. There is no artificial limit to the number of sensors. Each sensor device can have (2^31 - 1) sensors of each type. The limit that is present only affects the number of sensor devices. As far as I'm concerned, this limit is still long to being crossed even in OpenBSD, which has so many sensors on server-grade machines. > - An snmp module that uses the above userland interface to export sensors via > snmp. bsnmp already has a way to load modules at runtime (at least startup > time) to add new MIBS. This would allow remote monitoring of various sensors > if people prefer that to having a daemon on the box post alerts. If nothing > else, it lets you add mrtg or rrdtool type graphs of the history of sensors > if desired. Have you seen ports/sysutils/symon? It already supports NetBSD's and OpenBSD's sensor frameworks, and I was planning to submit a patch for FreeBSD, but you know how the events turned out. :-) syrinx@ was also planning to do the snmp part. > > Basically, I think there should be a "real" abstracted interface in userland > that can use various backends. One backend could be to query sensors from > drivers that provide them directly (lm(4), etc.). Another backend could use > the existing IPMI interface to query SDR sensors via IPMI commands to the > BMC. Different RAID controllers could provide backends that communicate with > the firmware to maintain whatever state is needed, etc. but w/o doing all > that in the device driver. People could write their own custom sensors w/o > having to write a kernel module. Maybe that's a bigger vision than you were > shooting for. I'm not sure phk@ will agree with this one either fwiw. :) I'd say -- go ahead! The current FreeBSD port of the OpenBSD's hardware sensors framework is very usable and stable as it is. If people want to create some radical FreeBSD-specific changes, they are obviously free to do so -- the BSD licence and spirit prevail! If you want to start from scratch, be that so, too. Linux has "you must install and configure it first" lm_sensors, NetBSD has "overengineered in some parts, underengineered in others" envsys(4), OpenBSD has the sensors.h framework with "half a hundred sensor drivers in GENERIC". If people have problems with the architecture of this framework as it is related to FreeBSD, they are more than welcome to design yet another incompatible sensor framework specifically for FreeBSD! Actually, this sounds like the next thing that should be added to the ideas page. :) Competition only makes us go forward, and I'm looking forward to your work! Unless this discussion is only about words, of course. ;) Cheers, Constantine. From owner-freebsd-arch@FreeBSD.ORG Sat Oct 20 19:14:30 2007 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E67216A468 for ; Sat, 20 Oct 2007 19:14:30 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3B0A513C448 for ; Sat, 20 Oct 2007 19:14:30 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 936B748271 for ; Sat, 20 Oct 2007 14:55:16 -0400 (EDT) Date: Sat, 20 Oct 2007 19:55:16 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: arch@FreeBSD.org Message-ID: <20071020184330.C70919@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: Lock profiling results on TCP and an 8.x project X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Oct 2007 19:14:30 -0000 Dear all: This is just an FYI e-mail since I have some early measurement results I thought I'd share. I've started to look at decomposing tcbinfo, and preparatory to that, I ran some lock profiling on simple TCP workloads to explore where and how contention is arising. I used a three-machine configuration in the netperf cluster -- two client boxes (tiger-1, tiger-3) linked to a 4-core amd64 server by two dedicated gig-e ethernet (cheetah). In one test, I ran netserver on cheetah, and in the other, netrate's httpd. I ran the respective clients on tiger-1 and tiger-3 in both tests. One important property of this configuration is that because there are independent network links, the ithreads for the two devices can be scheduled independently, and run the network stack to completion via direct dispatch, offering the opportunity for full parallelism in the TCP input path (subject to limits in our locking model). Each sample was gathered for approximately 10 seconds during the run (40 seconds oof CPUish time over 4 cores). In the netperf test, I used two independent TCP streams, one per interface with the TCP stream benchmark in the steady state. This should essentially consist of cheetah receiving large data packets and sending back small ACKs; in principle the two workloads are entirely independent, although in practice TCP locking doesn't allow that, and you get potential interactions due to the memory allocator, scheduler, etc. In the http test, I configured 32 workers each on tiger-1 and tiger-3, and serviced them with a 128-worker httpd on cheetah. The file transfered was 1k, and it was the same 1k file repeatedly sent via sendfile. Unlike the netperf test, this resulted in very little steady state TCP traffic--the entire request fits in one segment, and the file fits in a second segment. Also, workers are presumably available to move back and forth between work sources, and theres a single shared listen socket. I.e., opportunities for completely independent operation are significantly reduced, and there are lots of globally visible TCP state changes. Netperf test top wait_total locks: Seconds Instance 5.75s tcp_usrreq.c:729 (inp) 2.18s tcp_input.c:479 (inp) 1.67s tcp_input.c:400 (tcp) 0.32s uipc_socket.c:1424 (so_rcv) 0.28s tcp_input.c:1191 (so_rcv) 0.20s kern_timeout.c:419 (callout) 0.09s route.c:147 (radix node head) ... In this test, the top four locking points are responsible for consuming 25% of available CPU*. We can reasonably assume that the contention on 'inp' and to a lesser degree 'so_rcv' is between the ithread and netserver processes for each network interface and that they duke it out significantly due to generating ACKs, moving data in and out of socket buffers, etc. Only the 'tcp' lock reflects interference between the two otherwise independent sessions operating over the independent links. Http test top wait_total locks: Seconds Instance 8.50s tcp_input.c:400 (tcp) 2.21s tcp_usrreq.c:568 (tcp) 1.96s tcp_usrreq.c:955 (tcp) 0.78s subr_turnstile.c:546 (chain) 0.52s tcp_usrreq.c:606 (inp) 0.16s subr_turnstile.c:536 (chain) 0.13s tcp_input.c:2867 (so_rcv) 0.12s kern_timout.c:419 (callout) 0.08s route.c:147 (radix node head) ... In this test, the top four locking points are responsible for consuming 34% of available CPU*. Here, it is clear that the global 'tcp' lock is responsible for most of the suffering as a result of the lock getting held across the input path for most packets. This is in contrast to the steady state flow, in which most packets require only brief tcbinfo lookups and not extended holding time required for packets that may lead to state changes (syn, fin, etc). Also, this is the send path, which is directly dispatched from the user code all the way to the interface queue or device driver, so there's no heavy contention in the handoff between the two (and hence inp hammering) in this direction. Jeff and Attilio tell me that the turnstile contention is simply a symptom of heavy contention on the other mutexes in the work, and not a first order effect. These results appear to confirm that we need to look at breaking out the tcbinfo lock as a key goal for 8.0, with serious thoughts about an MFC once stabilized and if ABI-safe--these two workloads represent the extremes of TCP workloads, and real world configurations will fall in between, but as the number of cores goes up and the desire to spread work over CPUs goes up, so will contention on a single global lock. An important part of this will be establishing models for distributing the work over CPUs in such a way as to avoid contention while still allowing load balancing. Anyhow, just an early report as I continue my investigation of this issue... * When talking about percentage of available CPUs, I make the assumption that due to a sufficient quantity of CPUs, in most cases lock acquisition will occur as a result of adaptive spinning rather than sleeping. In the netperf case, this is not true, since the number of potential workers exceeds the number of CPUs, hence the turnstile contention. However, as sleeping on locks itself is very expensive, it's reasonable to assume we would recover a lot of CPU none-the-less. Robert N M Watson Computer Laboratory University of Cambridge