From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 29 11:02:27 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 032B1106566C for ; Sun, 29 Jan 2012 11:02:27 +0000 (UTC) (envelope-from pho@holm.cc) Received: from relay00.pair.com (relay00.pair.com [209.68.5.9]) by mx1.freebsd.org (Postfix) with SMTP id AABA48FC0A for ; Sun, 29 Jan 2012 11:02:26 +0000 (UTC) Received: (qmail 90964 invoked from network); 29 Jan 2012 10:35:44 -0000 Received: from 87.58.144.241 (HELO x2.osted.lan) (87.58.144.241) by relay00.pair.com with SMTP; 29 Jan 2012 10:35:44 -0000 X-pair-Authenticated: 87.58.144.241 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.4/8.14.4) with ESMTP id q0TAZhAb018571; Sun, 29 Jan 2012 11:35:43 +0100 (CET) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.4/8.14.4/Submit) id q0TAZgtd018570; Sun, 29 Jan 2012 11:35:42 +0100 (CET) (envelope-from pho) Date: Sun, 29 Jan 2012 11:35:42 +0100 From: Peter Holm To: Attilio Rao Message-ID: <20120129103542.GA18535@x2.osted.lan> References: Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org, Florian Smeets , Ryan Stone Subject: Re: Kernel threads inherit CPU affinity from random sibling X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 11:02:27 -0000 On Sat, Jan 28, 2012 at 02:39:17PM +0100, Attilio Rao wrote: > 2012/1/28 Attilio Rao : > > 2012/1/28 Ryan Stone : > >> On Fri, Jan 27, 2012 at 10:41 PM, Attilio Rao wrote: > >>> I think what you found out is very sensitive. > >>> However, the patch is not correct as you cannot call > >>> cpuset_setthread() with thread_lock held. > >> > >> Whoops!  I actually discovered that for myself and had already fixed > >> it, but apparently I included an old version of the patch in the > >> email. > >> > >>> Hence this is my fix: > >>> http://www.freebsd.org/~attilio/cpuset_root.patch > >> > >> Oh, I do like this better.  I tried something similar myself but > >> abandoned it because I misread how sched_affinity() was implemented by > >> 4BSD(I had gotten the impression that once TSF_AFFINITY is set it > >> could never be cleared). > > > > Do you have a pathological test-case for it? Are you going to test the patch? > > BTW, I've just now updated the patch in order to remove an added white > line and s/priority/affinity in comments. > I've tested this patch with what I got of threaded test scenarios, for 14 hours without finding any issues. - Peter From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 29 19:42:37 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A4F2106566C for ; Sun, 29 Jan 2012 19:42:37 +0000 (UTC) (envelope-from kerstin_mende-stief@genua.de) Received: from gg.genua.de (gg6.genua.de [IPv6:2001:a60:f08e:c000::1]) by mx1.freebsd.org (Postfix) with ESMTP id CFB6C8FC17 for ; Sun, 29 Jan 2012 19:42:36 +0000 (UTC) Received: from gg.genua.de (localhost.genua.de [127.0.0.1]) by gg.genua.de (8.14.3/8.14.3) with ESMTP id q0TJgYSv016238 for ; Sun, 29 Jan 2012 20:42:34 +0100 (CET) Received: (from localhost) by gg.genua.de (MSCAN) id 4/gg.genua.de/smtp-gw/mscan; Sun Jan 29 20:42:34 2012 From: Kerstin Mende-Stief To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Date: Sun, 29 Jan 2012 20:42:24 +0100 Message-ID: <1327866144.19568.53.camel@pataplan.genua.de> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Sun, 29 Jan 2012 19:56:28 +0000 Subject: Code contribution: Further development of a Security Suite for Unix/Linux (Anoubis) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2012 19:42:37 -0000 Hello, quite not sure, if you are the right people to contact. I spoke to various BSD-people on German open source events like OpenRheinRuhr, FrOSCon and Software Freedom Day and they recommended to contact this list first. I want to make a code contribution. The code is available at Sourceforge and I am looking for a community, that is willing and able to further develop the software. The software is a security suite for Unix/Linux systems called Anoubis and contains several security stages like sandbox, secure filesystem, application level firewall and playground. It is written in C/C++ and published under the BSD licence. Available distros are, amongst others, OpenBSD and sources for generic kernels. We at GeNUA developed the software suite on demand of a German Authority (Federal Office for Information Security) a couple of years ago. Unfortunately the project has been stopped after a while due to several reasons and we on the other hand missed to build up a community to keep the project alive. The code has been last updated in early 2010. The latest version of Anoubis is based on the kernel in OpenBSD 4.6. It would be a great pity to waste the software and all the work we put in. So I think it is worth a try to ask you, if you want to take the code and keep it up-to-date. Integrate it into FreeBSD (ports), keep it stand alone...just do what you want -- it doesn´t matter as long as the code stays alive :) Below are some links providing further information. Please let me know if you are interested in continuing the project. I ask you prefered, because we develop our solutions based on BSD and Anoubis would feel most comfortable in BSD environments further on :) We at GeNUA would be available for answering questions and giving you support within the beginning of your work. So please do not hesitate to contact me for further information. I will be present at the GUUG Fruehjahrsfachgespraech in Munich in March, I´ll be around at the CeBIT in Hannover and you can meet me at the Linuxtag in Berlin in May. My jabber ID is onlyk@jabber.ccc.de. Phone me, mail me.....find me :) Thank you very much and I ask for your apologies if this post may have bothered you. But I'd be so glad if I could find somebody, who takes the code and keep it alive. Best regards Kerstin Links: http://www.anoubis.org/index_1_en.html http://sourceforge.net/projects/anoubis/develop https://www.bsi.bund.de/ContentBSI/Themen/ProdukteTools/Anoubis/Anoubis.html (german site) -- Kerstin Mende-Stief Technical Sales Private Enterprise --------------------------------------------------------------- GeNUA (Gesellschaft fuer Netzwerk- und Unix-Administration) mbH --------------------------------------------------------------- Domagkstr. 7, D-85551 Kirchheim bei Muenchen Tel: +49 (89) 991950 107 Cel: +49 151 26426 107 Fax: +49 (89) 991950 999 Mail: Kerstin_Mende-Stief@GeNUA.de --------------------------------------------------------------- Geschaeftsfuehrer: Dr. Magnus Harlander, Dr. Michaela Harlander, Bernhard Schneck Amtsgericht Muenchen HRB 98238 From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 30 12:24:22 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD0B3106564A for ; Mon, 30 Jan 2012 12:24:22 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id AB65D8FC17 for ; Mon, 30 Jan 2012 12:24:22 +0000 (UTC) Received: by iaeo4 with SMTP id o4so8643271iae.13 for ; Mon, 30 Jan 2012 04:24:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=FeHYu7d1tTjWpxmtUJ1WScN0I/NkCXD358qLaAITq5g=; b=lbEfpNIMtSUC0SGOCkdOSBMVC/7Vdg4WS7MzKA8lDGQ2yKxXvDSmH0vntXjJfOt50U LTGBWsqNP4XE7aZaddG3Gj1xy5fPgKhSJLRMr3fZJuSai4Ss7oDysSDTpKaTmSYepB9G jGOq9C/BPfjzS3y1nhv7YDqOZQq8Wxstf3Mj0= MIME-Version: 1.0 Received: by 10.50.189.194 with SMTP id gk2mr18027409igc.0.1327924873243; Mon, 30 Jan 2012 04:01:13 -0800 (PST) Sender: ermal.luci@gmail.com Received: by 10.231.134.198 with HTTP; Mon, 30 Jan 2012 04:01:13 -0800 (PST) Date: Mon, 30 Jan 2012 13:01:13 +0100 X-Google-Sender-Auth: fR_g5CNymvbpIxq5T0sM5ysdytQ Message-ID: From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: freebsd-net , freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: [PATCH] multiple instances of ipfw(4) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 12:24:22 -0000 Hello, from needs on pfSense a patch for allowing multiple intances of ipfw(4) in kernel to co-exist was developed. It can be found here https://raw.github.com/bsdperimeter/pfsense-tools/master/patches/RELENG_9_0/CP_multi_instance_ipfw.diff It is used in conjuction with this tool https://raw.github.com/bsdperimeter/pfsense-tools/master/pfPorts/ipfw_context/files/ipfw_context.c It allows creation of contextes/instances and assignment of specific interfaces to specific contexts/instances. Surely i know that this is not the best way to implement generically but it gets the job done for us as it is, read below. What i would like to know is if there is interest to see such functionality in FreeBSD? I am asking first to see if there is some consensus about this as a feature, needed or not! If interest is shown i will transform the patch to allow: - ipfw(8) to manage the contextes create/destroy - ipfw(8) to manage interface membership. Closing the race of two parallell clients modifying different contextes. There is another design choice to be made about storing the membership of interfaces into contexts/instances, but i do not see that as blocking. It is quite handy feature, which can be exploited even to scale on SMP machines by extending it to bind a specific instance(with its interaces) to a specific CPU/core?! Comments/Feedback expected, Ermal From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 30 15:15:36 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 997C7106566C; Mon, 30 Jan 2012 15:15:36 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3049A8FC13; Mon, 30 Jan 2012 15:15:35 +0000 (UTC) Received: by ggnk5 with SMTP id k5so1225601ggn.13 for ; Mon, 30 Jan 2012 07:15:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=kXkCi0hPJDoz1lD+HTZ2cvV8w6bpTbavkpr6twsPJYU=; b=H71McfNozWSXLbV/TaurY5s/qgPmWEvWmzpPkdULJHFKwQnfaK1CnDU17zZQuwkStI K+GdxXAgCh2bF05Hn0H3wAT3cn+K9jLSA05h70rx9Knu4VCO8kIQVHMXFq0oxIymbiPu H9W+1YWwHfapUtSCsRk6g7RB7NqgMQsVyT3Fk= MIME-Version: 1.0 Received: by 10.50.207.72 with SMTP id lu8mr18131290igc.0.1327936534195; Mon, 30 Jan 2012 07:15:34 -0800 (PST) Sender: ermal.luci@gmail.com Received: by 10.231.134.198 with HTTP; Mon, 30 Jan 2012 07:15:34 -0800 (PST) In-Reply-To: References: Date: Mon, 30 Jan 2012 16:15:34 +0100 X-Google-Sender-Auth: rY-rq-MiUoRkyi1VdRs5bCXF2yU Message-ID: From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: [PATCH] multiple instances of ipfw(4) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 15:15:36 -0000 On Mon, Jan 30, 2012 at 3:36 PM, Ivan Voras wrote: > On 30/01/2012 13:01, Ermal Lu=E7i wrote: > >> Surely i know that this is not the best way to implement generically > > > ... probably, because it's similar to VNET... > It depends on the comparison. The same argument would hold true for multiple routing tables but still they coexist. Both usages have their scopes. > >> What i would like to know is if there is interest to see such >> functionality in FreeBSD? >> >> I am asking first to see if there is some consensus about this as a >> feature, needed or not! >> If interest is shown i will transform the patch to allow: >> - ipfw(8) to manage the contextes create/destroy >> - ipfw(8) to manage interface membership. Closing the race of two >> parallell clients modifying different contextes. > > >> It is quite handy feature, which can be exploited even to scale on SMP >> machines by extending it to bind a specific instance(with its >> interaces) to a specific CPU/core?! > > > ... which is also done by VNET+JAILS. > > You should probably port it to VNET :) See above. Nevertheless, VNET is still not production use so.... > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 Ermal From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 30 20:39:02 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF188106566B for ; Mon, 30 Jan 2012 20:39:02 +0000 (UTC) (envelope-from info@o-notation.org) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.17.9]) by mx1.freebsd.org (Postfix) with ESMTP id 50E038FC0C for ; Mon, 30 Jan 2012 20:39:01 +0000 (UTC) Received: from kant.vitec-loesung.de (p5DC92F3D.dip.t-dialin.net [93.201.47.61]) by mrelayeu.kundenserver.de (node=mreu3) with ESMTP (Nemesis) id 0LzDgT-1SeNYT2zCb-014VvP; Mon, 30 Jan 2012 21:39:00 +0100 Received: from ashpool.net-send.org (unknown [10.0.2.24]) by kant.vitec-loesung.de (Postfix) with ESMTP id C7A642A23E for ; Mon, 30 Jan 2012 21:38:38 +0100 (CET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1084) From: Matthias Zitzen In-Reply-To: <4F208787.7050605@freebsd.org> Date: Mon, 30 Jan 2012 21:38:37 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4F208787.7050605@freebsd.org> To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.1084) X-Provags-ID: V02:K0:aNa9q08SzE4WVwTJbL3yoJ6dbJazQaWUXeGkzzah8kL WeC/k3TNNKONokqRMu/lYEjmOTQcdUzP0RunVhlGQhdf4Etlqt uACyViDhoArEFxQRU4jEZ5DClHjrIEJM9nBKGxgeGn6/Hpw5RZ Lj6+rTenBWufne+UDkn5pu1nDdR/IkijCIF1p6cIVx8ZqzT5CX /HE7sUKOgRGpfqok3ozXgKQPckKoLpdWhAhW3D99DPBmTKBAfz TBsFy4YXQggWofs7brmxEjUHsFTECB3r6NKavo5Ni4p7Nn6bAJ 2FAGX7t7oLZaxenfgsX0yXK4WlHIs3BiJdC0ZhiLkNh+QhnFIz yuZN+h+g8xfQF8EomEn3YyP0RpF7CYtzn+vcBNQQC77fdQ6OjQ xazTTPDqQskCQ== Subject: Re: kqueue and note_rename X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 20:39:02 -0000 Am 25.01.2012 um 23:51 schrieb Julian Elischer: > On 1/25/12 10:44 AM, Matthias Zitzen wrote: >> Hello list, >> does anybody have an idea, how to obtain the new name of a renamed = file after the note_rename event is fired. I'm not very familiar with = filesystem-operations. I checked the typical functions like stat or = lstat, but these functions are working only with filenames. >=20 > there is no real way. > the new link to the inode could come from any point in the filesystem = so the > only way to find it would be an exhaustive search. > the only information that you could return that might make any sense = would be the inode number of the new parent. > That would allow you to follow the '..' links and do 'pwd' in effect. > I have not checked to see if that information is returned. If it isn't = it might be a really nice enhancement to see if it could be added. >=20 Is there a any other method to get fileevents on FressBSD? especially = with the filenames too? Matthias= From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 30 21:10:09 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF59F106564A; Mon, 30 Jan 2012 21:10:09 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 144018FC1C; Mon, 30 Jan 2012 21:10:08 +0000 (UTC) Received: by werm13 with SMTP id m13so3195044wer.13 for ; Mon, 30 Jan 2012 13:10:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=NnlDcVmT74gNiKFqLFj27knrHHj6HGAhGLfTSsE01UM=; b=dZ0nvXRcaB1BitYCWa6LjFzuiMk0pzo/UiGUe3hJuZtZa63INZ1BHEaMt01klO+IlM BPjRW5EXCvSNQwkJaWWlCk9bM/IHAgUpa0jo90vtOAoA2gBYTkgk/XO8d6M4dlEDedWO alewQOmfIh3XoqS1hEeQoFBuUFLNzF8DpTrc4= MIME-Version: 1.0 Received: by 10.216.131.67 with SMTP id l45mr7627971wei.21.1327957807871; Mon, 30 Jan 2012 13:10:07 -0800 (PST) Received: by 10.180.106.129 with HTTP; Mon, 30 Jan 2012 13:10:07 -0800 (PST) In-Reply-To: References: Date: Mon, 30 Jan 2012 16:10:07 -0500 Message-ID: From: Ryan Stone To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Cc: Peter Holm , Florian Smeets , freebsd-hackers@freebsd.org Subject: Re: Kernel threads inherit CPU affinity from random sibling X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 21:10:10 -0000 On Sat, Jan 28, 2012 at 8:22 AM, Attilio Rao wrote: > Do you have a pathological test-case for it? Are you going to test the patch? > > Thanks, > Attilio I tested the patch last night. Previously I was able to see a softclock thread preempted for over 1ms on machine where 4/8 cores were lightly loaded. With this patch I can see that the softclock threads are free to migrate cores and they no longer see long scheduling latencies. From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 30 22:22:53 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC240106566B; Mon, 30 Jan 2012 22:22:53 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 81E268FC17; Mon, 30 Jan 2012 22:22:53 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id q0UMMkFm003197 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 30 Jan 2012 14:22:52 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4F271882.602@freebsd.org> Date: Mon, 30 Jan 2012 14:24:02 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.25) Gecko/20111213 Thunderbird/3.1.17 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Ermal_Lu=E7i?= References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-net , freebsd-hackers@freebsd.org Subject: Re: [PATCH] multiple instances of ipfw(4) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 22:22:53 -0000 On 1/30/12 4:01 AM, Ermal Luçi wrote: > Hello, > > from needs on pfSense a patch for allowing multiple intances of > ipfw(4) in kernel to co-exist was developed. > It can be found here > https://raw.github.com/bsdperimeter/pfsense-tools/master/patches/RELENG_9_0/CP_multi_instance_ipfw.diff > > It is used in conjuction with this tool > https://raw.github.com/bsdperimeter/pfsense-tools/master/pfPorts/ipfw_context/files/ipfw_context.c > It allows creation of contextes/instances and assignment of specific > interfaces to specific contexts/instances. > > Surely i know that this is not the best way to implement generically > but it gets the job done for us as it is, read below. > > What i would like to know is if there is interest to see such > functionality in FreeBSD? > > I am asking first to see if there is some consensus about this as a > feature, needed or not! > If interest is shown i will transform the patch to allow: > - ipfw(8) to manage the contextes create/destroy > - ipfw(8) to manage interface membership. Closing the race of two > parallell clients modifying different contextes. > > There is another design choice to be made about storing the membership > of interfaces into contexts/instances, but i do not see that as > blocking. > > It is quite handy feature, which can be exploited even to scale on SMP > machines by extending it to bind a specific instance(with its > interaces) to a specific CPU/core?! for this I use multiple vimages, but just as there is room for multiplt routing tables AND vimage, there is probably room for multiple firewalls AND vimage. this is a bit more in the iptables direction I guess. > Comments/Feedback expected, > Ermal > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 31 08:53:31 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 736241065677; Tue, 31 Jan 2012 08:53:31 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 172038FC14; Tue, 31 Jan 2012 08:53:30 +0000 (UTC) Received: by iaeo4 with SMTP id o4so10620269iae.13 for ; Tue, 31 Jan 2012 00:53:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=EqXGhg4V/rlM5rzkuz1bpB1/onweZIQ8GUV+rTL+1WM=; b=OaG8+W7gGxNU3L3ohYS9bZ5RJk7YJZvxy/cxe/V2ms33HFoZc3ksAGi3v/fVQcIBZD oDV7DEH5MAdxJGJ0peNzB4vET7MR5MO9/1+8OUvb8Z7+lo24FgAVcrIIoBxo9JBhpRYc LSKsPNjYpl7Puf3/LjllK6ojHhm27ib+xcC9c= MIME-Version: 1.0 Received: by 10.50.88.163 with SMTP id bh3mr21138643igb.0.1328000010299; Tue, 31 Jan 2012 00:53:30 -0800 (PST) Sender: ermal.luci@gmail.com Received: by 10.231.134.198 with HTTP; Tue, 31 Jan 2012 00:53:30 -0800 (PST) In-Reply-To: References: Date: Tue, 31 Jan 2012 09:53:30 +0100 X-Google-Sender-Auth: zgwPPma5VZs-sQfTlSf4HGDFMZQ Message-ID: From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: vadim_nuclight@mail.ru Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: [PATCH] multiple instances of ipfw(4) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 08:53:31 -0000 On Mon, Jan 30, 2012 at 10:08 PM, Vadim Goncharov wrote: > Hi Ermal Lu?i! > > On Mon, 30 Jan 2012 13:01:13 +0100; Ermal Lu?i wrote about '[PATCH] multi= ple instances of ipfw(4)': > >> from needs on pfSense a patch for allowing multiple intances of >> ipfw(4) in kernel to co-exist was developed. >> It can be found here >> https://raw.github.com/bsdperimeter/pfsense-tools/master/patches/RELENG_= 9_0/CP_multi_instance_ipfw.diff > > Hmm, looking at the lines > > =A0 =A0 =A0 =A0if (oif && !(oif->if_flags & IFF_IPFW_FILTER)) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return (IP_FW_PASS); > > it appears that a patch is made against somewhat private, I couldn't find= that > in stock FreeBSD. Yeah its not so polished patch, and the remaining parts are still there in the same repo. Though its redundant to this patch. > >> It is used in conjuction with this tool >> https://raw.github.com/bsdperimeter/pfsense-tools/master/pfPorts/ipfw_co= ntext/files/ipfw_context.c >> It allows creation of contextes/instances and assignment of specific >> interfaces to specific contexts/instances. > > It is not clear how to add a rule to a specific instance with this progra= m. > Simple example: - Create a context with members ipfw_context -a testctx ipfw_context -a testctx -n myiface0 ipfw_context -a testctx -n myiface1 - Set the context ipfw_context -s testctx - Continue business as usual with ipfw/dummynet ipfw add .... ipfw pipe create .... ipfw table add .... >> Surely i know that this is not the best way to implement generically >> but it gets the job done for us as it is, read below. > >> What i would like to know is if there is interest to see such >> functionality in FreeBSD? > >> I am asking first to see if there is some consensus about this as a >> feature, needed or not! >> If interest is shown i will transform the patch to allow: >> - ipfw(8) to manage the contextes create/destroy >> - ipfw(8) to manage interface membership. Closing the race of two >> parallell clients modifying different contextes. > >> There is another design choice to be made about storing the membership >> of interfaces into contexts/instances, but i do not see that as >> blocking. > >> It is quite handy feature, which can be exploited even to scale on SMP >> machines by extending it to bind a specific instance(with its >> interaces) to a specific CPU/core?! > > Not so simple/straightforward questions. On the one hand, there are at le= ast > /some/ people who need this. So the ipfw 'call'/'return' actions were alr= eady > implemented, first appearing in 9.0-R / 8.3-R. And melifaro@ has patches = to > ipfw table allowing matching againt ifname, setting tablearg, which in > conjunction with 'call' or 'skipto' could be used to make essentially the= same > functionality as your proposed patch, already in stock FreeBSD. > Well it tends to get messy if you do not have a smart consumer handling the jumps. Its almost as reprogramming in ipfw language and somehow an admin needs to read this! The intention was be practical and allow easy troubleshooting. > On the other hand, both ipfw contexts and ipfw 'call' are very half-measu= res. > The only goal was to give people something right now, and see how much th= is > will be demanded, what feedback they'll give, etc. It is obvious there is= no > wide testing of 9.0-R (and 8.3-R too) right now. > It depends on the needs, surely and how colorful you want it to be. > What I mean here about half-measures? The ipfw 'call' is just a sketch of= my > old ideas to completely reorganize ipfw to support multiple rulesets. To = be > generic and Right Thing(tm), this is a HUGE work, because: > > - each ipfw chain becomes independent netgraph(4) node > - generic ng_pfil node usable not only for firewalls > - chains may be called from each other (see iptables) > - chains (actually netgraph nodes) may be bind to ifaces or any other pla= ce > - main unnamed chain is called from pfil as before > - rewrite ipfw & dummynet management from setsockopt() to netgraph messag= es > - completely rewrite ipfw dynamic rules to not conflict with multiple > =A0rulesets, as now they just jump to parent static rule (need to be more= like > =A0pf or iptables states). This item is hard but essential (you'll get a = mess > =A0jumping to another ruleset), and ipfw contexts don't handle ot > - while here, do other needed things, e.g. adding support for modules in = both > =A0kernel and userland, loadable opcodes, keywords, etc. > if you write a ip/tcp/udp/... stack on netgraph than i will write this :) Though its a matter of preference and how much work its needed to accomplish this! Surely ipfw has seen a lot of hacks in the past and will see in the future but i thik usability is more of a target rather than fancy design. But surely nothing should stop both ways. > Even if not add something like bpf, that's ipfw_ng is probably a more maj= or > change than both ipfw2 and ipfw3 :) > > Due to various reasons in my private life, I was unable to do it in my sp= are > time previous years. My new employer is a provider using FreeBSD on most > machines, so I hope I could finally begin doing it at work (and for work)= , > but only several months later after more actual tasks. > > But, all of this only makes sense to be generic for needs of broad masses= of > our users. And in pfSense ipfw users are actually only it's authors (all > others see web interface), so it's better and more practical to not rely = on > such complex solution, but rather on something more simple and specialize= d for > their needs. Such as your proposed ipfw contexts. > Surely enough you can take shortcuts in a customization but its not the point here. > -- > WBR, Vadim Goncharov. ICQ#166852181 =A0 =A0 =A0 mailto:vadim_nuclight@mai= l.ru > [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 Ermal From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 31 09:20:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1030C106566C; Tue, 31 Jan 2012 09:20:11 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id D53648FC14; Tue, 31 Jan 2012 09:20:10 +0000 (UTC) Received: from julian-mac.elischer.org (c-67-180-24-15.hsd1.ca.comcast.net [67.180.24.15]) (authenticated bits=0) by vps1.elischer.org (8.14.4/8.14.4) with ESMTP id q0V9K72Z005065 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 31 Jan 2012 01:20:09 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <4F27B293.5000605@freebsd.org> Date: Tue, 31 Jan 2012 01:21:23 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.25) Gecko/20111213 Thunderbird/3.1.17 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Ermal_Lu=E7i?= References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: vadim_nuclight@mail.ru, freebsd-net@freebsd.org, freebsd-hackers@freebsd.org, freebsd-ipfw@freebsd.org Subject: Re: [PATCH] multiple instances of ipfw(4) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 09:20:11 -0000 On 1/31/12 12:53 AM, Ermal Luçi wrote: > On Mon, Jan 30, 2012 at 10:08 PM, Vadim Goncharov > wrote: >> Hi Ermal Lu?i! >> >> On Mon, 30 Jan 2012 13:01:13 +0100; Ermal Lu?i wrote about '[PATCH] multiple instances of ipfw(4)': >> >>> from needs on pfSense a patch for allowing multiple intances of >>> ipfw(4) in kernel to co-exist was developed. >>> It can be found here >>> https://raw.github.com/bsdperimeter/pfsense-tools/master/patches/RELENG_9_0/CP_multi_instance_ipfw.diff >> Hmm, looking at the lines >> >> if (oif&& !(oif->if_flags& IFF_IPFW_FILTER)) >> return (IP_FW_PASS); >> >> it appears that a patch is made against somewhat private, I couldn't find that >> in stock FreeBSD. > Yeah its not so polished patch, and the remaining parts are still > there in the same repo. > Though its redundant to this patch. > >>> It is used in conjuction with this tool >>> https://raw.github.com/bsdperimeter/pfsense-tools/master/pfPorts/ipfw_context/files/ipfw_context.c >>> It allows creation of contextes/instances and assignment of specific >>> interfaces to specific contexts/instances. >> It is not clear how to add a rule to a specific instance with this program. >> > Simple example: > - Create a context with members > ipfw_context -a testctx > ipfw_context -a testctx -n myiface0 > ipfw_context -a testctx -n myiface1 > - Set the context > ipfw_context -s testctx > - Continue business as usual with ipfw/dummynet > ipfw add .... > ipfw pipe create .... > ipfw table add .... > > >>> Surely i know that this is not the best way to implement generically >>> but it gets the job done for us as it is, read below. >>> What i would like to know is if there is interest to see such >>> functionality in FreeBSD? >>> I am asking first to see if there is some consensus about this as a >>> feature, needed or not! >>> If interest is shown i will transform the patch to allow: >>> - ipfw(8) to manage the contextes create/destroy >>> - ipfw(8) to manage interface membership. Closing the race of two >>> parallell clients modifying different contextes. >>> There is another design choice to be made about storing the membership >>> of interfaces into contexts/instances, but i do not see that as >>> blocking. >>> It is quite handy feature, which can be exploited even to scale on SMP >>> machines by extending it to bind a specific instance(with its >>> interaces) to a specific CPU/core?! >> Not so simple/straightforward questions. On the one hand, there are at least >> /some/ people who need this. So the ipfw 'call'/'return' actions were already >> implemented, first appearing in 9.0-R / 8.3-R. And melifaro@ has patches to >> ipfw table allowing matching againt ifname, setting tablearg, which in >> conjunction with 'call' or 'skipto' could be used to make essentially the same >> functionality as your proposed patch, already in stock FreeBSD. >> > Well it tends to get messy if you do not have a smart consumer > handling the jumps. > Its almost as reprogramming in ipfw language and somehow an admin > needs to read this! > The intention was be practical and allow easy troubleshooting. > >> On the other hand, both ipfw contexts and ipfw 'call' are very half-measures. >> The only goal was to give people something right now, and see how much this >> will be demanded, what feedback they'll give, etc. It is obvious there is no >> wide testing of 9.0-R (and 8.3-R too) right now. >> > It depends on the needs, surely and how colorful you want it to be. > >> What I mean here about half-measures? The ipfw 'call' is just a sketch of my >> old ideas to completely reorganize ipfw to support multiple rulesets. To be >> generic and Right Thing(tm), this is a HUGE work, because: >> >> - each ipfw chain becomes independent netgraph(4) node what about the existing netgraph ipfw node someone wrote a few years ago? I saw it but don't know if it was sent out publicly. >> - generic ng_pfil node usable not only for firewalls >> - chains may be called from each other (see iptables) >> - chains (actually netgraph nodes) may be bind to ifaces or any other place >> - main unnamed chain is called from pfil as before >> - rewrite ipfw& dummynet management from setsockopt() to netgraph messages >> - completely rewrite ipfw dynamic rules to not conflict with multiple >> rulesets, as now they just jump to parent static rule (need to be more like >> pf or iptables states). This item is hard but essential (you'll get a mess >> jumping to another ruleset), and ipfw contexts don't handle ot >> - while here, do other needed things, e.g. adding support for modules in both >> kernel and userland, loadable opcodes, keywords, etc. >> > if you write a ip/tcp/udp/... stack on netgraph than i will write this :) > Though its a matter of preference and how much work its needed to > accomplish this! > Surely ipfw has seen a lot of hacks in the past and will see in the > future but i thik usability is more of a target > rather than fancy design. > > But surely nothing should stop both ways. > >> Even if not add something like bpf, that's ipfw_ng is probably a more major >> change than both ipfw2 and ipfw3 :) >> >> Due to various reasons in my private life, I was unable to do it in my spare >> time previous years. My new employer is a provider using FreeBSD on most >> machines, so I hope I could finally begin doing it at work (and for work), >> but only several months later after more actual tasks. >> >> But, all of this only makes sense to be generic for needs of broad masses of >> our users. And in pfSense ipfw users are actually only it's authors (all >> others see web interface), so it's better and more practical to not rely on >> such complex solution, but rather on something more simple and specialized for >> their needs. Such as your proposed ipfw contexts. >> > Surely enough you can take shortcuts in a customization but its not > the point here. > >> -- >> WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru >> [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 31 11:01:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BEA2A106564A; Tue, 31 Jan 2012 11:01:33 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 635B98FC20; Tue, 31 Jan 2012 11:01:33 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id DA28F7300A; Tue, 31 Jan 2012 12:02:04 +0100 (CET) Date: Tue, 31 Jan 2012 12:02:04 +0100 From: Luigi Rizzo To: Ermal Lu?i Message-ID: <20120131110204.GA95472@onelab2.iet.unipi.it> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , freebsd-hackers@freebsd.org Subject: Re: [PATCH] multiple instances of ipfw(4) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 11:01:33 -0000 On Mon, Jan 30, 2012 at 01:01:13PM +0100, Ermal Lu?i wrote: > Hello, > > from needs on pfSense a patch for allowing multiple intances of > ipfw(4) in kernel to co-exist was developed. > It can be found here > https://raw.github.com/bsdperimeter/pfsense-tools/master/patches/RELENG_9_0/CP_multi_instance_ipfw.diff > > It is used in conjuction with this tool > https://raw.github.com/bsdperimeter/pfsense-tools/master/pfPorts/ipfw_context/files/ipfw_context.c > It allows creation of contextes/instances and assignment of specific > interfaces to specific contexts/instances. > > Surely i know that this is not the best way to implement generically > but it gets the job done for us as it is, read below. > > What i would like to know is if there is interest to see such > functionality in FreeBSD? > > I am asking first to see if there is some consensus about this as a > feature, needed or not! > If interest is shown i will transform the patch to allow: > - ipfw(8) to manage the contextes create/destroy > - ipfw(8) to manage interface membership. Closing the race of two > parallell clients modifying different contextes. > > There is another design choice to be made about storing the membership > of interfaces into contexts/instances, but i do not see that as > blocking. > > It is quite handy feature, which can be exploited even to scale on SMP > machines by extending it to bind a specific instance(with its > interaces) to a specific CPU/core?! > > Comments/Feedback expected, if i understand what the patch does, i think it makes sense to be able to hook ipfw instances to specific interfaces/sets of interfaces, as it permits the writing of more readable rulesets. Right now the workaround is start the ruleset with skipto rules matching on interface names, and then use some discipline in "reserving" a range of rule numbers to each interface. Before making more changes to the code, it would help if you could give a high level description of 1. what the change does and how specific cases are handled. E.g. With this change you can create multiple rulesets (contexts ?) and bind one or more interfaces to a context. - what happens with outgoing packets where the context to be picked up depend on the route in effect at the time of the transmission ? - what happens with encapsulated interfaces (vlan) ? - can you skipto across contexts (i guess not) ? 2. how intrusive are code changes ? The kernel patch you show seems small, which makes sense as i believe all is needed is to start from a specific chain instead of the default one when an interface is bound to a context. A few comments: - if you use one of the if_ispare directly, instead of renaming it to if_context, this would make backporting and testing easier. - I think the explosion of sockopt names is a bad thing. The IP_FW3 command was introduced exactly to have a single entry point to the firewall and avoid a ton of new names in raw_ip.c and in.h - can you reduce the number of global ipfw-related variables ? There used to be one (layer3_chain), now you have 3 of them. You should delete layer3_chain and replace it with another single global (could be ip_fw_contexts) which contains the whole firewall state. - how do you handle reinjects (e.g. from dummynet or divert) ? I don't remember if the metadata that stores where you reinject the packet also has a pointer to the root of the chain. - i don't completely follow the connection between ip_fw_chain, ip_fw_ctx_iflist, ip_fw_ctxmember, ip_fw_ctx, ip_fw_ctx_list. The way i see it: - the ip_fw_chain could be embedded in the ip_fw_ctx, as they map 1:1 - why do you need ip_fw_ctx_iflist and ip_fw_ctxmember ? You should never need to determine context membership during packet processing, and for sockopt calls you could as well iterate over the list of interfaces; cheers luigi From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 31 11:50:10 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B87791065670; Tue, 31 Jan 2012 11:50:10 +0000 (UTC) (envelope-from dmitry.krivenok@emc.com) Received: from mexforward.lss.emc.com (mexforward.lss.emc.com [128.222.32.20]) by mx1.freebsd.org (Postfix) with ESMTP id 6C02D8FC15; Tue, 31 Jan 2012 11:50:09 +0000 (UTC) Received: from hop04-l1d11-si03.isus.emc.com (HOP04-L1D11-SI03.isus.emc.com [10.254.111.23]) by mexforward.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0VBZTnS008545 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 31 Jan 2012 06:35:29 -0500 Received: from mailhub.lss.emc.com (mailhub.lss.emc.com [10.254.222.226]) by hop04-l1d11-si03.isus.emc.com (RSA Interceptor); Tue, 31 Jan 2012 06:35:11 -0500 Received: from mxhub32.corp.emc.com (mxhub32.corp.emc.com [128.222.70.172]) by mailhub.lss.emc.com (Switch-3.4.3/Switch-3.4.3) with ESMTP id q0VBZAfj020058; Tue, 31 Jan 2012 06:35:11 -0500 Received: from mx19a.corp.emc.com ([169.254.1.78]) by mxhub32.corp.emc.com ([128.222.70.172]) with mapi; Tue, 31 Jan 2012 06:35:10 -0500 From: To: Date: Tue, 31 Jan 2012 06:35:08 -0500 Thread-Topic: mtx_trylock() on a spin mutex Thread-Index: AczgDGJ5vIba+6zzTaS8HmtVqYBAVg== Message-ID: <9C7BACB01B839A499792154E76C9ADE5563FDD39ED@MX19A.corp.emc.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-EMM-MHVC: 1 X-Mailman-Approved-At: Tue, 31 Jan 2012 12:11:30 +0000 Cc: Subject: mtx_trylock() on a spin mutex X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 11:50:10 -0000 Hello, Could someone please explain why mtx_trylock() must not be called on a spin= mutex? Is it conceptually wrong or is it a restriction of FreeBSD kernel implement= ation? I've written slightly modified version of mtx_trylock (mtx_trylock_spin) wh= ich calls=20 spinlock_enter/spinlock_exit and doesn't have KASSERT checking for lock cla= ss: KASSERT(m->mtx_object.lo_class =3D=3D &lock_class_mtx_sleep, ("mtx_trylock() of spin mutex %s @ %s:%d", m->mtx_object.= lo_name, file, line)); but, to my surprise, all my tests passed w/o any errors/warnings on FreeBSD= -8 with WITNESS/INVARIANTS/etc enabled. I admit that my tests may not cover all possible use cases or scenarios tho= ugh. I did a quick google search but didn't find anything useful, so your help i= s greatly appreciated. Thanks in advance! P.S. The assertion was added in the following revision: Revision 149737 - (view) (annotate) - [select for diffs]=20 Modified Fri Sep 2 20:21:49 2005 UTC (6 years, 4 months ago) by jhb=20 File length: 24623 byte(s)=20 Diff to previous 148557=20 - Add an assertion to panic if one tries to call mtx_trylock() on a spin mutex. - Don't panic if a spin lock is held too long inside _mtx_lock_spin() if panicstr is set (meaning that we are already in a panic). Just keep spinning forever instead. From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 31 14:48:00 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B7D21065686 for ; Tue, 31 Jan 2012 14:48:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 136088FC18 for ; Tue, 31 Jan 2012 14:48:00 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id B982B46B0D; Tue, 31 Jan 2012 09:47:59 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2C29CB965; Tue, 31 Jan 2012 09:47:59 -0500 (EST) From: John Baldwin To: dmitry.krivenok@emc.com Date: Tue, 31 Jan 2012 07:54:02 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: <9C7BACB01B839A499792154E76C9ADE5563FDD39ED@MX19A.corp.emc.com> In-Reply-To: <9C7BACB01B839A499792154E76C9ADE5563FDD39ED@MX19A.corp.emc.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201201310754.02343.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 31 Jan 2012 09:47:59 -0500 (EST) Cc: freebsd-hackers@freebsd.org Subject: Re: mtx_trylock() on a spin mutex X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2012 14:48:00 -0000 On Tuesday, January 31, 2012 6:35:08 am dmitry.krivenok@emc.com wrote: > Hello, > Could someone please explain why mtx_trylock() must not be called on a spin mutex? > Is it conceptually wrong or is it a restriction of FreeBSD kernel implementation? The current mutex API generally uses different methods for the different types of mutexes (at some point I may split spin locks out into a completely separate API to simplify things). If we wanted to do trylocks on spinlocks, then a new mtx_trylock_spin() method would be added as you've done. To date we haven't had any use cases for trylock operations on a spin lock. Even try locks on other locking primitives are used fairly rarely. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 1 00:28:31 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93C6F106564A for ; Wed, 1 Feb 2012 00:28:31 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by mx1.freebsd.org (Postfix) with ESMTP id E27A18FC0A for ; Wed, 1 Feb 2012 00:28:30 +0000 (UTC) Received: by wibhn14 with SMTP id hn14so698357wib.13 for ; Tue, 31 Jan 2012 16:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=b4/X6ygbAF91UuS7sXWWg+omwKqRhcEC99TzP4IvjFw=; b=XasdmzeWs5SuzVRnOeSwgQjIOaG8Pqxn5/j7/T1vlIZ/QWagvs5Wz1G1aR4YwkBonu l0XxvjuaB7JAoK7DNmzri8tAdeiZWNYOyhbGHvwTxUwOY4aUj4t/bXnyRh7JhKG5mbus kKVthRkq7Gy4ggUCF0GrAsZ8S1jIM0+P4M/s8= Received: by 10.180.101.161 with SMTP id fh1mr37839042wib.0.1328054748628; Tue, 31 Jan 2012 16:05:48 -0800 (PST) Received: from dft-labs.eu (dft-labs.eu. [80.87.128.179]) by mx.google.com with ESMTPS id x7sm15984625wif.10.2012.01.31.16.05.46 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 31 Jan 2012 16:05:47 -0800 (PST) Date: Wed, 1 Feb 2012 01:05:41 +0100 From: Mateusz Guzik To: freebsd-hackers@freebsd.org Message-ID: <20120201000541.GA31440@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Balazs Scheidler Subject: [patch] kqueue support for /dev/klog X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 00:28:31 -0000 Hello, one pr ( http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/156423 ) contains request to add kqueue support to /dev/klog. Assuming that this is a desirable feature, how about this patch (based on audit pipe's code): http://student.agh.edu.pl/~mjguzik/patches/dev_klog-kqueue.patch Tested with this trivial program: http://student.agh.edu.pl/~mjguzik/kqread.c Thanks, -- Mateusz Guzik From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 1 14:10:54 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ECAF6106564A for ; Wed, 1 Feb 2012 14:10:54 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by mx1.freebsd.org (Postfix) with ESMTP id 99CD88FC08 for ; Wed, 1 Feb 2012 14:10:53 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Rsatf-0006Ui-N0 for freebsd-hackers@freebsd.org; Wed, 01 Feb 2012 15:10:51 +0100 Received: from broadband-77-37-240-109.nationalcablenetworks.ru ([77.37.240.109]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 01 Feb 2012 15:10:51 +0100 Received: from vadim_nuclight by broadband-77-37-240-109.nationalcablenetworks.ru with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 01 Feb 2012 15:10:51 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Vadim Goncharov Date: Wed, 1 Feb 2012 14:10:42 +0000 (UTC) Organization: Nuclear Lightning @ Tomsk, TPU AVTF Hostel Lines: 184 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: broadband-77-37-240-109.nationalcablenetworks.ru X-Comment-To: Ermal Lu?i User-Agent: slrn/0.9.9p1 (FreeBSD) Cc: freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org Subject: Re: [PATCH] multiple instances of ipfw(4) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vadim_nuclight@mail.ru List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 14:10:55 -0000 Hi Ermal Lu?i! On Tue, 31 Jan 2012 09:53:30 +0100; Ermal Lu?i wrote about 'Re: [PATCH] multiple instances of ipfw(4)': >>> It is used in conjuction with this tool >>> https://raw.github.com/bsdperimeter/pfsense-tools/master/pfPorts/ipfw_context/files/ipfw_context.c >>> It allows creation of contextes/instances and assignment of specific >>> interfaces to specific contexts/instances. >> >> It is not clear how to add a rule to a specific instance with this program. >> > Simple example: > - Create a context with members > ipfw_context -a testctx > ipfw_context -a testctx -n myiface0 > ipfw_context -a testctx -n myiface1 > - Set the context > ipfw_context -s testctx > - Continue business as usual with ipfw/dummynet > ipfw add .... > ipfw pipe create .... > ipfw table add .... So one must need change context between invocations of ipfw? Not very user friendly and subject to races. How are you going to fix them? >>> Surely i know that this is not the best way to implement generically >>> but it gets the job done for us as it is, read below. >> >>> What i would like to know is if there is interest to see such >>> functionality in FreeBSD? >> >>> I am asking first to see if there is some consensus about this as a >>> feature, needed or not! >>> If interest is shown i will transform the patch to allow: >>> - ipfw(8) to manage the contextes create/destroy >>> - ipfw(8) to manage interface membership. Closing the race of two >>> parallell clients modifying different contextes. >> >>> There is another design choice to be made about storing the membership >>> of interfaces into contexts/instances, but i do not see that as >>> blocking. >> >>> It is quite handy feature, which can be exploited even to scale on SMP >>> machines by extending it to bind a specific instance(with its >>> interaces) to a specific CPU/core?! >> >> Not so simple/straightforward questions. On the one hand, there are at least >> /some/ people who need this. So the ipfw 'call'/'return' actions were already >> implemented, first appearing in 9.0-R / 8.3-R. And melifaro@ has patches to >> ipfw table allowing matching againt ifname, setting tablearg, which in >> conjunction with 'call' or 'skipto' could be used to make essentially the same >> functionality as your proposed patch, already in stock FreeBSD. >> > Well it tends to get messy if you do not have a smart consumer > handling the jumps. > Its almost as reprogramming in ipfw language and somehow an admin > needs to read this! > The intention was be practical and allow easy troubleshooting. Agreed that it is not easy, but that is just something available *right now* already allowing to simplify some messy rulesets with skipto's. It was done as a short-term solution until something more powerful will be done. >> On the other hand, both ipfw contexts and ipfw 'call' are very half-measures. >> The only goal was to give people something right now, and see how much this >> will be demanded, what feedback they'll give, etc. It is obvious there is no >> wide testing of 9.0-R (and 8.3-R too) right now. >> > It depends on the needs, surely and how colorful you want it to be. Yes, it's easier to get feedback on what people need when at least something was presented (implemented) to them. >> What I mean here about half-measures? The ipfw 'call' is just a sketch of my >> old ideas to completely reorganize ipfw to support multiple rulesets. To be >> generic and Right Thing(tm), this is a HUGE work, because: >> >> - each ipfw chain becomes independent netgraph(4) node >> - generic ng_pfil node usable not only for firewalls >> - chains may be called from each other (see iptables) >> - chains (actually netgraph nodes) may be bind to ifaces or any other place >> - main unnamed chain is called from pfil as before >> - rewrite ipfw & dummynet management from setsockopt() to netgraph messages >> - completely rewrite ipfw dynamic rules to not conflict with multiple >> rulesets, as now they just jump to parent static rule (need to be more like >> pf or iptables states). This item is hard but essential (you'll get a mess >> jumping to another ruleset), and ipfw contexts don't handle ot >> - while here, do other needed things, e.g. adding support for modules in both >> kernel and userland, loadable opcodes, keywords, etc. >> > if you write a ip/tcp/udp/... stack on netgraph than i will write this :) > Surely ipfw has seen a lot of hacks in the past and will see in the > future but i thik usability is more of a target > rather than fancy design. No, point is not in design, but in usability first (perhaps my English was too bad?). Design here is just a way to handle desired usability. How this will be for user: ipfw chain mychain create ipfw chain common create default-policy allow ipfw chain lanusers create ipfw chain mychain add 200 ... ipfw chain mychain delete 300 ... ipfw chain mychain destroy ipfw add 100 call common ip from any to any # call from main ruleset ipfw ng_bpf mypktbody create ipfw ng_bpf mypktbody config proto udp and udp[42:4]=0x01020304 ipfw add 200 deny ngmatch mypktbody ipfw bind mychain to interface em0 direction out ipfw bind lanusers to interface vlan100 ipfw pfil-order move ipfilter bottom Hope that's enough user-intuitive. And netgraph-based design here is very straightforward: for examples above, you just create netgraph nodes named "ipfwmychain", "ipfwcommon", etc., then do send NGM_IPFW_ADD message to the node with name of appropriate "chain" keyword, or main ng_ipfw if none. Netgraph here is not a stack, but a clean alternative to hack raw_ip.c for new setsockopt()'s (not very modular) or for user to be able manually assign node in complex configuration (usually the /sbin/ipfw will do). Aside that, netgraph messages do provide clean versioning - it was hard to keep compatible interfaces when ABI changed in 8.2. There is no more Netgraph or even stacks in this stacks, it just suits very well :) > Though its a matter of preference and how much work its needed to > accomplish this! Unfortunatelly, no. Here is the problem with dynamic rules which is the main: - suppose you have router with interfaces X, Y. - you have TCP/UDP session from A to B, incoming on X and leaving through Y - you have per-interface rules, for X on input, for Y on output Then, if a packet entered via iface X, and there is a *state* (dynamic rule) there, then, whatever you write in rules for Y - the first keep-state will accept packet, no matter what rule, it will be accepted. You can try to expoit the fact that in ipfw, dynamic rule just jumps to parent rule's action part, and, if that was "skipto", continues on a static ruleset (where it could be checked, which interface). But this becomes unmaintainable mess even on 3-4 interfaces (it's like assembly language programming), and if you have hundreds of ifaces, you may encounter there is not enough numbers in 65536 to handle all X*Y combinations (not all ifaces need be stateful) even if config is somehow generated automatically. There is company which really comes up to such situation, and ipfw context can't handle it. Then you encounter dynamic rules must be redesigned. Given another users' polls, they do need something like iptables chains, for maintenance. Then you look at hacks present in ipfw code from past, and eventually come up to something like I described earlier - another design which also allows several other good new features. >> Even if not add something like bpf, that's ipfw_ng is probably a more major >> change than both ipfw2 and ipfw3 :) >> >> Due to various reasons in my private life, I was unable to do it in my spare >> time previous years. My new employer is a provider using FreeBSD on most >> machines, so I hope I could finally begin doing it at work (and for work), >> but only several months later after more actual tasks. >> >> But, all of this only makes sense to be generic for needs of broad masses of >> our users. And in pfSense ipfw users are actually only it's authors (all >> others see web interface), so it's better and more practical to not rely on >> such complex solution, but rather on something more simple and specialized for >> their needs. Such as your proposed ipfw contexts. >> > Surely enough you can take shortcuts in a customization but its not > the point here. > But surely nothing should stop both ways. I wanted to say, don't be afraid or upset that ipfw context are not generic enough for broader needs. If that allows to solve your practical problems right now (relatively fast), then do it, it is good, it's better than wait for a more comprehensive solution. I've implemented "ipfw call" from the exactly the same thinking. -- WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru [Anti-Greenpeace][Sober FreeBSD zealot][http://nuclight.livejournal.com] From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 1 14:47:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89BDE106564A for ; Wed, 1 Feb 2012 14:47:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id F25528FC0C for ; Wed, 1 Feb 2012 14:47:10 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q11Eh49r009247 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 1 Feb 2012 16:43:04 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q11Eh41V066110; Wed, 1 Feb 2012 16:43:04 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q11Eh3re066109; Wed, 1 Feb 2012 16:43:03 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 1 Feb 2012 16:43:03 +0200 From: Konstantin Belousov To: Mateusz Guzik Message-ID: <20120201144303.GM3283@deviant.kiev.zoral.com.ua> References: <20120201000541.GA31440@dft-labs.eu> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Z/kiM2A+9acXa48/" Content-Disposition: inline In-Reply-To: <20120201000541.GA31440@dft-labs.eu> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean Cc: freebsd-hackers@freebsd.org, Balazs Scheidler Subject: Re: [patch] kqueue support for /dev/klog X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 14:47:11 -0000 --Z/kiM2A+9acXa48/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 01, 2012 at 01:05:41AM +0100, Mateusz Guzik wrote: > Hello, >=20 > one pr ( http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/156423 ) > contains request to add kqueue support to /dev/klog. >=20 > Assuming that this is a desirable feature, how about this patch (based > on audit pipe's code): > http://student.agh.edu.pl/~mjguzik/patches/dev_klog-kqueue.patch >=20 > Tested with this trivial program: > http://student.agh.edu.pl/~mjguzik/kqread.c Thank you for the submission. Committed as r230866, will merge to stable in 1 week. --Z/kiM2A+9acXa48/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk8pT3cACgkQC3+MBN1Mb4jdoQCeKJvN0zP0ndsu0Xi/GBCIm421 k+0An26fj4yZyIPpWm3/9jNsf3y64Nbz =0RYr -----END PGP SIGNATURE----- --Z/kiM2A+9acXa48/-- From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 1 15:02:18 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61395106566B for ; Wed, 1 Feb 2012 15:02:18 +0000 (UTC) (envelope-from aram_baghomian@hotmail.com) Received: from snt0-omc2-s19.snt0.hotmail.com (snt0-omc2-s19.snt0.hotmail.com [65.55.90.94]) by mx1.freebsd.org (Postfix) with ESMTP id 383EE8FC0A for ; Wed, 1 Feb 2012 15:02:18 +0000 (UTC) Received: from SNT140-W48 ([65.55.90.73]) by snt0-omc2-s19.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 1 Feb 2012 06:54:16 -0800 Message-ID: X-Originating-IP: [2.145.222.85] From: aram baghomian To: Date: Wed, 1 Feb 2012 14:54:17 +0000 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 01 Feb 2012 14:54:16.0971 (UTC) FILETIME=[5ED5C5B0:01CCE0F1] Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: crypto module X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 15:02:18 -0000 Hi=2C I want to know which kernel modules keep the information about implemented = algorithms in crypto.ko. for example in linux there is xfrm_algo modules that keep the information a= bout these algorithms. = From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 1 15:02:59 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 780BE106567A for ; Wed, 1 Feb 2012 15:02:59 +0000 (UTC) (envelope-from aram_baghomian@hotmail.com) Received: from snt0-omc2-s40.snt0.hotmail.com (snt0-omc2-s40.snt0.hotmail.com [65.54.61.91]) by mx1.freebsd.org (Postfix) with ESMTP id 5043F8FC08 for ; Wed, 1 Feb 2012 15:02:59 +0000 (UTC) Received: from SNT140-W48 ([65.55.90.72]) by snt0-omc2-s40.snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Wed, 1 Feb 2012 06:50:58 -0800 Message-ID: X-Originating-IP: [2.145.222.85] From: aram baghomian To: Date: Wed, 1 Feb 2012 14:50:58 +0000 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 01 Feb 2012 14:50:58.0626 (UTC) FILETIME=[E89CBA20:01CCE0F0] Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: (no subject) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 15:02:59 -0000 Hi=2C I want to know which kernel modules keep the information about implemented = algorithms in crypto.ko. for example in linux there is xfrm_algo modules that keep the information a= bout these algorithms. = From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 1 15:19:00 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E251106566C for ; Wed, 1 Feb 2012 15:19:00 +0000 (UTC) (envelope-from rodrigo@bebik.net) Received: from smtp3-g21.free.fr (smtp3-g21.free.fr [IPv6:2a01:e0c:1:1599::12]) by mx1.freebsd.org (Postfix) with ESMTP id 9FA788FC0C for ; Wed, 1 Feb 2012 15:18:58 +0000 (UTC) Received: from oldfaithful.bebik.local (unknown [82.227.164.69]) by smtp3-g21.free.fr (Postfix) with ESMTP id 0E7A2A634C; Wed, 1 Feb 2012 16:18:50 +0100 (CET) Received: by oldfaithful.bebik.local (Postfix, from userid 1001) id 04DB12F8D8; Wed, 1 Feb 2012 16:21:50 +0100 (CET) Date: Wed, 1 Feb 2012 16:21:50 +0100 From: Rodrigo OSORIO To: aram baghomian Message-ID: <20120201152150.GB16895@oldfaithful.bebik.local> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org Subject: Re: crypto module X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 15:19:00 -0000 On 01/02/12 14:54 +0000, aram baghomian wrote: > > > > > Hi, > > I want to know which kernel modules keep the information about implemented algorithms > in crypto.ko. > for example in linux there is xfrm_algo modules that keep the information about > these algorithms. > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" Hi, If I didn't miss the point, the implemented algorithms are those located in the /sys/crypto directory[1]. Regards Rodrigo [1] http://svnweb.freebsd.org/base/stable/9/sys/crypto/ From owner-freebsd-hackers@FreeBSD.ORG Wed Feb 1 17:51:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 670C0106566B for ; Wed, 1 Feb 2012 17:51:33 +0000 (UTC) (envelope-from PMahan@adaranet.com) Received: from barracuda.adaranet.com (smtp.adaranet.com [72.5.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 4AD758FC08 for ; Wed, 1 Feb 2012 17:51:33 +0000 (UTC) X-ASG-Debug-ID: 1328117732-586eaf4c0001-P5m3U7 Received: from SJ-EXCH-1.adaranet.com ([10.10.1.29]) by barracuda.adaranet.com with ESMTP id tIiwMRvAJLoAUAjz; Wed, 01 Feb 2012 09:35:32 -0800 (PST) X-Barracuda-Envelope-From: PMahan@adaranet.com Received: from SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523]) by SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523%14]) with mapi; Wed, 1 Feb 2012 09:35:32 -0800 From: Patrick Mahan X-Barracuda-BBL-IP: fe80::7042:d8c2:5973:c523 X-Barracuda-RBL-IP: fe80::7042:d8c2:5973:c523 To: aram baghomian , "freebsd-hackers@freebsd.org" Date: Wed, 1 Feb 2012 09:35:40 -0800 X-ASG-Orig-Subj: RE: crypto module Thread-Topic: crypto module Thread-Index: Aczg8qZrXLMYTKMJTyazOhuetNx3mAAFMtaw Message-ID: <32AB5C9615CC494997D9ABB1DB12783C02D685B803@SJ-EXCH-1.adaranet.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.10.1.29] X-Barracuda-Start-Time: 1328117732 X-Barracuda-URL: http://172.16.10.203:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at adaranet.com Cc: Subject: RE: crypto module X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 17:51:33 -0000 >-----Original Message----- >From: owner-freebsd-hackers@freebsd.org [mailto:owner-freebsd- >hackers@freebsd.org] On Behalf Of aram baghomian >Sent: Wednesday, February 01, 2012 6:54 AM >To: freebsd-hackers@freebsd.org >Subject: crypto module > > > > > >Hi, > >I want to know which kernel modules keep the information about implemented >algorithms >in crypto.ko. >for example in linux there is xfrm_algo modules that keep the information >about >these algorithms. > > No, the algorithms are access via statically allocated structures defined in sys/opencrypto/xform.c Not that there couldn't be one written, but the xforms would need to be dynamically done and have each crypto algorithm register themselves. Patrick ---------------------------------------------------- Patrick Mahan Lead Technical Kernel Engineer Adara Networks Disclaimer: The opinions expressed here are solely the responsibility of th= e author and are not to be construed as an official opinion of Adara Networks. From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 2 09:12:32 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 465F41065672; Thu, 2 Feb 2012 09:12:32 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id E28488FC0A; Thu, 2 Feb 2012 09:12:31 +0000 (UTC) Received: by ggnk5 with SMTP id k5so1506226ggn.13 for ; Thu, 02 Feb 2012 01:12:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=E4zsNyKPzGse+l82R6RaU/UefCXXR2lwDY3nI9vTKB4=; b=r6JJbbLrzJgbQ7d87/5BxSSQeJOjHerlYpbrEreappI5o4wOt06hjsJLJGfAFt+Yoy wTuSxbbP6WsiiTQvr3BdqRpPbSxs7lDk65ENP317EJVZU48fPN41jQVJbBwcXn7cQG8e 0Lj/nihFy1gQRGmXhP3X54yFRpZ051ABYT+hw= MIME-Version: 1.0 Received: by 10.50.170.41 with SMTP id aj9mr2358525igc.0.1328173950167; Thu, 02 Feb 2012 01:12:30 -0800 (PST) Sender: ermal.luci@gmail.com Received: by 10.231.134.198 with HTTP; Thu, 2 Feb 2012 01:12:30 -0800 (PST) In-Reply-To: <20120131110204.GA95472@onelab2.iet.unipi.it> References: <20120131110204.GA95472@onelab2.iet.unipi.it> Date: Thu, 2 Feb 2012 10:12:30 +0100 X-Google-Sender-Auth: ufuFW3M-I2wlrwobm5UDMiQ62OM Message-ID: From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= To: Luigi Rizzo Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net , freebsd-hackers@freebsd.org Subject: Re: [PATCH] multiple instances of ipfw(4) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2012 09:12:32 -0000 On Tue, Jan 31, 2012 at 12:02 PM, Luigi Rizzo wrote: > On Mon, Jan 30, 2012 at 01:01:13PM +0100, Ermal Lu?i wrote: >> Hello, >> >> from needs on pfSense a patch for allowing multiple intances of >> ipfw(4) in kernel to co-exist was developed. >> It can be found here >> https://raw.github.com/bsdperimeter/pfsense-tools/master/patches/RELENG_= 9_0/CP_multi_instance_ipfw.diff >> >> It is used in conjuction with this tool >> https://raw.github.com/bsdperimeter/pfsense-tools/master/pfPorts/ipfw_co= ntext/files/ipfw_context.c >> It allows creation of contextes/instances and assignment of specific >> interfaces to specific contexts/instances. >> >> Surely i know that this is not the best way to implement generically >> but it gets the job done for us as it is, read below. >> >> What i would like to know is if there is interest to see such >> functionality in FreeBSD? >> >> I am asking first to see if there is some consensus about this as a >> feature, needed or not! >> If interest is shown i will transform the patch to allow: >> - ipfw(8) to manage the contextes create/destroy >> - ipfw(8) to manage interface membership. Closing the race of two >> parallell clients modifying different contextes. >> >> There is another design choice to be made about storing the membership >> of interfaces into contexts/instances, but i do not see that as >> blocking. >> >> It is quite handy feature, which can be exploited even to scale on SMP >> machines by extending it to bind a specific instance(with its >> interaces) to a specific CPU/core?! >> >> Comments/Feedback expected, > > if i understand what the patch does, i think it makes sense to be > able to hook ipfw instances to specific interfaces/sets of interfaces, > as it permits the writing of more readable rulesets. Right now the > workaround is start the ruleset with skipto rules matching on > interface names, and then use some discipline in "reserving" a range > of rule numbers to each interface. > > Before making more changes to the code, > it would help if you could give a high level description of > > 1. what the change does and how specific cases are handled. E.g. > =A0 =A0 =A0 =A0With this change you can create multiple rulesets (context= s ?) > =A0 =A0 =A0 =A0and bind one or more interfaces to a context. In man simple words it can give you multiple rulesets in ipfw(4) together with ability to restrict the ruleset to specific interfaces. As it is written today it forces you to create the relation of interface(s) and rulesets explicitly. Though its debatable you would want the freedom of one interface to more than one ruleset. The target of this patch was strictly layer2 filtering with ipfw(4) and allowing easy managebility of multiple instances of an userland application using ipfw(4) at layer2 for filtering/forwarding/QoS purposes. > =A0 =A0 =A0 =A0- what happens with outgoing packets where the context > =A0 =A0 =A0 =A0 =A0to be picked up depend on the route in effect at the t= ime > =A0 =A0 =A0 =A0 =A0of the transmission ? For this i have not given much thought but makes sense only on satetful tracking of sessions in ipfw(4). See below for a solution implemented along with this patch. > =A0 =A0 =A0 =A0- what happens with encapsulated interfaces (vlan) ? Well by the very nature of this patch is that you have to explicitly assign an interface to the ruleset. This by definition of vlan/cloned interfaces in FreeBSD means you have to assign them to the ruleset explicitly too. In pfSense for example, since ipfw(4) is used exlusivly at layer2 it was necessary to introduce another flag and be very explicit on which interface is considered for filtering in ipfw(4). Also problems were faced on determining what was considered incoming and what outgoing when calling the ipfw hooks. This was changed to be explicit as well. Otherwise you have to play a tricky game of rules and what interface a particual pattern of traffic belongs and write respective rules about it, which is no fun at all from implementations and administration point of vie= w. > =A0 =A0 =A0 =A0- can you skipto across contexts (i guess not) ? > No its not there but it can be a possible addition. > 2. how intrusive are code changes ? The kernel patch you show > =A0 seems small, which makes sense as i believe all is needed is > =A0 to start from a specific chain instead of the default one when > =A0 an interface is bound to a context. A few comments: > =A0 =A0 =A0 =A0- if you use one of the if_ispare directly, instead of > =A0 =A0 =A0 =A0 =A0renaming it to if_context, this would make backporting= and > =A0 =A0 =A0 =A0 =A0testing easier. > =A0 =A0 =A0 =A0- I think the explosion of sockopt names is a bad thing. > =A0 =A0 =A0 =A0 =A0The IP_FW3 command was introduced exactly to have a si= ngle > =A0 =A0 =A0 =A0 =A0entry point to the firewall and avoid a ton of new nam= es > =A0 =A0 =A0 =A0 =A0in raw_ip.c and in.h > =A0 =A0 =A0 =A0- can you reduce the number of global ipfw-related variabl= es ? > =A0 =A0 =A0 =A0 =A0There used to be one (layer3_chain), now you have 3 of= them. > =A0 =A0 =A0 =A0 =A0You should delete layer3_chain and replace it with ano= ther > =A0 =A0 =A0 =A0 =A0single global (could be ip_fw_contexts) which contains= the > =A0 =A0 =A0 =A0 =A0whole firewall state. > =A0 =A0 =A0 =A0- how do you handle reinjects (e.g. from dummynet or diver= t) ? > =A0 =A0 =A0 =A0 =A0I don't remember if the metadata that stores where you > =A0 =A0 =A0 =A0 =A0reinject the packet also has a pointer to the root of = the > =A0 =A0 =A0 =A0 =A0chain. As described above a change to explicitly mark a packet incoming/outgoing interface in ipfw(4) metadata(mbuf tag) was needed. This is a patch, which is a bit verbose with other changed things in ipfw https://github.com/bsdperimeter/pfsense-tools/blob/master/patches/RELE= NG_9_0/CP_speedup.diff, has also the various bits i am talking for saving the interface and direction explicitly. > =A0 =A0 =A0 =A0- i don't completely follow the connection between ip_fw_c= hain, > =A0 =A0 =A0 =A0 =A0ip_fw_ctx_iflist, ip_fw_ctxmember, ip_fw_ctx, ip_fw_ct= x_list. > =A0 =A0 =A0 =A0 =A0The way i see it: > =A0 =A0 =A0 =A0 =A0- the ip_fw_chain could be embedded in the ip_fw_ctx, = as they > =A0 =A0 =A0 =A0 =A0 =A0map 1:1 yep that is probably a better place to put the chain and easys the integration with ipfw(8) command. > =A0 =A0 =A0 =A0 =A0- why do you need ip_fw_ctx_iflist and ip_fw_ctxmember= ? > =A0 =A0 =A0 =A0 =A0 =A0You should never need to determine context members= hip > =A0 =A0 =A0 =A0 =A0 =A0during packet processing, and for sockopt calls yo= u could > =A0 =A0 =A0 =A0 =A0 =A0as well iterate over the list of interfaces; ip_fw_ctx_iflist is used for being able to reassign an interface to its configured context. This is useful on dynamic interfaces that come and go example vlan(4). Its easier to handle this in kernel than in userland, where you have to detect the event and do the various reconfiguration things. ip_fw_ctxmember is just a simple way for the various new sockopt to pass the argument. It for sure can be recycled and implemented in a more clean way. I will try to make a patch with your recommandations and integrate the add/remove/delete of interfaces/contexts in ipfw and resend again. > > cheers > luigi > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 Ermal From owner-freebsd-hackers@FreeBSD.ORG Thu Feb 2 10:40:29 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F8D41065673 for ; Thu, 2 Feb 2012 10:40:29 +0000 (UTC) (envelope-from ray@ddteam.net) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id 45E648FC1C for ; Thu, 2 Feb 2012 10:40:29 +0000 (UTC) Received: from terran.dlink.ua (unknown [192.168.10.90]) (Authenticated sender: ray) by smtp.dlink.ua (Postfix) with ESMTPSA id DB446C493C for ; Thu, 2 Feb 2012 12:24:24 +0200 (EET) Date: Thu, 2 Feb 2012 12:25:03 +0200 From: Aleksandr Rybalko To: freebsd-hackers@freebsd.org Message-Id: <20120202122503.759c9e6e.ray@ddteam.net> Organization: DDTeam.net X-Mailer: Sylpheed 2.7.1 (GTK+ 2.20.1; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: dynamic attach of hinted devices X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2012 10:40:29 -0000 Hi FreeBSD hackers, +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ No opinions on arch@, so i will try at hackers@, since here is bigger audience :) +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ at first I want to say this: :) WARNING: FOLLOWING DEVCTL PATCH MAY EASILY PANIC YOUR SYSTEM, PLEASE DO NOT TRY IT ON PRODUCTION SERVERS AND TRY IT WITH FILESYSTEMS MOUNTED AS READONLY :))))) So I introduce two patches first one [1] used to migrate from static_hints or hints in the static_kenv to dynamic hints. sysctl kern.hintmode=2 will copy hints from static hints or from static kenv and put it into dynamic kenv. Those will allow to manipulate hints values and attach hinted devices with devctl tool. Second [2] allow attach/detach devices with userland tool devctl. devctl tool allow add and initialize new devices which is not able to be auto-enumerating, such a hinted devices. Both designed to have ability update EEPROM items in runtime, since some device can't work in mode when it accessible like a EEPROM chip. Example: # sysctl kern.hintmode=2 # kenv hint.mx25l.0.at="spibus0" # kenv hint.mx25l.0.cs=0 # kenv hint.mx25l.0.chipname="at25128" # devctl hinted spibus 0 mx25l 0 mx25l0: at cs 0 mode 0 on spibus0 mx25l0: at25128, sector 64 bytes, 256 sectors GEOM: new disk flash/spi0 Someone may found it also useful for testing device attach/detach code (memory leaks, resource allocation, etc). So, say me please your opinion. 1. http://my.ddteam.net/files/2012-01-31.to_dynamic_hints.patch 2. http://my.ddteam.net/files/2012-01-31.devctl2.patch WBW -- Aleksandr Rybalko From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 4 16:27:01 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B4531065670 for ; Sat, 4 Feb 2012 16:27:01 +0000 (UTC) (envelope-from monthadar@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id CB3608FC12 for ; Sat, 4 Feb 2012 16:27:00 +0000 (UTC) Received: by iaeo4 with SMTP id o4so9637862iae.13 for ; Sat, 04 Feb 2012 08:27:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=FMuqkpefO29NTl2IjoSHKbGLFRnjS3AoUcMFYAxGKEQ=; b=aeJ7L7F9Bn3T4Qrd2Bl+V8K/OFekRpfD2kPZrKUPlWBuylU45dUXNMliN+M2TYD/zu GFBfp/GqrheWIymHnhsN8Bs31BDikUB5KenLWm1H93aUu8wEI9GEQiyUsaBgCnQVa0vs AYPcrAm5wMuBP6ZDZBmG6E4EOzcn8K36emM8s= MIME-Version: 1.0 Received: by 10.50.208.41 with SMTP id mb9mr13548823igc.25.1328370988525; Sat, 04 Feb 2012 07:56:28 -0800 (PST) Received: by 10.50.213.74 with HTTP; Sat, 4 Feb 2012 07:56:28 -0800 (PST) In-Reply-To: <20120202122503.759c9e6e.ray@ddteam.net> References: <20120202122503.759c9e6e.ray@ddteam.net> Date: Sat, 4 Feb 2012 16:56:28 +0100 Message-ID: From: Monthadar Al Jaberi To: Aleksandr Rybalko Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org Subject: Re: dynamic attach of hinted devices X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Feb 2012 16:27:01 -0000 On Thu, Feb 2, 2012 at 11:25 AM, Aleksandr Rybalko wrote: > Hi FreeBSD hackers, > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > No opinions on arch@, so i will try at hackers@, since here is bigger > audience :) > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > at first I want to say this: :) > WARNING: FOLLOWING DEVCTL PATCH MAY EASILY PANIC YOUR SYSTEM, PLEASE > DO NOT TRY IT ON PRODUCTION SERVERS AND TRY IT WITH FILESYSTEMS MOUNTED > AS READONLY :))))) > > So I introduce two patches first one [1] used to migrate from > static_hints or hints in the static_kenv to dynamic hints. > > sysctl kern.hintmode=2 will copy hints from static hints or from static > kenv and put it into dynamic kenv. Those will allow to manipulate hints > values and attach hinted devices with devctl tool. > > Second [2] allow attach/detach devices with userland tool devctl. > > devctl tool allow add and initialize new devices which is not able to > be auto-enumerating, such a hinted devices. > > Both designed to have ability update EEPROM items in runtime, since > some device can't work in mode when it accessible like a EEPROM chip. > > Example: > # sysctl kern.hintmode=2 > # kenv hint.mx25l.0.at="spibus0" > # kenv hint.mx25l.0.cs=0 > # kenv hint.mx25l.0.chipname="at25128" > # devctl hinted spibus 0 mx25l 0 > mx25l0: at cs 0 mode 0 on spibus0 > mx25l0: at25128, sector 64 bytes, 256 sectors > GEOM: new disk flash/spi0 This is nice. I tried it on a generic device and it loaded with a hint flawlessly :) > > Someone may found it also useful for testing device attach/detach code > (memory leaks, resource allocation, etc). Couldnt it also be useful for redirecting data, for example attaching a flash memory to your own bus and then out through something else? Or it could also help in testing and verifying interaction between userspace and driver without having real hardware and not modifying hint file/recompile. > > So, say me please your opinion. Good work, its always a plus for development with dynamic tools! > > 1. http://my.ddteam.net/files/2012-01-31.to_dynamic_hints.patch > 2. http://my.ddteam.net/files/2012-01-31.devctl2.patch > > WBW > -- > Aleksandr Rybalko > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" -- Monthadar Al Jaberi From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 4 17:22:07 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07F6A106566B for ; Sat, 4 Feb 2012 17:22:07 +0000 (UTC) (envelope-from phk@freebsd.org) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 4D02C8FC08 for ; Sat, 4 Feb 2012 17:22:06 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 5BCB95DFA for ; Sat, 4 Feb 2012 17:05:46 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id q14H5jTR012193 for ; Sat, 4 Feb 2012 17:05:46 GMT (envelope-from phk@freebsd.org) To: hackers@freebsd.org From: Poul-Henning Kamp Content-Type: text/plain; charset=ISO-8859-1 Date: Sat, 04 Feb 2012 17:05:45 +0000 Message-ID: <12192.1328375145@critter.freebsd.dk> Cc: Subject: A dual-ISP hack with jail/vnet and ipfw X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Feb 2012 17:22:07 -0000 Natd(8) knows how to deal with multiple NAT instances for different interfaces, which is useful when you have multiple ISPs. The problem with it, is that it becomes incredibly hairy to configure your IPFW rules, in particular if you have other policy to implement too. I spent some quality time with a 9.0-Stable nanobsd image today, and the script below is my proof of concept of a simpler way to do that. The idea is to let a jail deal with the two ISPs and use an epair to deliver a "normal default route interface" to the rest of the firewall, making its configuration simpler and easier to understand. A discard interface is used to hold 127/8 and the default route in the jail. If the default route were to one of the ISP interfaces, and that if_ goes down, you loose the default route, and packets don't even reach ipfw(8) in the first place. Depending on how paranoid you are, you can run the natd(8) in an empty jail, (see last line). In a sense this is "DMZ-in-a-box", and there are a number of interesting ideas to explore, for instance running an openvpn instance in the jail, but put its TUN/TAP interface in the default (non-jail) vnet. The only disadvantage I have found yet, is that you see all packets with EPAIR_IN destination, instead of the physical destination IP, the source address however is OK. Another thing I noticed is that we should probably consider giving tcpdump/pcap in the unjailed part the ability to packet-dump interfaces in all vnets. This would match all the other the "semi-transparent" properties of jails. NB: This is only a proof of concept, you may want to think more about the IPFW rules before going live. Enjoy, Poul-Henning PS: feel free to adopt for any purpose you like, including doc/wiki/etc. #!/bin/sh set -x # ISP #1 VR2_IP=192.168.60.101 VR2_GW=192.168.60.1 # ISP #2 VR3_IP=10.0.0.1 VR3_GW=10.0.0.2 # IN/OUT ethernet pair EPAIR_OUT=192.168.5.2 EPAIR_IN=192.168.5.1 EPAIR_WID=/30 # Kill old jail jail -r ext > /dev/null 2>&1 || true jdir=/var/tmp/jail_ext rm -rf $jdir if true ; then mkdir $jdir ( cd / find \ libexec/ld-elf.so.1 \ sbin/ipfw \ sbin/natd \ sbin/dhclient \ sbin/ifconfig \ sbin/sysctl \ lib/libalias.so.7 \ lib/libbsdxml.so.4 \ lib/libjail.so.1 \ lib/libsbuf.so.6 \ lib/libipx.so.5 \ lib/libc.so.7 \ lib/libutil.so.9 \ lib/libalias_*.so \ etc/libalias.conf \ etc/services \ -print | cpio -dumpv $jdir ) else jdir=/ fi # Create new jail jail -c vnet name=ext path=$jdir persist F="jexec ext ipfw" $F -f flush $F add 1 deny ip from any to any # No filtering on the epair $F add 100 allow ip from any to any via epair0b # Dispatch to proper natd instance $F add 1200 skipto 22000 ip from any to any in via vr2 $F add 1300 skipto 23000 ip from any to any in via vr3 # The global instance, outgoing packets $F add 1400 divert 40000 ip from any to any $F add 1410 fwd $VR3_GW ip from $VR3_IP to any # Non-matched // TRAFIC POLICY // $F add 1420 skipto 12000 ip from any to any prob .5 $F add 1430 skipto 13000 ip from any to any # Outgoing vr2 $F add 12000 divert 20000 ip from any to any $F add 12100 fwd $VR2_GW ip from $VR2_IP to any $F add 12200 deny ip from any to any # Outgoing vr3 $F add 13000 divert 30000 ip from any to any $F add 13100 fwd $VR3_GW ip from $VR3_IP to any $F add 13200 deny ip from any to any # Incoming vr2 $F add 22000 divert 20000 ip from any to any $F add 22100 allow ip from any to any # Incoming vr3 $F add 23000 divert 30000 ip from any to any $F add 23100 allow ip from any to any # Set up a discard interface to hold the default route ifconfig disc0 destroy ifconfig disc0 create ifconfig disc0 vnet ext jexec ext ifconfig disc0 127.0.0.1/8 jexec ext route add default -iface disc0 # Create ethernet pair ifconfig epair0a destroy ifconfig epair0 create ifconfig epair0a ${EPAIR_IN}${EPAIR_WID} # Default route, (not quite default for my tests) route del -net 10/8 route add -net 10/8 ${EPAIR_OUT} # Move other end into jail ifconfig epair0b vnet ext # Move external interfaces to jail ifconfig vr2 vnet ext ifconfig vr3 vnet ext jexec ext ifconfig epair0b ${EPAIR_OUT}${EPAIR_WID} # Get addresses from you ISP's (DHCP/static) jexec ext dhclient -b vr2 jexec ext ifconfig vr3 10.0.0.1/30 # Enable forwarding in the jail jexec ext sysctl net.inet.ip.forwarding=1 # Build a natd.conf for the jail, allow inbound ssh ( echo deny_incoming echo globalport 40000 echo alias_address 127.0.0.1 echo echo instance vr2 echo port 20000 echo alias_address $VR2_IP echo redirect_port tcp ${EPAIR_IN}:22 ${VR2_IP}:22 echo echo instance vr3 echo port 30000 echo alias_address $VR3_IP echo redirect_port tcp ${EPAIR_IN}:22 ${VR3_IP}:22 ) > $jdir/etc/natd_ext.conf jexec ext natd -f /etc/natd_ext.conf # Remove the roadblock $F delete 1 # Remove the evidence # XXX: Even safer: put jail in md(4) disk, rm, remount r/o rm -rf $jdir/* -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.