From owner-freebsd-jail@freebsd.org Tue Feb 6 09:15:29 2018 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47D8DEE6729 for ; Tue, 6 Feb 2018 09:15:29 +0000 (UTC) (envelope-from artemrts@ukr.net) Received: from frv191.fwdcdn.com (frv191.fwdcdn.com [212.42.77.191]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D9FE87EFBA for ; Tue, 6 Feb 2018 09:15:28 +0000 (UTC) (envelope-from artemrts@ukr.net) Received: from frv198.fwdcdn.com ([212.42.77.198]) by frv191.fwdcdn.com with esmtp ID 1eiyxy-0004vK-W7 for freebsd-jail@freebsd.org; Tue, 06 Feb 2018 10:51:02 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id:To: Subject:From:Date:Sender:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=DmVFa6xOHlrqUxq5XpcEVQcYzI6OmxuZCi/aRV8fFDg=; b=dmI3yW4/EBj3cl8bwr5YbPhSGE Snorxr4mW/q/dNLulNP/qXjqzvzJvsZ2fV1/sixjbvhVCNbqx+Z9salgjxFaaGtmFzooeAEbyCFT2 +XbNtghcOGc7E2uEmllZdbVENsQWrBoFIYDpuNDsPs6cwPwm79QH33rofZhfjQISdvms=; Received: from [10.10.10.52] (helo=frv52.fwdcdn.com) by frv198.fwdcdn.com with smtp ID 1eiyxr-0004cm-1o for freebsd-jail@freebsd.org; Tue, 06 Feb 2018 10:50:55 +0200 Date: Tue, 06 Feb 2018 10:50:54 +0200 From: wishmaster Subject: Jail and RACCT To: freebsd-jail@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1517906631.504491795.gkoqnk26@frv52.fwdcdn.com> Received: from artemrts@ukr.net by frv52.fwdcdn.com; Tue, 06 Feb 2018 10:50:54 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Feb 2018 09:15:29 -0000 Hi, with help of racct I can limit CPU per jail. This rules works as expected for individual jails jail:jcctv:pcpu:deny=190 jail:jwww:pcpu:deny=190 jail:jphp:pcpu:deny=190 jail:jdb:pcpu:deny=190 But I need pcpu=190 for all jails. Is it possible? jail:*:pcpu:deny=190 - does not work for me -- Vitaly From owner-freebsd-jail@freebsd.org Tue Feb 6 17:00:36 2018 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5476DEE8FD0 for ; Tue, 6 Feb 2018 17:00:36 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB6A0739F0 for ; Tue, 6 Feb 2018 17:00:35 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from moby.local ([79.107.63.13]) by mail.gmx.com (mrgmx103 [212.227.17.174]) with ESMTPSA (Nemesis) id 0MarAM-1ePmH50z9e-00KPd7; Tue, 06 Feb 2018 18:00:33 +0100 Subject: Re: Jail and RACCT To: wishmaster , freebsd-jail@freebsd.org References: <1517906631.504491795.gkoqnk26@frv52.fwdcdn.com> From: Nikos Vassiliadis Message-ID: Date: Tue, 6 Feb 2018 19:00:32 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <1517906631.504491795.gkoqnk26@frv52.fwdcdn.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:cRqfn9Md1odd1H1jfdHWUDawbC3SQOdwx/NGjx0q6piLs+Fg3vp ApcfCtuK4QRvhwXzjikuWvNagZKvVTWFzawbaXuGuURGmeZblBnCz0SA46nD2WCeBzerog1 pBhJJpL0CecjcFTVX1G3kPjVkXSP9yYMMrUP7qte83fDNYTZG0FK/uuVmpDPe0mOxYUR+Gd bM6/NTSADe/xdNQnrBYlg== X-UI-Out-Filterresults: notjunk:1;V01:K0:emUitYZiEaE=:8+BffYL9u4+UxlQtqq0IqH ux3WjBOKQYe3i7W3LTqHf86gKzfrRDOA+VrFehlfu0I/H5ZfvUm0wQrBhZP1NS7niX1TC2pzI QkCaw4foswbo5Cb8eK7gL3JQvI0gvhqj1jjiMFJh1xU8gMzR6mgNzyRjJj8MLkz90zaN+qlQQ 9kHUFLSwHVRWXymMEknCgMsxNtkpk1VgiN9MnySXbRT5/RHNFBB1xL57KHUYCo8pZ6ulnrnSz DIv9hya+S2AcOURiQtMtGkJuVf/ifyYcVn4BbcrjWzUUcpBzXLPBkujwgobbU3vbetfqmKWYQ slc03FhsTxrKOdcph7eO8RtPf7rPa8uKa7YzZcT7lMU4JhDwcE+zcdVAomp8o5wdevem9CRm9 v/BeotUttAaeON+zh97IXsKEnKAIFGOj565PtQyNWgd5oL2YYpbocmlt+378dM4A2mZSQJZaz Vx62/4/7VQw0QtXSBJu12IhXW4EaiWqGdZZZ9SYwl78UN01JU3UAvfQuoxkcm2O1ygoqJrvdB 6F/spHzoxz+rlxrJNEtusvXzwqt/PUoI1ZruDBsut4mPhswjr86xUPbvOIKePCxC8v3lZveHG scp6fAL/fzix0Z3LQ1EySf4lniCzNH58Cv3WvEdprOkSMRGCUlBitVDd8/2jwAXwRhU8jYXWb ZqogVwC+73lPKMyVRGtAOh9JNdp6qjAi4HKIEsxXBPTNVjxsHAaNNcfDjM+RqsFmpMD854aYk 1UbB4tI6867/wRFCSnJnrdxM3cBrmJpu/C4g4a+IStrBqc2ZO0WUrxRnFrQz86CL355O2vHAA sHIeuPZosYYVLoWNmnsvnpBWZ4vz8t8Ku+4iv7oevhSa0ufO3g= X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Feb 2018 17:00:36 -0000 On 02/06/18 10:50, wishmaster wrote: > Hi, > > with help of racct I can limit CPU per jail. > > This rules works as expected for individual jails > > jail:jcctv:pcpu:deny=190 > jail:jwww:pcpu:deny=190 > jail:jphp:pcpu:deny=190 > jail:jdb:pcpu:deny=190 > > But I need pcpu=190 for all jails. Is it possible? > > jail:*:pcpu:deny=190 - does not work for me Hi, just thought, if you add a parent jail and apply the rule to it? From owner-freebsd-jail@freebsd.org Thu Feb 8 08:40:05 2018 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC434F0F492 for ; Thu, 8 Feb 2018 08:40:05 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1877C7B125 for ; Thu, 8 Feb 2018 08:40:04 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from freyja.zeit4.iv.bundesimmobilien.de ([87.138.105.249]) by mail.gmx.com (mrgmx102 [212.227.17.168]) with ESMTPSA (Nemesis) id 0LbPza-1f8cNQ0h46-00kzwf for ; Thu, 08 Feb 2018 09:39:58 +0100 Date: Thu, 8 Feb 2018 09:39:57 +0100 From: "O. Hartmann" To: freebsd-jail@freebsd.org Subject: VIMAGE: vnet, epair and lots of jails on bridgeX - routing Message-ID: <20180208093957.28379746@freyja.zeit4.iv.bundesimmobilien.de> Organization: Walstatt MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:5yYi0Ajzoxz6An+MAUdGhSQyf1LbpUypiLRxac4Y9lcZ8Wf8p/z jmQT/k+6zTI0RQe1/yyCjJ7r1Fcm8a1pf6+GX2mGIVi+7ixG76cMHa5EFeLlPvpTMQwH4Of YTscUKtEKp7dWfS6od1q411D/O1JI86FkwoFmCMCn/bgIdWaKGBsC7RTyM0emTBDssdN/HR b3XXYpNNdnHYIySDd0+QA== X-UI-Out-Filterresults: notjunk:1;V01:K0:Kc6qQTfEUFU=:hxr/yZ/+6Q9XKLCltFs7MI ieymOHQ7B3rhBUqQpBMbVm7qYJ5UlodgpIP2DPwXmDngjnHxLiQQcZL6NzIjEsnlTlS7A8xXU Jy9budGFgwTzQ4Q9FiM8smu0KVp3GCjLZ3+BO/4VyBtsN22tkI8QERYyRX5cMunAuy9dT9PJ6 TuMTjXW+qPf/p+33BKEz0CtlMVDyY72ZKJk2BM6gsB0TIJ2CAWunPTvf7sWz0fDVrEzTn+FZb dlBlMzzyVDoPRXOcMqcp8jdLUaNZdCc+IexYkm5WM46phGqLbx33AzY+VBXss0RkeO5TOWKBD HHH/T0ZElkGAxfkt9dp+BVlQ3NEEItrj0HwZsEkSlFG22ekDx5eCxuZ2An/KItKWu/pB+RSjX UhR9Y0/iwtWiAtNf0Tfpgi01wBIh9C5PjqUNL9JwJ9i5TBuOXy1QRwGcPlKGJMK0PDkYaSwws legwcpcXHltv4xZ/RxRxORq9IjWeMdDHwlsxT6zGdl7y2OAMJ3DeiDRVKYiGVPafpGvcXH4Wa GaZf9gO2PKi1711G1+i96txWtf7lTQbjBuBdaY7rlR7r4TtS0Xfuo/J30PoF18PlKzXgijf84 i4nMTZ4/8OFIDjWScQToMTKciGO0grcypJ8o39tFJwjcqAvhlC8JJxylutLmC676VeeznpJnk N8Bs32tqgFq2rE+S3RRUjh/VyCJr/qnOqGtsdI/MLjdsUprcuvQ3hMB4FvqNPgtcjkAzoYvjU QnQ65+0G1A+n5An4gpZ6GURnK1EOjyC/50S8vm/4mNGhQEuOxKhJUjzmzimQ/2gx72gzOG2NG 6iZzaw01poa78f475Mw1PrTLRuJdwhQdjtLSNFbYZIWDzDl/aU= X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Feb 2018 08:40:05 -0000 Hello, I fight with the following problem without any kind of success and I need some help and/or advice. We are running several CURRENT and 11.1-RELENG-p6 boxes. CURRENT is at the most recent version as of today. VIMAGE is compiled in into all kernels. IPFW is compiled into all kernels and is the one and only firewall used. On CURRENT, the host's ipfw is set to "OPEN" (using the rc.-scripts so far). By convention, I address the host running the kernel by "host". Every jail is created/configured with its own "vnet" cloned network stack (vnet=new). All hosts do have at least three physical NICs. The host itself is supposed to be member of the "friendly" network via a dedicated NIC. The two remaining NICs are split into fractions belonging to an "hostile" network on which I'd like to place exposed jails (for now), and to the "friendly" network, on which also jails will be hosted, but via a dedicated NIC. Inbetween those two networks, the host will have a third, intermediate, network, call it the "service" network. The following will be true for ALL jails created, including the host itself: net.link.bridge.pfil_member=0 net.link.bridge.pfil_bridge=0 net.link.bridge.pfil_onlyip=0 First, I clone/create three bridge(4) devices, bridge0 (considered to be the "glue" between the "service" jails), bridge1 (considered to be the glue between the jails on the friednly network side) and bridge2, which is the glue between the jails on the hostile side. bridge1 has eth1 as a member, which provides the physical access to the friendly network, eth2 is member of bridge2, which provides access to the hostile network. By convention, when creating epair(4), the a-portion belongs to the jail itself and is assigned with an IPv6 address. The b-portion of the epair(4) is member of its bridge according to its realm (friendly, service or hostile network). Additionally, there is a special jail, the router, which has three epair(4) devices, the b-portion of the epair is member of the appropriate bridge(4) and this router jail has static routes assigned, pointing to the appropriate epairXXXa that is suppoesd to be the link into the correct bridge/network. IPFW is set to open on this jail (for now). On this special jail it is set: net.inet.ip.forwarding=1. I hope, the topology is clear so far. All epairs or epair endpoints as well as the bridges are UP! Double checked this. Jails on bridge0 (service net) have IPs in the range 10.10.0.0/24, the b-portion of the routing jail's epair is member of bridge0, as described above, and the a-portion of the epair has IP 10.10.0.1. Default route on each jeail on bridge0 is set to 10.10.0.1 accordingly. Consider a similar setup on the other jails on the friendly and hostile network, except the fact that their bridges do have a physical NIC to which they may have access to a real network. The setup might not be ideal and/or applicable for the purpose of separartion of networks virtually, but that shouldn't be the subject here. More important is that I assume that I haven't understood some essentials, because the setup doens't work as expected. Furthermore, it behaves on FreeBSD 11.1-RELENG-p6 sometimes completely unpredictable - but in that special case, I think I ran IPFW on the host as "WORKSTATION" and dynamic rules may play an important role here. But focussing on the CURRENT box, the host's IPFW is set to OPEN. With jexec -l hostA I gain access to host A on the "service" bridge0 and I want to ping its neighbour, hostB, on the same bridge and in the same net. It doesn't work! From the routing jail, I CAN NOT ping any host on bridge0. The routing jail has these network settings: [... routing jail ...] lo0: flags=8049 metric 0 mtu 16384 options=600003 inet 127.0.0.1 netmask 0xff000000 groups: lo [epair to bridge0 - service net] epair4000a: flags=8843 metric 0 mtu 1500 options=8 ether 02:57:d0:00:07:0a inet 10.10.0.1 netmask 0xffffff00 broadcast 10.10.0.255 media: Ethernet 10Gbase-T (10Gbase-T ) status: active groups: epair [epair to bridge1, friendly net] epair4001a: flags=8843 metric 0 mtu 1500 options=8 ether 02:57:d0:00:09:0a inet 192.168.11.1 netmask 0xffffff00 broadcast 192.168.11.255 media: Ethernet 10Gbase-T (10Gbase-T ) status: active groups: epair [epair to bridge2, hostile net] epair4002a: flags=8843 metric 0 mtu 1500 options=8 ether 02:57:d0:00:0b:0a inet 10.10.10.1 netmask 0xfffffc00 broadcast 10.10.10.255 media: Ethernet 10Gbase-T (10Gbase-T ) status: active groups: epair routing: netstat -Warn Routing tables Internet: Destination Gateway Flags Use Mtu Netif Expire 10.10.0.0/24 link#2 U 11 1500 epair4000a 10.10.0.1 link#2 UHS 4 16384 lo0 10.10.10.0/24 link#4 U 210 1500 epair4002a 10.10.10.1 link#4 UHS 44 16384 lo0 127.0.0.1 link#1 UH 0 16384 lo0 192.168.11.0/24 link#3 U 9 1500 epair4001a 192.168.11.1 link#3 UHS 0 16384 lo0 Consider a jail hostCC on bridge2 in the hostile network, IP 10.10.10.128. I can ping that jail, although it has conceptionally the very same setup as the unreachable jails on bridge0! It is weird. On bridge0, no jail can be pinged, it looks like the ethernet is somehwo down on that bridge. I would expect to ping each host member of the very same bridge! On 11.1-RELENG-p6, there are other weird issues, I was able to ping those jails, even ssh to them, but that vanished after several restarts of the jails system (each bridge, epair is created by jail.conf and destroyed after the jails has been deactivated and doing so a considerable amount brings down the FreeBSD 11.1-RELENG-p6 host verys successfully - it crashes!). So, since VIMAGE is now default in CURRENT's GENERIC, I consider its functionality at least "predictable", but I fail somehow here. Does someone have a deeper insight or realise the mistake I'm celebrating here? Thanks in adavnce, Oliver From owner-freebsd-jail@freebsd.org Sat Feb 10 07:53:05 2018 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 175BBF04D3D; Sat, 10 Feb 2018 07:53:05 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E8B271833; Sat, 10 Feb 2018 07:53:04 +0000 (UTC) (envelope-from ohartmann@walstatt.org) Received: from thor.intern.walstatt.dynvpn.de ([77.179.129.41]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0La3c1-1f7cmA1LDx-00lpQC; Sat, 10 Feb 2018 08:52:55 +0100 Date: Sat, 10 Feb 2018 08:52:21 +0100 From: "O. Hartmann" To: "Bjoern A. Zeeb" Cc: "O. Hartmann" , freebsd-jail@freebsd.org, freebsd-current Subject: Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing Message-ID: <20180210085248.7b9af104@thor.intern.walstatt.dynvpn.de> In-Reply-To: <2D57FE3A-744A-4A44-B572-5338AB9E187D@lists.zabbadoz.net> References: <20180208093052.7f5d7a98@freyja.zeit4.iv.bundesimmobilien.de> <20180209172259.1ec9b9f4@thor.intern.walstatt.dynvpn.de> <2D57FE3A-744A-4A44-B572-5338AB9E187D@lists.zabbadoz.net> Organization: WALSTATT User-Agent: OutScare 3.1415926 X-Operating-System: ImNotAnOperatingSystem 3.141592527 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; boundary="Sig_/iZ3C2Do29sYhN8rpa4JmBL/"; protocol="application/pgp-signature" X-Provags-ID: V03:K0:vEgHj1sxHwePDeyJjnZM50fzClp8+HZzt122w1gU1aNx8FJy2Pz FecjtkqUzU5MS/RnuMJzXrcjbzSLj09mbcHPTk9ZgRVM8L5kaq5aqXq1+aqSXp5y2F8/MYh yK/xpUC1MgZfu7dE9244gjvxbTABW2iz6gl+5bgb99GBXLiuHkTov4oSqas43GRkQxJYzxM EC9Qt/Ss08/3kY/h9g0Jg== X-UI-Out-Filterresults: notjunk:1;V01:K0:MY4N0F7pXmk=:ufCY0ULHzRRMTXgrKOi+i8 sXAYSdia46wkS3hLw9yvhVGea/oQTpx+opja5VwuUbURsKFXHoTYUvhxeYOczZAz/QHjFmga/ 4kiJkMvF0QXRik6pf+cZS1NvvXHmj3QPupoWVDRp31bBTRbCusze9DzfKutoHBomsQNc71UGa xxIt72DVoamCPCXbKoFSbSZd/OCDgQPFgvA9wGRiTMujLpk3t5oJVsRbBgZj/g5mLdzVYTRUj gB3LM2WIm4ryiI1RBbgvhVBii+17T2tVgf4Vu3FUgh8da48kS0RmpKCDC9yyyEe8QaxeknY+E +Zf2BvW3VO7mFSI6TIfASKMVtLohxVHVXh5f6Q/KXjjX6XfXE65spV3TDaJxeHOq8U0Pe614U VP8njaYf4LjLdaHZVFL8fKaPcW/FsDCv2j0ZyupO50+R1Z9C9L1aIRpTm7HuFwSVH0wiCzOsc 3F/5I+Fxm/Tsq6R3r9dZH8UVnLP1pU2ykFe9U0Lu8h9KRBFp63xQU9BEcgZkpnEVgao1ydNuI jrOI/OyzhaPW7rbpErSPxQO5x5erBq0iTYnrEcMOqATLRuwiBFCk6Y3RQLAup/0gHOQpXydwn NXNBJZAFe7WhXej6aI05KIW57TMvknktx0J5oIWbTiOIXIxfNxHi2nmROrmwyPUspGwlV1P3F 9HdgPZqyMaw8GI+VvKOUJXc0wzZkO2F6SJOHYcjIz/x2jP6uhwC7b2Smyn6HqC/KGAGDo0cwq sR/ZrMCdAUI+MmsThECUDoLFKUyzbhw7zHhr2RCUKY/f9rgQHJ/rW+LzReOXsivntmIgPb5g0 Rxfem9Ykn6bIQBUtIDYYt29TyZZ5w== X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Feb 2018 07:53:05 -0000 --Sig_/iZ3C2Do29sYhN8rpa4JmBL/ Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Am Fri, 09 Feb 2018 16:43:17 +0000 "Bjoern A. Zeeb" schrieb: > On 9 Feb 2018, at 16:22, O. Hartmann wrote: >=20 > > Am Thu, 8 Feb 2018 09:31:15 +0100 > > "O. Hartmann" schrieb: > > > > Is this problem to trivial? =20 >=20 > I read through it yesterday and found myself in the position that I need= =20 > a whiteboard or paper and pencil or an ASCII art of your situation. But= =20 > by the time I made it to the question I was basically lost. Could you=20 > massively simplify this and maybe produce the ASCII art? >=20 > /bz > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" All right. I'm not much of an artist and at this very moment, I haven't much experienc= e with neat ASCII art tools. But I'll provide a sketch later, but I also will simplify = the situation. Consider three "vswitches", basically based on the creation of bridges, bri= dge0, bridge1, bridge2. Create at least three individual vnet-jails attached to each vbrid= ge. Those jails have epair pseudo devices. The jail itself owns the "a-part" of the e= pair and the b-part is "member of the bridge". Each jail's epairXXXa has an IP assigned = of the network the vswitch is part of. I mention a- and b-part of the epair here, because = I thought it could matter, but I think for symmetry reasons it doesn't. Now consider a further, special jail. This jail is supposed to have three e= pair devices, each one is reaching into one of the vbridges. This jail is the router/rout= ing jail. Later, this jail should filter via IPFW the traffic between the three vbrid= ges according to rules, but this doesn't matter here, beacuase the basics are not working= as expected. Now the problems. It doesn't matter on which jail of the three vswitches I = login, the moment a vbridge has more than two member epairs (one is alway member of t= he routing jail, now consider a database jail and a webserver jail), pinging each jail= or the routing jail fails. It works sometimes for a couple of ICMP packets and the= n stops. If each vbridge has only one member jail, I have NO PROBLEMS traversing acc= ordingly to the static routing rules from one vbridge to any other, say from vbridge1 t= o vbridge0 or vbridge2 and any permutation of that. The moment any of the bridges gets an additional member epair interface (so= the bridge has at least three members including the on reaching into the virtual route= r jail) the vbridge seems to operate unpredictable (to me). Pinging jails memeber of th= at vbridge are unreachable. Technical information: The kernel has options IPFIREWALL, VIMAGE. The host's ipfw (kernel) decline= s packets by default. Each jail is configured to have ipfw "open". Thanks for the patience. Kind regards, O. Hartmann --Sig_/iZ3C2Do29sYhN8rpa4JmBL/ Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCWn6k0AAKCRDS528fyFhY lNE+AgCbqIMTxE2O3ejPWmxVBxfd3Kh5NSZ+NPpHkuJ7Gh/U6yuZLbJWsbgpccGR degqacPcwWakbJAnqdQ9uXurJXSnAf9e76H89cTGqs9KCrWTKrUWUrH5fKFcLhO/ dN47cv6ZUn7xKCcqeudC2NKQA1C18DG+W6DqD22LL50xLIGHzDlm =ZIqA -----END PGP SIGNATURE----- --Sig_/iZ3C2Do29sYhN8rpa4JmBL/-- From owner-freebsd-jail@freebsd.org Sat Feb 10 08:55:57 2018 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7573CF09084; Sat, 10 Feb 2018 08:55:57 +0000 (UTC) (envelope-from zec@fer.hr) Received: from mail.fer.hr (mail.fer.hr [161.53.72.233]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.fer.hr", Issuer "TERENA SSL CA 3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9DDE73C32; Sat, 10 Feb 2018 08:55:56 +0000 (UTC) (envelope-from zec@fer.hr) Received: from x23 (31.147.122.18) by MAIL.fer.hr (161.53.72.233) with Microsoft SMTP Server (TLS) id 14.3.361.1; Sat, 10 Feb 2018 09:54:43 +0100 Date: Sat, 10 Feb 2018 09:54:49 +0100 From: Marko Zec To: "O. Hartmann" CC: "Bjoern A. Zeeb" , , freebsd-current Subject: Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing Message-ID: <20180210095449.3117e6d9@x23> In-Reply-To: <20180210085248.7b9af104@thor.intern.walstatt.dynvpn.de> References: <20180208093052.7f5d7a98@freyja.zeit4.iv.bundesimmobilien.de> <20180209172259.1ec9b9f4@thor.intern.walstatt.dynvpn.de> <2D57FE3A-744A-4A44-B572-5338AB9E187D@lists.zabbadoz.net> <20180210085248.7b9af104@thor.intern.walstatt.dynvpn.de> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.31; amd64-portbld-freebsd11.1) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [31.147.122.18] X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Feb 2018 08:55:57 -0000 On Sat, 10 Feb 2018 08:52:21 +0100 "O. Hartmann" wrote: > Am Fri, 09 Feb 2018 16:43:17 +0000 > "Bjoern A. Zeeb" schrieb: > > > On 9 Feb 2018, at 16:22, O. Hartmann wrote: > > > > > Am Thu, 8 Feb 2018 09:31:15 +0100 > > > "O. Hartmann" schrieb: > > > > > > Is this problem to trivial? > > > > I read through it yesterday and found myself in the position that I > > need a whiteboard or paper and pencil or an ASCII art of your > > situation. But by the time I made it to the question I was > > basically lost. Could you massively simplify this and maybe > > produce the ASCII art? > > > > /bz > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to > > "freebsd-current-unsubscribe@freebsd.org" > > All right. > > I'm not much of an artist and at this very moment, I haven't much > experience with neat ASCII art tools. But I'll provide a sketch > later, but I also will simplify the situation. > > Consider three "vswitches", basically based on the creation of > bridges, bridge0, bridge1, bridge2. Create at least three individual > vnet-jails attached to each vbridge. Those jails have epair pseudo > devices. The jail itself owns the "a-part" of the epair and the > b-part is "member of the bridge". Each jail's epairXXXa has an IP > assigned of the network the vswitch is part of. I mention a- and > b-part of the epair here, because I thought it could matter, but I > think for symmetry reasons it doesn't. > > Now consider a further, special jail. This jail is supposed to have > three epair devices, each one is reaching into one of the vbridges. > This jail is the router/routing jail. Later, this jail should filter > via IPFW the traffic between the three vbridges according to rules, > but this doesn't matter here, beacuase the basics are not working as > expected. > > Now the problems. It doesn't matter on which jail of the three > vswitches I login, the moment a vbridge has more than two member > epairs (one is alway member of the routing jail, now consider a > database jail and a webserver jail), pinging each jail or the routing > jail fails. It works sometimes for a couple of ICMP packets and then > stops. > > If each vbridge has only one member jail, I have NO PROBLEMS > traversing accordingly to the static routing rules from one vbridge > to any other, say from vbridge1 to vbridge0 or vbridge2 and any > permutation of that. > > The moment any of the bridges gets an additional member epair > interface (so the bridge has at least three members including the on > reaching into the virtual router jail) the vbridge seems to operate > unpredictable (to me). Pinging jails memeber of that vbridge are > unreachable. > > Technical information: > > The kernel has options IPFIREWALL, VIMAGE. The host's ipfw (kernel) > declines packets by default. Each jail is configured to have ipfw > "open". > > Thanks for the patience. If you could provide a script which sets up the topology you described in two lengthy posts then others could reproduce this, and your chances of getting useful feedback would certainly increase. We also have a graphical tool (https://github.com/imunes/imunes) which can set up a topology like you described in a few clicks of a mouse, albeit using netgraph and ng_eiface instead of epairs, but I assume this is irellevant as long as you are not aiming for maximum packet throughputs. If you attempt to use this tool, note that selecting "stpswitch" will create if_bridge instances, whereas "lanswitch" creates ng_bridge instances. Good luck, Marko > > Kind regards, > > O. Hartmann From owner-freebsd-jail@freebsd.org Sat Feb 10 11:17:47 2018 Return-Path: Delivered-To: freebsd-jail@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5551F135CE; Sat, 10 Feb 2018 11:17:46 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-io0-f180.google.com (mail-io0-f180.google.com [209.85.223.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2AC1479C3A; Sat, 10 Feb 2018 11:17:45 +0000 (UTC) (envelope-from cochard@gmail.com) Received: by mail-io0-f180.google.com with SMTP id x188so5993322iod.1; Sat, 10 Feb 2018 03:17:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=iGOCw4brYEQX1TrCaApbFMjc8ZWo3Afcfcm44WwTXMc=; b=o3yZZRwSsEWxZZu50llE2WUltaIJme0I3/gFqdI279mBUkkhOo8d0rCgCeO/2/qOC/ uNoI/rbWMtQvYfpLSS5c3NKPQNc9ijnl5IyXmRr1cWk6f5y1fUiGMgjBtSMJBa0DQ8fw Ibhq7GtUHT2lVjAkbT4IcEgzORavG1Y82h8NYxS5nkKzQ63YThA1FhgeqevNgOSS8cW1 jQNry/T5nGMNYfpq4Qnm+9SbDQZMbF3qTptyIgZhLm8Htzpl2BAfKxeTl/5oQkcu2crc 1GQYZNo5T2wFdu7w0h7QH+Xt9aRpS+vpeaOTkfMgiAaUP0+NZZmEVAxxITlbO3y2e5j4 O6Xw== X-Gm-Message-State: APf1xPAcyQKmIEO6SrIUAZYaWl5MOaG1xiQUCHHvhW/zLelWNOKGtEHE 9DLvZ811ETsrtylS7ZLbHqtN4QIy X-Google-Smtp-Source: AH8x227CC8wEJDY/FBYIW6nDtrFqQxUu5Y3ntKX70lya7iumoeHlT8oeZz6Pku62Vgxe6PAt8FqrCg== X-Received: by 10.107.173.12 with SMTP id w12mr1020500ioe.282.1518259960180; Sat, 10 Feb 2018 02:52:40 -0800 (PST) Received: from mail-pf0-f182.google.com (mail-pf0-f182.google.com. [209.85.192.182]) by smtp.gmail.com with ESMTPSA id y81sm1817003ita.40.2018.02.10.02.52.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 10 Feb 2018 02:52:39 -0800 (PST) Received: by mail-pf0-f182.google.com with SMTP id p1so2312469pfh.4; Sat, 10 Feb 2018 02:52:39 -0800 (PST) X-Received: by 10.99.168.76 with SMTP id i12mr4798000pgp.420.1518259958879; Sat, 10 Feb 2018 02:52:38 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.179.149 with HTTP; Sat, 10 Feb 2018 02:52:18 -0800 (PST) In-Reply-To: <20180210085248.7b9af104@thor.intern.walstatt.dynvpn.de> References: <20180208093052.7f5d7a98@freyja.zeit4.iv.bundesimmobilien.de> <20180209172259.1ec9b9f4@thor.intern.walstatt.dynvpn.de> <2D57FE3A-744A-4A44-B572-5338AB9E187D@lists.zabbadoz.net> <20180210085248.7b9af104@thor.intern.walstatt.dynvpn.de> From: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= Date: Sat, 10 Feb 2018 11:52:18 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing To: "O. Hartmann" Cc: freebsd-jail@freebsd.org, freebsd-current Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Feb 2018 11:17:47 -0000 On Sat, Feb 10, 2018 at 8:52 AM, O. Hartmann wrote= : > > The moment any of the bridges gets an additional member epair interface > (so the bridge > has at least three members including the on reaching into the virtual > router jail) the > vbridge seems to operate unpredictable (to me). Pinging jails memeber of > that vbridge > are unreachable. > > =E2=80=8BFirst idea: Did you try with a more simple setup, like with just 3 jails members of one bridge ? =3D> I've tried it on a -head, and all 4 members (3 jails and the host) reach to communicate. Second idea: Can you check that all epairs have different MAC address each ?=E2=80=8B I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D176671 Regards, Olivier