From owner-freebsd-net@freebsd.org Sat Feb 6 08:07:13 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4AC7AA9F24D for ; Sat, 6 Feb 2016 08:07:13 +0000 (UTC) (envelope-from free@oneex.me) Received: from mail.oneex.me (mail.oneex.me [91.193.143.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 02830912 for ; Sat, 6 Feb 2016 08:07:11 +0000 (UTC) (envelope-from free@oneex.me) Received: from [192.168.0.110] (unknown [85.12.216.123]) by mail.oneex.me (Postfix) with ESMTPSA id 9A033C3F50; Sat, 6 Feb 2016 12:57:43 +0500 (YEKT) Authentication-Results: mail.oneex.me; dmarc=fail header.from=oneex.me Authentication-Results: mail.oneex.me; spf=pass smtp.mailfrom=free@oneex.me DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oneex.me; s=mail; t=1454745463; bh=B60Htbg7Y21jiZazMccq0lMRBBj+3sHCKTba/H3hZcM=; h=To:References:Subject:From:Cc:Date:In-Reply-To; b=DAIzTJ2KBTPXqimXTH9Oe8gttmjgQ4rT+GjzjXODmwHpyJyl86F1mJKNzKnjqyMdz eJpOjVnR+mVhedIZj1qljt82uzw1S3ZYRkyQyTXFDEXhhZSxFVlFrcOfGe5wz9C453 MG1Jrh1V8vH3xh9qvxupY/7f6rmdAJ5eIuuUwLuo= To: freebsd-net@freebsd.org References: Subject: Re: Problem with ipfw, in-kernel NAT and port redirection to jails From: Alexey Roslyakov Cc: wow@0x89.net Message-ID: <56B5A77B.2010108@oneex.me> Date: Sat, 6 Feb 2016 12:57:47 +0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Feb 2016 08:07:13 -0000 Hello. I have same problem when I'm trying redirect incoming traffic into the jailed web server. I repeated my installation few times on different releases - problem with redirected ports was here all time (except 9.3 - there was random result). As a temporary solution am using pf nat for redirect ports. My test configuration: /etc/rc.conf: ifconfig_vtnet0="inet 192.168.1.18/24" defaultrouter="192.168.1.1" cloned_interfaces="lo1" /etc/jail.conf: exec.start = "/bin/sh /etc/rc"; exec.stop = "/bin/sh /etc/rc.shutdown"; exec.clean; j1 { path = /home/jail1; mount.devfs; host.hostname = j1; interface = "lo1"; ip4.addr = 10.8.0.1; persist; } rc.firewall: ipfw nat 1 config if vtnet0 redirect_port tcp 10.8.0.1:80 80 ipfw add 500 nat 1 ip from any to 192.168.1.18 in via vtnet0 ipfw add 600 nat 1 ip from 10.8.0.1 to any out via vtnet0 ipfw add allow ip from any to any pf.conf: ext_if = "vtnet0" int_if = "lo1" jail_net = $int_if:network nat on $ext_if from $jail_net to any -> ($ext_if) rdr pass on $ext_if inet proto tcp from any to ($ext_if:0) port 80 -> 10.8.0.1 port 80 In jail I'm try nginx, apache24 and nc as source for redirection. Test file was generated: dd if/dev/random of=tmp.raw bs=1M count=2 On 10.1 and 10.2 there is no big differences, when using ipfw nat we can get only part of file (I'm using curl on different machine: curl http://192.168.1.18/tmp.raw > /dev/null): with nginx: Received = 33045 with apache: Received = 33092 with nc: Received = 16384 and result seems to be very stable in numbers. On 9.3: nginx: random bytes received, has no successful downloads apache: random bytes received, sometimes download entire file nc: entire file received My virtual environment is proxmox 3. Maybe it's related to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=137346 or just not properly configured ipfw nat?