From owner-freebsd-pf@freebsd.org Sun Nov 22 00:28:33 2015 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A75B2A33107 for ; Sun, 22 Nov 2015 00:28:33 +0000 (UTC) (envelope-from megha1991@gmail.com) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 737281D3B for ; Sun, 22 Nov 2015 00:28:33 +0000 (UTC) (envelope-from megha1991@gmail.com) Received: by igl9 with SMTP id 9so32548145igl.0 for ; Sat, 21 Nov 2015 16:28:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version; bh=ZV2PU7/ZuywkJ47wWjRmLCqM3HQRopM2Up5OPHp2ofQ=; b=l6KSmwmH6JDe603K7r75yUzqYJFz3u+FEcbcPOhoXWMPukyV4cb78Aah7ArYp2xR0M +xyEWh0abhtGn/n+FT114a+AMmVfzMh5WmGwOH5rUaxi0iWWfz6o34Hq5U2fAkCJQNT/ v4GYzslUHTdgwQ8SZ4To5r+VFJGy0qCa1xOdMVf27wbXqlkN7eht/hgQ0C0rPOvhPJ7Z hfnGHJO4MdpiaXKjGsMsAdfQWuV+qgd6xYF/rJuhnUyskzNx5X3rsYXr6Nv8ntQ/N5Sj t6VNxCCGpP6GpW6+3qNDanJ/thD0DcpCwMb1aBeAuV+rCbMyq0Euxin/A8vhhv/5qsW6 L8wA== X-Received: by 10.50.2.68 with SMTP id 4mr1274810igs.56.1448152112917; Sat, 21 Nov 2015 16:28:32 -0800 (PST) Received: from [192.168.1.9] (c-68-37-7-91.hsd1.mi.comcast.net. [68.37.7.91]) by smtp.gmail.com with ESMTPSA id 79sm2629811ioh.19.2015.11.21.16.28.31 for (version=TLSv1/SSLv3 cipher=OTHER); Sat, 21 Nov 2015 16:28:32 -0800 (PST) From: Megha Garg Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Audio Transcription - $0.50 Per Audio Minute Message-Id: <05FB8B6C-77BD-4122-BC3F-1692D3F87D8F@gmail.com> Date: Sat, 21 Nov 2015 19:28:30 -0500 To: freebsd-pf@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Nov 2015 00:28:33 -0000 Hi I wanted my audio transcribed confidentially. It is an interview. = What is the process for me to do this?= From owner-freebsd-pf@freebsd.org Sun Nov 22 19:15:01 2015 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED067A35808 for ; Sun, 22 Nov 2015 19:15:01 +0000 (UTC) (envelope-from kp@vega.codepro.be) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A6509144E for ; Sun, 22 Nov 2015 19:15:01 +0000 (UTC) (envelope-from kp@vega.codepro.be) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id A229A7E2E; Sun, 22 Nov 2015 20:14:58 +0100 (CET) Received: by vega.codepro.be (Postfix, from userid 1001) id 9748D7FD3B; Sun, 22 Nov 2015 20:14:58 +0100 (CET) Date: Sun, 22 Nov 2015 20:14:58 +0100 From: Kristof Provost To: =?utf-8?Q?Mi=C5=82osz?= Kaniewski Cc: freebsd-pf@freebsd.org Subject: Re: Creating span interface using 'dup-to' option Message-ID: <20151122191458.GD2307@vega.codepro.be> References: <20151108000315.GC2336@vega.codepro.be> <20151108192951.GD2336@vega.codepro.be> <20151115173349.GE13268@vega.codepro.be> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20151115173349.GE13268@vega.codepro.be> X-Checked-By-NSA: Probably User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Nov 2015 19:15:02 -0000 On 2015-11-15 18:33:49 (+0100), Kristof Provost wrote: > On the other hand, perhaps there's something we can do about the state > matching. The problems all start because we match state on the > duplicated packet. That's not correct, because the rule is set on e.g. > em0, but the duplicated packet is sent out on em1. > In fact, from a first reading of the code I don't actually understand > why we're getting that state match. > I've looked at the state matching for a bit. It turns out that by default packets will match state on any interface (specifically, the state is saved to the 'all' interface, rather than to the specific interface it was created on). That default can be changed with 'set state-policy if-bound'. I'd expect adding that would work around the problem you see. Regards, Kristof From owner-freebsd-pf@freebsd.org Wed Nov 25 10:36:08 2015 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3EDA4A36846 for ; Wed, 25 Nov 2015 10:36:08 +0000 (UTC) (envelope-from jdavidlists@gmail.com) Received: from mail-ig0-x22c.google.com (mail-ig0-x22c.google.com [IPv6:2607:f8b0:4001:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D7AB146B for ; Wed, 25 Nov 2015 10:36:08 +0000 (UTC) (envelope-from jdavidlists@gmail.com) Received: by igbxm8 with SMTP id xm8so34932827igb.1 for ; Wed, 25 Nov 2015 02:36:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=3kUgtPTIZHiuoqgB5CiONhX+6jx5GlW1dvZ3r8fEG88=; b=kLooaE/oRO9W5TUg7b3y/soVFNManGawsZlU7w0XDxWtnBfIua8G2T5fhJmzpTLGu7 wwHY5oeI9zcIWPjwLFhLOALuLV+CDvmN1wnBfQkhB6Ag7vACcPVOVpM3NJqLtSQZO5uW /eGk2dsCl+QtPgev+QMPNPvEzMzKnwTN3dO1VJyKjCpiF3YHLSX2zL5BfYViVlh37R+z jU6HjENR4FwZ+ZljvXTGfQMmDFZIk716P05NVmDL1a/ZumQXlCRpQQ5k67AdErDTQavW kPHa0YVtdxW2eYw2tsn8uN/HySnKp//tcwLxLEYLVa/jPOrmpc/NrG+lUKT2NKNjTVmW nXdg== MIME-Version: 1.0 X-Received: by 10.50.57.44 with SMTP id f12mr3149586igq.29.1448447767146; Wed, 25 Nov 2015 02:36:07 -0800 (PST) Sender: jdavidlists@gmail.com Received: by 10.36.91.10 with HTTP; Wed, 25 Nov 2015 02:36:07 -0800 (PST) Date: Wed, 25 Nov 2015 05:36:07 -0500 X-Google-Sender-Auth: U0ACQgqzNyhgbw8GXyY99ZbTjKo Message-ID: Subject: =?UTF-8?Q?PF_synproxy_state_doesn=E2=80=99t_negotiate_TCP_options_in?= =?UTF-8?Q?_10=2E2?= From: J David To: freebsd-pf@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Nov 2015 10:36:08 -0000 It appears that =E2=80=9Csynproxy state=E2=80=9D rules cause TCPs connectio= n to be negotiated without any options except MSS. Notably, it prevents the connection from using window scaling and selective-ACK, which can seriously limit throughput in many cases. Here=E2=80=99s a tcpdump of SYN and SYN ACK for a client and server with a synproxy state rule for HTTP on the server side: 10:15:04.038953 IP client.49266 > server.80: Flags [S], seq 2525361772, win 65535, options [mss 1460,nop,wscale 9,sackOK,TS val 730329818 ecr 0], length 0 10:15:04.111492 IP server.80 > client.49266: Flags [S.], seq 67243122, ack 2525361773, win 0, options [mss 1460], length 0 Here is the same tcpdump with the =E2=80=9Csynproxy state=E2=80=9D rule cha= nged to =E2=80=9Ckeep state:=E2=80=9D 10:30:51.169885 IP client.38552 > server.80: Flags [S], seq 719298516, win 65535, options [mss 1460,nop,wscale 9,sackOK,TS val 731276949 ecr 0], length 0 10:30:51.241967 IP server.80 > client.38552: Flags [S.], seq 3305307963, ack 719298517, win 65535, options [mss 1460,nop,wscale 9,sackOK,TS val 1216301733 ecr 731276949], length 0 In the test case these tcpdumps are taken from, the throughput difference between the two otherwise-identical rules is a factor of 10x. Is this behavior intentional? If so, perhaps it should be mentioned on the man page? If not, should we open a bug report on this? Thanks! From owner-freebsd-pf@freebsd.org Wed Nov 25 10:52:22 2015 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F81DA36C2B for ; Wed, 25 Nov 2015 10:52:22 +0000 (UTC) (envelope-from kp@vega.codepro.be) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED5261B92 for ; Wed, 25 Nov 2015 10:52:21 +0000 (UTC) (envelope-from kp@vega.codepro.be) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id 6A596D7E9; Wed, 25 Nov 2015 11:52:18 +0100 (CET) Received: by vega.codepro.be (Postfix, from userid 1001) id 64D421AA45; Wed, 25 Nov 2015 11:52:18 +0100 (CET) Date: Wed, 25 Nov 2015 11:52:18 +0100 From: Kristof Provost To: J David Cc: freebsd-pf@freebsd.org Subject: Re: PF synproxy state =?utf-8?Q?doesn?= =?utf-8?B?4oCZdA==?= negotiate TCP options in 10.2 Message-ID: <20151125105218.GA2469@vega.codepro.be> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Checked-By-NSA: Probably User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Nov 2015 10:52:22 -0000 On 2015-11-25 05:36:07 (-0500), J David wrote: > It appears that “synproxy state” rules cause TCPs connection to be > negotiated without any options except MSS. > ... > Is this behavior intentional? If so, perhaps it should be mentioned > on the man page? If not, should we open a bug report on this? > It's 'intentional' in the sense that it's simply not implemented in pf. In the synproxy case pf generates the TCP packet from scratch. All that's implemented there is the MSS option. I suspect nothing more is implemented because of the complexity. Using synproxy means there's no communication with the 'real' server until the connection is (from the outside perspective) established, so pf can't really know what values to negotiate. You're right that it'd be good to document this in the man page though. Regards, Kristof From owner-freebsd-pf@freebsd.org Thu Nov 26 21:11:02 2015 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC5FBA3A177 for ; Thu, 26 Nov 2015 21:11:01 +0000 (UTC) (envelope-from milosz.kaniewski@gmail.com) Received: from mail-vk0-x22a.google.com (mail-vk0-x22a.google.com [IPv6:2607:f8b0:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 904041E18; Thu, 26 Nov 2015 21:11:01 +0000 (UTC) (envelope-from milosz.kaniewski@gmail.com) Received: by vkha189 with SMTP id a189so58901997vkh.2; Thu, 26 Nov 2015 13:11:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=B4pq2A4O2W0cQVsl3TrBEYs2u29Fp+tftjHyPp0c8KI=; b=uAQnEcUizAd1KNg2XO3FoPri45dtCAj7gvkUtsThJpfsq/oMBRMLvMFz1W0F8LiQXP 8oW7FfsTznRIqstNLBWvVZguTie0GioLkkeTxJWLqg+U4BEADTMxQU7No0/q74W+a5nJ 2/Zp+AK5DZQW8hLj+Gw9wlni/Eqa59fqNR3PNTQw66u8CnUa6t3AV17LJSI/DWWJCPpr 8i7aGTMt+AQJvIESHLZyz5lXdG422jHedPIDJpBirLNXnXjEnNvU3QC8zBK70zTBwePR +V+2BOCH37Gt38r/K4DdGuFsZQzQzjs0IMm/XnrQFRCRCHmqixKPudoP1xVPKZC9cB7V 4OTg== MIME-Version: 1.0 X-Received: by 10.31.47.204 with SMTP id v195mr41798783vkv.119.1448572260294; Thu, 26 Nov 2015 13:11:00 -0800 (PST) Received: by 10.31.64.67 with HTTP; Thu, 26 Nov 2015 13:11:00 -0800 (PST) In-Reply-To: <20151122191458.GD2307@vega.codepro.be> References: <20151108000315.GC2336@vega.codepro.be> <20151108192951.GD2336@vega.codepro.be> <20151115173349.GE13268@vega.codepro.be> <20151122191458.GD2307@vega.codepro.be> Date: Thu, 26 Nov 2015 22:11:00 +0100 Message-ID: Subject: Re: Creating span interface using 'dup-to' option From: =?UTF-8?Q?Mi=C5=82osz_Kaniewski?= To: Kristof Provost Cc: freebsd-pf@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Nov 2015 21:11:02 -0000 2015-11-22 20:14 GMT+01:00 Kristof Provost : > On 2015-11-15 18:33:49 (+0100), Kristof Provost wrote: > > On the other hand, perhaps there's something we can do about the state > > matching. The problems all start because we match state on the > > duplicated packet. That's not correct, because the rule is set on e.g. > > em0, but the duplicated packet is sent out on em1. > > In fact, from a first reading of the code I don't actually understand > > why we're getting that state match. > > > I've looked at the state matching for a bit. It turns out that by > default packets will match state on any interface (specifically, the > state is saved to the 'all' interface, rather than to the specific > interface it was created on). > That default can be changed with 'set state-policy if-bound'. I'd expect > adding that would work around the problem you see. > Thanks, it did the trick :) I made couple of tests and my dup-to options started to duplicate packets in a right way when I set 'if-bound' policy. I didn't know that it is possible to control packets states policy. At beginning I was surprised that default behaviour is to make states floating between interfaces. But now I think that it can have sense. For example in my case I use pf to forward hundreds of thousands of connections. If I would use floating state policy then I would have as many states as connections. But if I switch to 'if-bound' policy then I would get two times more states than connection= s (one state for original interface and one for interface where packets are duplicated with dup-to). So i think that this workaround is very useful and in many cases it would b= e sufficient. But there are some scenarios, like mine, where floating states could provide big profits and it would be really nice if We could use them. Thank you very much for your help with my problem. Best regards, Mi=C5=82osz