From owner-freebsd-net@FreeBSD.ORG Sun Aug 18 21:02:04 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id ADAF32EA for ; Sun, 18 Aug 2013 21:02:04 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm20-vm3.bullet.mail.ne1.yahoo.com (nm20-vm3.bullet.mail.ne1.yahoo.com [98.138.91.150]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 441492CC7 for ; Sun, 18 Aug 2013 21:02:04 +0000 (UTC) Received: from [98.138.101.130] by nm20.bullet.mail.ne1.yahoo.com with NNFMP; 18 Aug 2013 21:01:57 -0000 Received: from [98.138.89.173] by tm18.bullet.mail.ne1.yahoo.com with NNFMP; 18 Aug 2013 21:01:57 -0000 Received: from [127.0.0.1] by omp1029.mail.ne1.yahoo.com with NNFMP; 18 Aug 2013 21:01:57 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 555768.76669.bm@omp1029.mail.ne1.yahoo.com Received: (qmail 26153 invoked by uid 60001); 18 Aug 2013 21:01:57 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1376859717; bh=4TTP0jtNmzj5ueyxcmHzEfQ8ABeQ77a8YX7EeLjyGjg=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=bdMEaPSQ1lsk7dVc7oJRgEtYuuO8+lLUqg+k65W8Dp2Ri6ru5aBklCQGIGqGn4VAkXhQZSXffII/aoDZIV9Tfp3LoFQUjqKjWPOI5kg1b9BHaJfIliHIs0+LMVbgtEXmsO05RATEiRosaQXu5H2mO2EBLmci/EzVoHVfDdiYNH0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=Clb0qAKxMd0yG+TP31Q7o03DrOkRuZZUmY0GbxwZdFwTa0/VJ52DMDGm9jBjyhtbz/Hd0x36DU90yQEvJdv/tYHjVAfAVuc3pG3GgQklEdUjKLH2q27loELBu6TAutshRZnvr9EDYgC1b1ecdzzW9bQEjXK77dZZcbVXyk9Yq2c=; X-YMail-OSG: cYbY2h4VM1lA_uPHrjFy4_Pj_VeKBy_rs_LxR.ksIfYxShj dNZpkcSzbe.jDLVPloLtJ253NTFrgweuPEHFKSEs_XTdRiT3z.zqkN4nj..d y.J7hGIcX5bvnWa2RVN8Er0ck9SK6SQMfGak7uPVd6JOkVw.7iGIr_bcPgMv f.m4_7SLeBnD6aE2b3N2PPRxnwcThSM6uHqJCxeB6wuwvoDg_qEKvdQfdbjw sZG3.C806cu8SP7O9YscxqXM5vdGCWCk81hjR1vM.1Nu3nzPhd8mKjk2_0PR JNOhaRviPBUcAqxka2kBghZeex56gKcBiUDOmbLbAAKhtT4xK8IT1xbauEfh lrlVfn8uXvGHeJOUdEsCRxPiq_FbFJpxMa5.LeTMBAlT0ac9Uk6xyzZvnCBK gE7Y7beSH9iAByuI1Te1SF2b_gykxF1eZWRx_FnGnnPqQrP0PtMpgqDkU1Ps REIxVRTzsvyc_Ai5bzBCsByZVsPIsJime3jbHM23__Vw_VIgnKv4i8QijIQw dovXjkSI1w2jr_J8d2jPruZUG9iaL_Wv2Lji2RUlA5Mw0LvrMVeriv6ztn_I SZs_uYrg8yJ3RGxJPwPdcDNgcQSigm5eqF2UP4Ur.YrzEhCg- Received: from [98.203.118.124] by web121605.mail.ne1.yahoo.com via HTTP; Sun, 18 Aug 2013 14:01:57 PDT X-Rocket-MIMEInfo: 002.001, VGhhdCdzIGZpbmUsIGl0J3MgYSB0ZXN0IHRvb2wsIG5vdCBhIHNvbHV0aW9uLiBJdCBqdXN0IHNlZW1zIHRoYXQgaXQgZ2V0cyBwdXNoZWQgYXMgaWYgaXQncyBzb21lIHNvcnQgb2YgcmVhbAp3b3JsZCBzb2x1dGlvbiwgd2hpY2ggaXQncyBub3QuIFRoZSBpZGVhIHRoYXQgYnJpbmdpbmcgcGFja2V0cyBpbnRvIHVzZXIgc3BhY2UgdG8gZm9yd2FyZCB0aGVtIHJhdGhlcgp0aGFuIGp1c3QgcmVwbGFjaW5nIHRoZSBicmlkZ2UgbW9kdWxlIHdpdGggc29tZXRoaW5nIG1vcmUgZWZmaWNpZW50IGlzIGp1c3Qgc2kBMAEBAQE- X-Mailer: YahooMailWebService/0.8.154.571 References: <520A6D07.5080106@freebsd.org> <520AFBE8.1090109@freebsd.org> <520B24A0.4000706@freebsd.org> <520B3056.1000804@freebsd.org> <20130814102109.GA63246@onelab2.iet.unipi.it> <1376745244.6575.YahooMailNeo@web121606.mail.ne1.yahoo.com> <1376748170.66110.YahooMailNeo@web121601.mail.ne1.yahoo.com> <1376833738.94737.YahooMailNeo@web121605.mail.ne1.yahoo.com> <71EA3DFB-B410-432D-98E0-B6341556BE6D@netgate.com> <1376851152.3322.YahooMailNeo@web121606.mail.ne1.yahoo.com> Message-ID: <1376859717.20232.YahooMailNeo@web121605.mail.ne1.yahoo.com> Date: Sun, 18 Aug 2013 14:01:57 -0700 (PDT) From: Barney Cordoba Subject: Re: it's the output, not ack coalescing (Re: TSO and FreeBSD vs Linux) To: Adrian Chadd In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Barney Cordoba List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 21:02:04 -0000 That's fine, it's a test tool, not a solution. It just seems that it gets pushed as if it's some sort of real world solution, which it's not. The idea that bringing packets into user space to forward them rather than just replacing the bridge module with something more efficient is just silliness. If "pushing packets" was a useful task, the solution would be easy. Unfortunately you need to do something useful with the packets in between. Reminds me of polling. The problem is that over time, people actually view it as a solution, when it was never more than a kludge in the first place. BC ________________________________ From: Adrian Chadd To: Barney Cordoba Cc: "freebsd-net@freebsd.org" Sent: Sunday, August 18, 2013 3:18 PM Subject: Re: it's the output, not ack coalescing (Re: TSO and FreeBSD vs Linux) On 18 August 2013 11:39, Barney Cordoba wrote: Great. Never has the been a better explanation for the word Kludge than netmap. Nah. Netmap is a reimplementation of some reasonably well known ways of pushing bits. Luigi just pushed it up to eleven and demonstrated what current hardware is capable of. I have never bought the "We need eleventy cores just to push 10ge of real traffic!" before. Luigi did note down where the per-packet inefficiencies were. What we have to do now is sit down and for each of those, figure out what the root causes are and how to mitigate it. There's some architectural things that need tidying up (read: CPU pinning, queue handling, some locking hilarity) but if they're solved, we'll end up having dual core boxes push line rate packets for routing. So the gauntlet has been thrown. Let's fix this shit up. -adrian From owner-freebsd-net@FreeBSD.ORG Sun Aug 18 21:03:35 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 01141396 for ; Sun, 18 Aug 2013 21:03:34 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm24-vm4.bullet.mail.ne1.yahoo.com (nm24-vm4.bullet.mail.ne1.yahoo.com [98.138.91.184]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B1AB92CDE for ; Sun, 18 Aug 2013 21:03:34 +0000 (UTC) Received: from [98.138.101.131] by nm24.bullet.mail.ne1.yahoo.com with NNFMP; 18 Aug 2013 21:03:27 -0000 Received: from [98.138.226.165] by tm19.bullet.mail.ne1.yahoo.com with NNFMP; 18 Aug 2013 21:03:27 -0000 Received: from [127.0.0.1] by omp1066.mail.ne1.yahoo.com with NNFMP; 18 Aug 2013 21:03:27 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 660521.76344.bm@omp1066.mail.ne1.yahoo.com Received: (qmail 11356 invoked by uid 60001); 18 Aug 2013 21:03:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1376859807; bh=fXT1nexAUr16FvMCAHv5UU7/bwMq76/1imph4sLPDEY=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=KWiIiPbOtMkwHCmx/zu054fRksf3vYKTCa6KbxYTaqNo6zPNcon5ErRUhM47B/Cfa4jGYV+1bP33D4Am3U403Aji8/HEooAYPvCa6TXDWrbcXt28iMJsA1et/dA39Zrtk7wNTr3+Yu3GQ+DYUQJYfiYjxSyu7VcgMuPThjZDDf8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=ihehRRbY4slVcYI1OMAOHAnt41bYzXc3QgcGT7aVr/p67sP7wyZsSwSaWJ9O7qc3sCf0obCuhf4EG6j7gITwGbbk4hq/nS0GRTGfYXGvc3IlP3hMzEz0DhHM2HFUpS+GK8/2KJdmq6A/faZGmxz0BlrO74hdzHAJ1MytBdzp7lU=; X-YMail-OSG: e.kET7gVM1nIfviteMoDG7426ulQcJHH.eEUK1MQs52h7yj Mox45zqQjWFxAhEP2vQu5qKHJBa0OT9gY_ceDKpC4k1DExwB4qmQn4Qtqayf kEwNt27cD0sAf8zD_VAFgNir3Dxp8b24OJNkM9KMLOF9aTdAJwDnEXFFzxno __v3U80pjtXmaBoP_Rtu5CM3xunWWMaQu6pBekxgU6delHFS37OgEQnqchQR j_x4vAgS8klZkdEgRWwqKeWUpWLcPO82NYAFQz_O9MQxxlgM.CpiOMEC7zL1 M0jNpMaXt9N0Lh.NddVqNvM9tzxnXvD4WhTMcjLdfgqTE.F8ZCHg7.B7kgvh 3n.MH255NzFhB3z3a_jWTH7Ek_..knNiB8LSpMZLHAYU50WMQeF4Xh0gWKQf 0uEzV9x3238x5LzX6F2PKnAXRTIOExVU51s_HqGcP1H88zePmDiLWOFcb0.Y CKgM0i7D8nZMuXCZI9crQZbSOu637TpxD2Vp.aCXZ_niG.2Zgw.HCTh19cDC RqEbgjafJHLI4YZPYSOOaZvglsOHNJ989ORUi8tysx8RP86JpXfwFthFLToc s0VbnaMKZ33zwXujd7PLbZmALGHQW_9F9ladV2NBKz_nzp32gpDvIy9B8Oe. bAxV5XDh2fdzQQe4sTmFRYpVS4yiXeNdYbZjkikD3Gg-- Received: from [98.203.118.124] by web121606.mail.ne1.yahoo.com via HTTP; Sun, 18 Aug 2013 14:03:27 PDT X-Rocket-MIMEInfo: 002.001, Q3JpdGljaXNtIGlzIHRoZSBiZWRyb2NrIG9mIGlubm92YXRpb24uCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KIEZyb206IFZpamF5IFNpbmdoIDx2aWpqdS5zaW5naEBnbWFpbC5jb20.ClRvOiBCYXJuZXkgQ29yZG9iYSA8YmFybmV5X2NvcmRvYmFAeWFob28uY29tPiAKQ2M6IEFkcmlhbiBDaGFkZCA8YWRyaWFuQGZyZWVic2Qub3JnPjsgImZyZWVic2QtbmV0QGZyZWVic2Qub3JnIiA8ZnJlZWJzZC1uZXRAZnJlZWJzZC5vcmc.IApTZW50OiBTdW5kYXksIEF1Z3VzdCAxOCwgMjAxMyAzOjQBMAEBAQE- X-Mailer: YahooMailWebService/0.8.154.571 References: <520A6D07.5080106@freebsd.org> <520AFBE8.1090109@freebsd.org> <520B24A0.4000706@freebsd.org> <520B3056.1000804@freebsd.org> <20130814102109.GA63246@onelab2.iet.unipi.it> <1376745244.6575.YahooMailNeo@web121606.mail.ne1.yahoo.com> <1376748170.66110.YahooMailNeo@web121601.mail.ne1.yahoo.com> <1376833738.94737.YahooMailNeo@web121605.mail.ne1.yahoo.com> <71EA3DFB-B410-432D-98E0-B6341556BE6D@netgate.com> <1376851152.3322.YahooMailNeo@web121606.mail.ne1.yahoo.com> <0E167A7F-092C-429F-9B5A-E05DB6126A96@gmail.com> Message-ID: <1376859807.11113.YahooMailNeo@web121606.mail.ne1.yahoo.com> Date: Sun, 18 Aug 2013 14:03:27 -0700 (PDT) From: Barney Cordoba Subject: Re: it's the output, not ack coalescing (Re: TSO and FreeBSD vs Linux) To: Vijay Singh In-Reply-To: <0E167A7F-092C-429F-9B5A-E05DB6126A96@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Barney Cordoba List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 21:03:35 -0000 Criticism is the bedrock of innovation.=0A=0A=0A___________________________= _____=0A From: Vijay Singh =0ATo: Barney Cordoba =0ACc: Adrian Chadd ; "freebsd-= net@freebsd.org" =0ASent: Sunday, August 18, 2013= 3:46 PM=0ASubject: Re: it's the output, not ack coalescing (Re: TSO and Fr= eeBSD vs Linux)=0A =0A=0ABarney, did you get picked on a lot as a kid? Wond= er why you're so caustic and negative all the time?=0A=0ASent from my iPhon= e=0A=0AOn Aug 18, 2013, at 11:39 AM, Barney Cordoba wrote:=0A=0A> Great. Never has the been a better explanation for the wo= rd Kludge than netmap.=0A> =0A> =0A> ________________________________=0A> F= rom: Adrian Chadd =0A> To: Jim Thompson =0A> Cc: Barney Cordoba ; FreeBSD Net ; Luigi Rizzo ; Lawrence Stewart =0A> Sent: Sunday, August 18, 2013 11:57 AM=0A> Subject: Re: it's= the output, not ack coalescing (Re: TSO and FreeBSD vs Linux)=0A> =0A> =0A= > Right. Well, post some profiling data, let's figure this out sometime.=0A= > =0A> Luigi can do bridging with 2 cores using netmap. So it's technically= =0A> possible. There's just a lot of kernel gunk in the way of doing it ye = olde=0A> way.=0A> =0A> =0A> =0A> -adrian=0A> =0A> =0A> On 18 August 2013 07= :25, Jim Thompson wrote:=0A> =0A>> =0A>> On Aug 18, 2013,= at 8:48 AM, Barney Cordoba =0A>> wrote:=0A>> =0A= >>> I could fill a tx queue with 10gb of traffic with=A0 yesteryear's cpus.= =0A>> It's not an achievement. Being able to bridge=0A>>> real traffic at 1= 0gb/s with 2 cores is=0A>> =0A>> Or forward at layer 3.=0A>> =0A>> Or filte= r packets.=0A>> =0A>> Or IPSEC.=0A>> =0A>> Or...=0A> ______________________= _________________________=0A> _____________________________________________= __=0A> freebsd-net@freebsd.org mailing list=0A> http://lists.freebsd.org/ma= ilman/listinfo/freebsd-net=0A> To unsubscribe, send any mail to "freebsd-ne= t-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Aug 18 21:16:38 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id E73285D2; Sun, 18 Aug 2013 21:16:38 +0000 (UTC) (envelope-from rizzo.unipi@gmail.com) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 22E0B2D5C; Sun, 18 Aug 2013 21:16:37 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id ev20so2747031lab.25 for ; Sun, 18 Aug 2013 14:16:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=7iXdMvxF4jSTV2c+RJu5MXCx1wl/+tIjeunknIs0zos=; b=U34pMmN1mgnjp0tDqFhogHsygZe8FBriVNeBTA+1RUFt5S5eLzhxWRZprjucNo92OZ FAKepcu0fe0ArAc/i2VdGBaP6+caEUpSZ3pKINLhyHWdYqYqQyjktvr0IAFMRjO0f6ZD 5iwnsFO8B/S4vAzll6xmROOVTc0jclgRX8Je+/HMotyVjKVWeUswt5bPJ5KYAJJCKqrW r+c3hUSIXkNQgBqaexrzHqxlwE3J6SKbA29Oo9P83OJojD84SwAQf0mVjgScrIsc6z8r VmuAjoFtPWADrLxeSNhhRBVRULTiQTkCwAWNTAYlr7HYkixMEEkAs2H+ghNuEZiVJwTR 62dA== MIME-Version: 1.0 X-Received: by 10.112.33.205 with SMTP id t13mr12763835lbi.22.1376860595872; Sun, 18 Aug 2013 14:16:35 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.200.165 with HTTP; Sun, 18 Aug 2013 14:16:35 -0700 (PDT) In-Reply-To: <1376859717.20232.YahooMailNeo@web121605.mail.ne1.yahoo.com> References: <520A6D07.5080106@freebsd.org> <520AFBE8.1090109@freebsd.org> <520B24A0.4000706@freebsd.org> <520B3056.1000804@freebsd.org> <20130814102109.GA63246@onelab2.iet.unipi.it> <1376745244.6575.YahooMailNeo@web121606.mail.ne1.yahoo.com> <1376748170.66110.YahooMailNeo@web121601.mail.ne1.yahoo.com> <1376833738.94737.YahooMailNeo@web121605.mail.ne1.yahoo.com> <71EA3DFB-B410-432D-98E0-B6341556BE6D@netgate.com> <1376851152.3322.YahooMailNeo@web121606.mail.ne1.yahoo.com> <1376859717.20232.YahooMailNeo@web121605.mail.ne1.yahoo.com> Date: Sun, 18 Aug 2013 23:16:35 +0200 X-Google-Sender-Auth: RD-M6p02drs4aT4QN-F_27u-WdE Message-ID: Subject: Re: it's the output, not ack coalescing (Re: TSO and FreeBSD vs Linux) From: Luigi Rizzo To: Barney Cordoba Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-net@freebsd.org" , Adrian Chadd X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Aug 2013 21:16:39 -0000 On Sun, Aug 18, 2013 at 11:01 PM, Barney Cordoba wrote: > That's fine, it's a test tool, not a solution. It just seems that it gets > pushed as if it's some sort of real > world solution, which it's not. The idea that bringing packets into user > space to forward them rather > than just replacing the bridge module with something more efficient is > just silliness. > you might want to have a look at the VALE switch http://info.iet.unipi.it/~luigi/vale/ the upcoming version can attach physical interfaces to the switch and keep all the processing within the kernel. > If "pushing packets" was a useful task, the solution would be easy. > Unfortunately you need to do > something useful with the packets in between. > > there are different definitions of what is "useful": sources, sinks, forwarding, dropping (anti DoS), logging, ids, are all useful for different people. The mistake, i think, is to expect that there is one magic solution to handle all the useful cases. cheers luigi