From owner-freebsd-net@freebsd.org Thu Jul 26 08:28:21 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72C3E1047603; Thu, 26 Jul 2018 08:28:21 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 150A884BC9; Thu, 26 Jul 2018 08:28:21 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id B1BA824306; Thu, 26 Jul 2018 08:28:20 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [192.168.228.1] (ptr-8rgnodvo8tn4qa3lwai.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240b:b802:95b1:ed91:aab2:81ba]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id EE7F85D75E; Thu, 26 Jul 2018 10:28:18 +0200 (CEST) From: "Kristof Provost" To: "Patrick Lamaiziere" Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Subject: Re: PF problems with 11-stable Date: Thu, 26 Jul 2018 10:28:18 +0200 X-Mailer: MailMate (2.0BETAr6116) Message-ID: In-Reply-To: <20180726101627.2db93a49@mr185083> References: <20180722155341.065c3d4d@romy.j20.helspy.pw> <20180726095805.28f86c64@mr185083> <20180726101627.2db93a49@mr185083> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2018 08:28:21 -0000 On 26 Jul 2018, at 10:16, Patrick Lamaiziere wrote: > Le Thu, 26 Jul 2018 09:58:05 +0200, > Patrick Lamaiziere a écrit : > > Hello, > >>> Hey, >>> I am on >>> 11.2-STABLE FreeBSD 11.2-STABLE #9 r336597 >>> Sun Jul 22 14:08:38 CEST 2018 >>> >>> and I see 2 problems with PF that are still there: >>> 1.) set skip on lo >>> does not work even though ifconfig lo matches. >>> SOLVED TEMPORARILY BY: set skip on lo0 >> >> I've seen this while upgrading from 10.3 to 11.2-RELEASE. I've added >> lo0 to set skip too. >> >> When the problem occurs, lo is marked '(skip)' (pfctl -vs >> Interfaces) but not lo0. >> >> But I can't reproduce this, this happened only one time. > > I don't know if this is related but there were some kernel logs about > 'loopback' : > > Feb 15 17:11:48 fucop1 kernel: ifa_del_loopback_route: deletion failed: > 47 Feb 15 17:11:48 fucop1 kernel: ifa_add_loopback_route: insertion > failed: 47 Jul 16 13:50:36 fucop1 kernel: ifa_maintain_loopback_route: > deletion failed for interface ix2: 3 Jul 16 14:07:31 fucop1 kernel: > ifa_maintain_loopback_route: deletion failed for interface ix2: 3 Jul > 16 14:07:31 fucop1 kernel: ifa_maintain_loopback_route: deletion failed > for interface igb1: 3 Jul 16 14:10:43 fucop1 kernel: > ifa_maintain_loopback_route: insertion failed for interface igb0: 17 > No, those error messages are not related. The issue with interface groups is known, and is being worked on. The pfctl -n issue should be fixed as of r336164 Regards, Kristof From owner-freebsd-net@freebsd.org Thu Jul 26 14:27:58 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F9CB1050D31 for ; Thu, 26 Jul 2018 14:27:58 +0000 (UTC) (envelope-from ricsip@gmail.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4A2B91BAF for ; Thu, 26 Jul 2018 14:27:57 +0000 (UTC) (envelope-from ricsip@gmail.com) Received: by mail-io0-x22c.google.com with SMTP id i18-v6so1478568ioj.13 for ; Thu, 26 Jul 2018 07:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=i+MfVbPXfh6KDirH4jirIAtn+29u83LYCtXfn6lCGqI=; b=nk4zREj/p+2GQa6Ez0dfoDIX7LPtCwhgDXJw0Y7v7P2CeNRmtZs2GrgJTj/Z9FToGY MoSdPCgC9atVKKmWu6lcFJnK2B/os82HefDT8x0jJc7wTwsRmug6msI3HRnIXV0MC4xv 2KdcNbzljGZzYaGNVKMZJBDzFE9gnKu1lMRoOkfZRipRzfP+GDRqCgi4fsmokjeVAHP0 Sr5SzucOLJwnMBXad7EKX+REnitU2Slh8t47VXSjrVf2sqtXRnKSm2e0Cn9l+QYnre45 OV/zqa7a3YFWAL9UF5N7YNQAEDuFskT3mdnjOWlzjhiyUl3LJ6PpQdHXHMpvJB1cn7EL 03Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=i+MfVbPXfh6KDirH4jirIAtn+29u83LYCtXfn6lCGqI=; b=un8M0V+mzoZTdnLHW23CBlkAFDaN1682Yu2Xbtn7UkMGGu5Mz67SK/02zlrTYyvBO6 fYrKi5x9ZXH4xDR/repk3OqOl6ASRBvdbKB33ua7oGWCDwpEsP3sciUO2VEYSbZ8boPp OJ6RL7HG203QDRUKP8ddoiIyU68nDEnqMseTJJjqjOc4r8B6wsUa3t5/QBnR8j1dRSQR LdMjHEFyXSbQPlh6C3IUZ5+Jjj4ChArvWxtwGiCwKUppdI8/6quGCgYzJfvgtfvSRlo3 9lblGTnwBsWfIdW5kQmXhTO2z/Tc4LahFW2JHZeVREvzA41/aYeeXi5MFNisCeUHW4BI SCZQ== X-Gm-Message-State: AOUpUlEhpqkD66ymbpOBKDcTQktouXdIxE+bBdoYhrDggrqCqZMJMjLe g+Lo6LyMp+b/GpThFVzpKw4daa4rWH/MLt9d32lIgO6iZ04= X-Google-Smtp-Source: AAOMgpdKREXXwy3TsejObF5rnwsnQapwVtEN17ya4uS+KIKfUnkxp4wESINTh5oWO4DpY3kfOeSaCx5odqJ4xu3lv0o= X-Received: by 2002:a6b:26ca:: with SMTP id m193-v6mr1798499iom.91.1532615277024; Thu, 26 Jul 2018 07:27:57 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a02:878a:0:0:0:0:0 with HTTP; Thu, 26 Jul 2018 07:27:26 -0700 (PDT) From: Richard Pasztor Date: Thu, 26 Jul 2018 16:27:26 +0200 Message-ID: Subject: PPPoE RX traffic is limited to one queue To: freebsd-net@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jul 2018 14:27:58 -0000 Dear all, continuing the discussion from here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203856 summary: (technically affects any NIC) Multi-queue NIC can use only RX queue "0", if PPPoE session is established (due to point-to-point connection, the RSS cannot load balance among multiple queues, at least that is what I was told and at the moment I have no idea if that is true or not) Result: 1 Gigabit traffic cannot be reached on some embedded SoC (in my case PC Engines APU2, Quad-core AMD 1Ghz CPU) due to less-than-server-grade single-core performance to handle the flow. Note: I am not planning to achieve wire-speed 1Gbit using IMIX (smallest packet size possible), on the contrary, I was testing using iperf3 with max. MTU + MSS Note2: I am not at the level of building a proper PPPoE simlator network to properly validate the final performance, all my tests were performed using pure IP routing. So expect PPPoE can be by definition only worse than what I can possibly reach using pure IP. I was suggested to set the following: net.isr.numthreads=4 net.isr.maxthreads=4 net.isr.bindthreads=1 net.isr.dispatch=deferred It didnt improve the situation, max. throughput was about the same, but CPU load in interrupt handling effectively became doubled. On the other hand, installing a small Linux firewall distrib (IPfire), I could easily manage to get 900+ MBit of traffic with less interrupt load on the system. So I dont know how could Linux overcome the above said limitation. As the original bugreport was against a possibly "igb" Intel driver issues, and that was said not the case, the PR was closed. So no advancement is expected in this topic, hence I opened it here, hoping there is some solution for this issue. Regards, Richard