From nobody Thu Jan 6 23:10:11 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C83EF1941633 for ; Thu, 6 Jan 2022 23:10:30 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JVMX619Bqz4SmD for ; Thu, 6 Jan 2022 23:10:30 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-pj1-x1029.google.com with SMTP id l10-20020a17090a384a00b001b22190e075so10174944pjf.3 for ; Thu, 06 Jan 2022 15:10:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=iIdlo34jbK9HhTnJsr7urD8s1VFniXNlWOxY9PWNuZ8=; b=agzzsgMK+T0n8mcqNwKuE438VcCbORP/GU1W6dXH1esZtPFib0la4G3sVkYgOsFvOf Anr2qOv5F3EzlA1wdTVsrumZ7wqWhQ90TQI91Qvmx9hep9e+8VydiJt6pCz09WLo/jUt 2DgRAIS/DlOpfAmw6NR+h4KxQKK8fX2xdkH5nZQOONUvTQi67dxnwoR32OB9CgXFQ49s 2FCe3bhEWypDHi2+qacE7mA8WGvY+qXUdDtVwhpm69jH/jORJAqJSunMVCVWPv01fcmk aGCUDs0ert6C32ftcnOEm0jDzrJ8ylSwsEKFsnrq0OQGiV7psJdTiqUu9HgEAuUNhbuU T61A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=iIdlo34jbK9HhTnJsr7urD8s1VFniXNlWOxY9PWNuZ8=; b=Vtl3BdpKGZRP44oC5esQoJSAthCdxm3smZvup3WquwzmclP6dvduPWSJkchpvkP4RJ m/2tztRiVBgpJ5Qi+8tqeFZSf5RJxlsDcCqdJTr3nkImb+cJ17nZYMoZiIwR0//ezMvq 6AyUKFFphslnvvZFsR9Kl2ZSgFN+hB0e1sHrCBMm3ajAFbxVw7ZTk2Uif0TebNHPD6Rh M6NRK517i1CBvcCTbcBIiApAn3dMvgg/DRCHOnHiVVyc7STyZdH59sJzFIoeo7UfLfh3 T+bR4mZO4/M6pBP4rhtXolpG3pS5O6x2L78tyM+8pW8vbTGrNPb4RNSvuKfg2+dXSDDu 0iPQ== X-Gm-Message-State: AOAM531TKPQPKNlm3BtXnzI7zlvMy8OnfaZ6zM4o3k1YtQryphnlC6iX d/KmvnuG4NJJZD9On7y7WQhQ4l34aetb+cxDfg/FcCau X-Google-Smtp-Source: ABdhPJydyPAzcX1+mkW02/UKfuRUu3/iN1CVGnsZWbFX6EZQpHuYCLVBZJ6iMIZvGrL2qmH+e5iZSZTYqalauxIxxZI= X-Received: by 2002:a17:902:9698:b0:149:b7bf:9925 with SMTP id n24-20020a170902969800b00149b7bf9925mr23523793plp.49.1641510622351; Thu, 06 Jan 2022 15:10:22 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 From: Ryan Stone Date: Thu, 6 Jan 2022 18:10:11 -0500 Message-ID: Subject: How should TCP LRO handle TH_PUSH? To: "" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4JVMX619Bqz4SmD X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=agzzsgMK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rysto32@gmail.com designates 2607:f8b0:4864:20::1029 as permitted sender) smtp.mailfrom=rysto32@gmail.com X-Spamd-Result: default: False [2.87 / 15.00]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_SPAM_MEDIUM(0.99)[0.994]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1029:from]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_SPAM_LONG(0.87)[0.873]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N I've been working on writing unit tests for LRO (see my message to freebsd-testing@ for more details on this). I've submitted reviews for two issues found by my tests that I believe to be outright bugs. I did find one more issue where I'm not sure whether it's really a bug or not. If LRO sees a TCP packet that does not have TH_PUSH set, and then merges a subsequent packet that does have TH_PUSH set into it, what should the value of the TH_PUSH flag in the merged packet be? When I wrote my test I expected to see TH_PUSH set, but that isn't our current behaviour. On the one hand I'm not sure that this is strictly correct, but on the other hand I don't think we do anything with TH_PUSH on a received packet anyway. I did code up a proposed fix for this, but I wanted to get feedback as to whether it's worth worrying about before sending the review. Does anybody have any opinions? Thanks, Ryan From nobody Thu Jan 6 23:19:34 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E5A8919434EB for ; Thu, 6 Jan 2022 23:19:44 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JVMkm5FDQz4TcP for ; Thu, 6 Jan 2022 23:19:44 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:3191:d1d8:e866:8eae]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id 90AE2721BE008; Fri, 7 Jan 2022 00:19:35 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.40.0.1.81\)) Subject: Re: How should TCP LRO handle TH_PUSH? From: tuexen@freebsd.org In-Reply-To: Date: Fri, 7 Jan 2022 00:19:34 +0100 Cc: "" Content-Transfer-Encoding: 7bit Message-Id: <783C3EE7-8685-42E4-9B97-3FC28542F19D@freebsd.org> References: To: Ryan Stone X-Mailer: Apple Mail (2.3693.40.0.1.81) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: 4JVMkm5FDQz4TcP X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N > On 7. Jan 2022, at 00:10, Ryan Stone wrote: > > I've been working on writing unit tests for LRO (see my message to > freebsd-testing@ for more details on this). I've submitted reviews > for two issues found by my tests that I believe to be outright bugs. > I did find one more issue where I'm not sure whether it's really a bug > or not. If LRO sees a TCP packet that does not have TH_PUSH set, and > then merges a subsequent packet that does have TH_PUSH set into it, > what should the value of the TH_PUSH flag in the merged packet be? > > When I wrote my test I expected to see TH_PUSH set, but that isn't our > current behaviour. On the one hand I'm not sure that this is strictly > correct, but on the other hand I don't think we do anything with > TH_PUSH on a received packet anyway. I did code up a proposed fix for > this, but I wanted to get feedback as to whether it's worth worrying > about before sending the review. Does anybody have any opinions? I guess conceptually you would set the PUSH flag on the merged segment, if it was set on any segment being merged. But since it doesn't matter on the receive path, we should only do this, if doing so does not affect the performance. Best regards Michael > > Thanks, > Ryan > From nobody Wed Jan 19 07:33:04 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8AB79196180E for ; Wed, 19 Jan 2022 07:33:15 +0000 (UTC) (envelope-from Richard.Scheffenegger@netapp.com) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2079.outbound.protection.outlook.com [40.107.96.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jdy6c5GTxz4RV7 for ; Wed, 19 Jan 2022 07:33:12 +0000 (UTC) (envelope-from Richard.Scheffenegger@netapp.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FkS8kG71langmqQ5FyUzsAA5ScPmaTVGS064iQQFTrBXrP4NxDAiHbGSExQ8h83xxOa1PRPf5xpXFD6ryolORUek4w4yvgMrui2Q3apWu9xjCyWI+zMU3yCmvDmTs0oNcugTrQI/9I3o7zjbjaccAQm/zAw2VJR+DO8OiW8bbRxKBrcm1aQmyoYMb3IJNmJqXoa0t8ImyvSqTF8PZL5q5AQOzt/6hungl9ikFUf8tgHA4iM9TBSgDzNrpUBLUq4hTGK3ZvDf/Irnd4y7pklW0H7VN+9dSrDp3yVYFgz+F7PD4KWsWvDG8cU7Wy+b0hZDvudo7suZtLyVupQq1EgENA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EVCYOjr4xboeqFmlbP71woU5+4SQJTJj+Gci9s1giP4=; b=mfE7ldxRodVzen/AoHXDm77pk5tJiM6Fhdd2Hx7C3LO92dvnfk1tOHUgVwb1SD7THgLxa0FI5KtlH7JkBG0KpGbjbMMWrPlMgrlbun87Uyd/YqFot/SQyp1UoJSp1Ahmsk2LxICotobZuKRbdNofGmWwjOom2GTpyYwR9tC9uIYFoaA/ypMvZTHQDSd3f3ro7vlOIf9eqlbhTGBEEXCg5+mkqUIl8HwV6yDL91M0E+QF7LPM4Qa6VeybbUTVbm1ESYKOQ1i1UciEtJozx2dRe16avxx0wGFep8nBPaEi7Ey7Q83m9Q+lIYjvo++wkflzXmtDDffHFbQFOBvnLyTddw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netapp.com; dmarc=pass action=none header.from=netapp.com; dkim=pass header.d=netapp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netapp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EVCYOjr4xboeqFmlbP71woU5+4SQJTJj+Gci9s1giP4=; b=hwcvLVsOb/E+MhU1BVU2bkf8Q7JErBaUEmpS0w0pbC1+W3Y0OoEjQdkyg57Ap0czHJQueV/d3hd1qOTlSfAtZuGtxUNpkUR1NXIcqN74O7W0jtfh7hy/0yYK2dWvDa6e4NAFN1TDpuFJU1qgA5/fpszlhDoI0J5O4spg46BYNr9gLiYaut8+JGZaiOf1cKsVzLBJ+R3uLMSZuP9s8qFKi6jpXLNuq5GnhNg4aqBhvREDY4Lotbaa/wGx6JpZc9G/kCfYY7rhXXM3FZ3jjmKtbYSF3OPxSNnQ+7aylpxi7TIdzO3wEeOWfBFqMcHOAjapbQEhMbRteukGJa5cG+EQ6w== Received: from CO6PR06MB7635.namprd06.prod.outlook.com (2603:10b6:303:ab::8) by DS7PR06MB6902.namprd06.prod.outlook.com (2603:10b6:5:2d0::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.12; Wed, 19 Jan 2022 07:33:04 +0000 Received: from CO6PR06MB7635.namprd06.prod.outlook.com ([fe80::f013:1e9:ac0c:7e32]) by CO6PR06MB7635.namprd06.prod.outlook.com ([fe80::f013:1e9:ac0c:7e32%4]) with mapi id 15.20.4909.008; Wed, 19 Jan 2022 07:33:04 +0000 From: "Scheffenegger, Richard" To: Ryan Stone , "" Subject: RE: How should TCP LRO handle TH_PUSH? Thread-Topic: How should TCP LRO handle TH_PUSH? Thread-Index: AQHYA1KcRxNwuiEN8Ue4e+UuoJqsS6xqBUxA Date: Wed, 19 Jan 2022 07:33:04 +0000 Message-ID: References: In-Reply-To: Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3c7eb00e-a972-4505-4d48-08d9db1dee4c x-ms-traffictypediagnostic: DS7PR06MB6902:EE_ x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: wdw9z42UbbLdH7K8KLD/BYtAfGcxvJq20j+wUostbQhii+qXLewF4B5DazFAvYfvKGD0gY3EwnEZ1pXdrKm9DPEaocbW5MeD16WcqxseEkstdc5oI2CU18Kik7CcCopTzu3T1teack/PyHQHFYqcab+czzByaiHzrpfWVzo4AiELd45pU67HEDhYO4oNWusrDU4lXBVjLTyhUJiUhN/uY4cb+1hQQl1fAH97Jfe9dxydDB2aJIyB150+0q0l7O7inqdNOhWm5PzyWC3kvChftjV8fWoM49Eq9rhvr3gH6Z5i0C7InBvwnIkDE7HOdWo9okMkm+Ti1/XuziqFVOLQUXniejwWALtkt4WRnZz0olF4QRY9hbXRYd0AWmZuRjURV7o5xJqMHg3pDQLPse66SmlMwHBeByTrHZeeCdK48WQ3dUGCBGiTaCLoSdJ5ldWIwCzObZOtvJnwbQibwFw00vSH0+De7IK/kQwhZNy5psh9vmleOsVjvNu6v2zF9mBbKLD6kuStJne//iunhe28KhgPdljK25iYQetRlOZT9//crEeGfaUBt1Fe3KOmhl7HCtvJ0C8FhfTebrCen3c2VYJYko4F7xhkg3J/a4ysKliyGSmF1Z4n4TY/fJTDH0cbHRUwYM+aLpTvFZPmeCqrV7RSoBBZC1akpGPSBRjqUucJgl6ZyCtpRgLyH/EzazxwgiwOhu1c3XMqzMKYqVhh5teZD7u3GmeI03+h1aBKZWQ= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR06MB7635.namprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(66446008)(71200400001)(66556008)(38070700005)(76116006)(6506007)(53546011)(8676002)(55016003)(122000001)(9686003)(2906002)(186003)(33656002)(26005)(64756008)(7696005)(508600001)(110136005)(316002)(52536014)(66476007)(66946007)(38100700002)(5660300002)(83380400001)(66574015)(86362001)(8936002)(491001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?Z0pJR1JxWTBCc3RKTEtOL3VXang3RWdKd1RkZlRSZEVDNTZibU53TzJ5dEUv?= =?utf-8?B?VUtpeVgwejJGbVlDMVU1SnU0Zi93L2RwQ1dCa0R1WVhJcnc1OVJYVS92UGNF?= =?utf-8?B?UGJvRUZFbDdYcUw1NFdxVE0ySHhFZVZ0TzZncC8yNnJXNlJmU0t2dUpZc1pq?= =?utf-8?B?RmxXZjJya1N4WVBJSHNPUXYvWFgrOE5sUU1sd3lrbmM1OUNJa0JzVjN0ZXA5?= =?utf-8?B?a2VTWkVpMFVZK2ROZEVSQjZEZEk2T3c2Rm5qL0R2MlpBVHpxU3MxN0tGcnc5?= =?utf-8?B?ZFplZm1YRGRsS3I0UWlSTlBGN21PdXBhaDMxRFQzdU1ydGNoMDZJTUJnQ2NF?= =?utf-8?B?d3dCVzh3OUR5c1JaR2Q3MjhpaUQ4QkRsb3dLODRUcWY0aDZVWHVZdWNUMWJP?= =?utf-8?B?QXFiekk1SEM1SGJYUTY0bEhZUnJJb2cvY3g5YS9PMHlkV1plQVhvanFwc3dJ?= =?utf-8?B?Z0ZVcXRmdVVPTU1IY3I3ZlhXWE1NcXV2ZzlJTUNaZWoyTG40czV4eEY5L1I5?= =?utf-8?B?ZlQ4ZGoxZU9aaWZxalN2VDZ4V2JvaHRtdi9OeXN3WHEwRzRsVGZ5VFgyK29N?= =?utf-8?B?TXdaZnVjMm9EL25aRThXVkp2bXBVUmMzcDlaa3JhejJkYU11djJvZHpLTTFI?= =?utf-8?B?Wm14NGd4a3BFbXhuN1hUQThzRExXSm4vaWM5UGNqSk1MQUczdzFuZVpjc01H?= =?utf-8?B?MzNuSlRnQVBhTDh0aEZpU1FocCtYY1FuUEM2S0Fsd1p3bld2dzdyOGpOVXdK?= =?utf-8?B?ak1ZSGJYTHBsbURkWGlpcG9aTEtZUTJNTmROME52b3dFTFpBVHp3dXNnOVRC?= =?utf-8?B?Z014Y240QXRYenQyNlVGbVM5QmprNEd6TjVWekR2eEVnNW1SVmdaNWl0Y214?= =?utf-8?B?YStFYjFiNXZFRi9WdHBTbE5Kb3Y0QjIxNTRraTlCUHg5TGhrMkMvRm9GRGxu?= =?utf-8?B?UXRTdmVDOEVBQ2RLK3FrZFJMUXNEc2ZjN2IxeDBVTDg2TXVlK0lKQUM1Rmhu?= =?utf-8?B?T1JERW5yVFpTdVNuVU5hSzRzajZlQVZoWmVmU3FsTmhQdDZ4Si8yZWd6bVh4?= =?utf-8?B?dExDYW8xeTZwV1o1UnF2eDZNbzdNQnREUEM3dUROSE9TbjRWMWl1b0NFaitS?= =?utf-8?B?dVAyMFNZVitpZW1RdWRQUTdzUGRmL1l3RWN2Vi9iQWZYNGtmdUVjdXJwMWFh?= =?utf-8?B?WG5IN2xybDc0Sm1FTmRwSTFTdG1HSEk5bTl6cWR4VXA5a2l2ZURSWmdURGdR?= =?utf-8?B?YURCeTRKTEY2cjVKTkJzVFcwd2NpTzlnQTFKZTVWaEdEaFVkMFRkL2lybjMx?= =?utf-8?B?MUtidjVDbTJXejdPQ05jNy9aSmhqVC9KdldQYXZESC9xYlppeUNYcEZOWjdt?= =?utf-8?B?dGluUnZMZVdVMGRhS05yWnI3WDlMM1F2SjlVRmlxYS82VzNUeFp4Y2FaVW5r?= =?utf-8?B?WmtsNUMvZUpYWnIvdWF4SWlweHlVZ0N6d3hOSmxkdGlldEdLTDZxN2w5Q1FV?= =?utf-8?B?enMram11d3dvbVhCaWdaS0ZQS05VYzFoMkdKNkdpeEdmZ1RkbEVBa3djZ3ZO?= =?utf-8?B?a0tuRnExTlNnTmtWNG5HR1lMM3FMaUVQczVPMHY5THZZaFl4anFkUC95Skxz?= =?utf-8?B?UjNHRzlvdHJzSFlTejU1TUF3Rzd4VC9kVFdxQzBBcEw2ZnBEei9EUk9xNnlW?= =?utf-8?B?YlRNVnRSb25uTDBEbHZjQ243WHNod0VmZSs5SUg5VUp1TlVJRi91QXVwQ1Yr?= =?utf-8?B?YXJoUDI3NnRTZ2hxbGhIRjJ5ZjRMTkdHL1dZaXhvTkxVUnVUMFAvQjY2R1ov?= =?utf-8?B?UkF3OUdzTUpBSFUyRzZ5eEJ5Z3RpYkRYeTRFSC9BSGFLNXlIbHFZbHZqV0xG?= =?utf-8?B?TjdacHJjbnM5T3MzMkxjVGVmR3k1d1B6WUpuM3l5akFtYXVQY1JHQ1pTOXJT?= =?utf-8?B?WFdPelNjbFYvVUxYYWVqQzJwVVdXTHZuUTdidFQrNjVHOEViTWVzRFB3ajBu?= =?utf-8?B?Ri9CdElGakdLYVJGbWs5b0ZCV2ZNZlFWdys0T0NXNlVkc2xWWDlZMGlKK1FC?= =?utf-8?B?YlZLSGphL1pKNjhWWWRyckVKS3VCN2dqaE1Xb0x2M21OK3Y0VXpqOGJLK3hK?= =?utf-8?B?WDNIK09pdG02V2N6Sm9PZHBYNFZ1WmZGRlF5T1NIOHA1S3FId2tXL0FsU2lh?= =?utf-8?Q?CMQelx7UwXiG5iExZIZHGkY=3D?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: netapp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO6PR06MB7635.namprd06.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3c7eb00e-a972-4505-4d48-08d9db1dee4c X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2022 07:33:04.7210 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4b0911a0-929b-4715-944b-c03745165b3a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: kY6ZGUNtjauCDkWkCRccUK0+ix13r+HP2RT3eqPtGXdcN8+3YVsFApoov4kQiZmv+SSaVU2PR6kkcR7cV4kVZw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR06MB6902 X-Rspamd-Queue-Id: 4Jdy6c5GTxz4RV7 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netapp.com header.s=selector1 header.b=hwcvLVsO; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=netapp.com; spf=pass (mx1.freebsd.org: domain of Richard.Scheffenegger@netapp.com designates 40.107.96.79 as permitted sender) smtp.mailfrom=Richard.Scheffenegger@netapp.com X-Spamd-Result: default: False [-2.90 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[netapp.com:s=selector1]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; DWL_DNSWL_LOW(-1.00)[netapp.com:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[netapp.com:+]; MIME_BASE64_TEXT(0.10)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.107.96.79:from]; DMARC_POLICY_ALLOW(-0.50)[netapp.com,quarantine]; MLMMJ_DEST(0.00)[freebsd-transport]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; RCVD_TLS_LAST(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.96.79:from] X-ThisMailContainsUnwantedMimeParts: N QXMgZGlzY3Vzc2VkIGluIHRoZSBsYXN0IHRyYW5zcG9ydCBjYWxsLCBjYW4geW91IHBsZWFzZSBz aGFyZSB0aGUgYmVoYXZpb3Igd2l0aCBvdGhlciBUQ1AgZmxhZ3MgYXMgd2VsbD8NCg0KVVJHIGlz IG1vc3RseSBoaXN0b3JpYyANCg0KRUNFIGlzIHRyaWNreSAtIHByZXN1bWFibHkgdGhlIGZsYWcg b2YgdGhlIGxhc3QgKGluLXNlcXVlbmNlKSBwYWNrZXQgc2hvdWxkIHRha2UgcHJlY2VkZW5jZSB3 aXRob3V0IEFjY0VDTi4gV2l0aCBBY2NFQ04sIHRoZSBuZXcgY29kZXBhdGggd2hlcmUgTFJPIHBy b3ZpZGVzIHRoZSBmdWxsIHNlcXVlbmNlIG9mIGFsbCByZWNlaXZlZCBoZWFkZXIgYml0cyBpcyBw cmVmZXJyZWQuDQpDV1Igc2hvdWxkIGJlIGxhdGNoZWQgLSBpZiBhbnkgcGFja2V0IGluIHRoZSBM Uk8gaGFzIGl0LCBpdCBzaG91bGQgYmUga2VwdA0KUFNIIHNob3VsZCAoY29uY2VwdHVhbGx5KSBi ZSBsYXRjaGVkLCBldmVuIHRob3VnaCB0aGUgRkJTRCBzdGFjayBkb2Vzbid0IGRvIGFueXRoaW5n IHdpdGggaXQgLSBvdGhlciB0aGFuIHBvc3NpYmx5IHRyaWdnZXJpbmcgYW4gaW1tZWRpYXRlIEFD SyAoZ29vZCB0byByZWR1Y2UgdGhlIGRlcGVuZGVuY3kgb24gdGhlIGRlbGF5ZWQgQUNLIHRpbWVy KS4NCg0KV2hhdCBpcyB0aGUgYmVoYXZpb3IgZm9yIHRoZSBvdGhlciBmbGFncyAoRklOLCBSU1Qs IFNZTik/DQoNCkJlc3QgcmVnYXJkcywNCiAgIFJpY2hhcmQNCg0KDQotLS0tLU9yaWdpbmFsIE1l c3NhZ2UtLS0tLQ0KRnJvbTogb3duZXItZnJlZWJzZC10cmFuc3BvcnRAZnJlZWJzZC5vcmcgPG93 bmVyLWZyZWVic2QtdHJhbnNwb3J0QGZyZWVic2Qub3JnPiBPbiBCZWhhbGYgT2YgUnlhbiBTdG9u ZQ0KU2VudDogRnJlaXRhZywgNy4gSsOkbm5lciAyMDIyIDAwOjEwDQpUbzogPGZyZWVic2QtdHJh bnNwb3J0QGZyZWVic2Qub3JnPiA8ZnJlZWJzZC10cmFuc3BvcnRAZnJlZWJzZC5vcmc+DQpTdWJq ZWN0OiBIb3cgc2hvdWxkIFRDUCBMUk8gaGFuZGxlIFRIX1BVU0g/DQoNCk5ldEFwcCBTZWN1cml0 eSBXQVJOSU5HOiBUaGlzIGlzIGFuIGV4dGVybmFsIGVtYWlsLiBEbyBub3QgY2xpY2sgbGlua3Mg b3Igb3BlbiBhdHRhY2htZW50cyB1bmxlc3MgeW91IHJlY29nbml6ZSB0aGUgc2VuZGVyIGFuZCBr bm93IHRoZSBjb250ZW50IGlzIHNhZmUuDQoNCg0KDQoNCkkndmUgYmVlbiB3b3JraW5nIG9uIHdy aXRpbmcgdW5pdCB0ZXN0cyBmb3IgTFJPIChzZWUgbXkgbWVzc2FnZSB0byBmcmVlYnNkLXRlc3Rp bmdAIGZvciBtb3JlIGRldGFpbHMgb24gdGhpcykuICBJJ3ZlIHN1Ym1pdHRlZCByZXZpZXdzIGZv ciB0d28gaXNzdWVzIGZvdW5kIGJ5IG15IHRlc3RzIHRoYXQgSSBiZWxpZXZlIHRvIGJlIG91dHJp Z2h0IGJ1Z3MuDQpJIGRpZCBmaW5kIG9uZSBtb3JlIGlzc3VlIHdoZXJlIEknbSBub3Qgc3VyZSB3 aGV0aGVyIGl0J3MgcmVhbGx5IGEgYnVnIG9yIG5vdC4gIElmIExSTyBzZWVzIGEgVENQIHBhY2tl dCB0aGF0IGRvZXMgbm90IGhhdmUgVEhfUFVTSCBzZXQsIGFuZCB0aGVuIG1lcmdlcyBhIHN1YnNl cXVlbnQgcGFja2V0IHRoYXQgZG9lcyBoYXZlIFRIX1BVU0ggc2V0IGludG8gaXQsIHdoYXQgc2hv dWxkIHRoZSB2YWx1ZSBvZiB0aGUgVEhfUFVTSCBmbGFnIGluIHRoZSBtZXJnZWQgcGFja2V0IGJl Pw0KDQpXaGVuIEkgd3JvdGUgbXkgdGVzdCBJIGV4cGVjdGVkIHRvIHNlZSBUSF9QVVNIIHNldCwg YnV0IHRoYXQgaXNuJ3Qgb3VyIGN1cnJlbnQgYmVoYXZpb3VyLiAgT24gdGhlIG9uZSBoYW5kIEkn bSBub3Qgc3VyZSB0aGF0IHRoaXMgaXMgc3RyaWN0bHkgY29ycmVjdCwgYnV0IG9uIHRoZSBvdGhl ciBoYW5kIEkgZG9uJ3QgdGhpbmsgd2UgZG8gYW55dGhpbmcgd2l0aCBUSF9QVVNIIG9uIGEgcmVj ZWl2ZWQgcGFja2V0IGFueXdheS4gIEkgZGlkIGNvZGUgdXAgYSBwcm9wb3NlZCBmaXggZm9yIHRo aXMsIGJ1dCBJIHdhbnRlZCB0byBnZXQgZmVlZGJhY2sgYXMgdG8gd2hldGhlciBpdCdzIHdvcnRo IHdvcnJ5aW5nIGFib3V0IGJlZm9yZSBzZW5kaW5nIHRoZSByZXZpZXcuICBEb2VzIGFueWJvZHkg aGF2ZSBhbnkgb3BpbmlvbnM/DQoNClRoYW5rcywNClJ5YW4NCg0K From nobody Wed Jan 19 14:02:35 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4187D1962DAD for ; Wed, 19 Jan 2022 14:02:54 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jf6mF3Xv3z3Dxv for ; Wed, 19 Jan 2022 14:02:53 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-pj1-x1035.google.com with SMTP id b1-20020a17090a990100b001b14bd47532so2774475pjp.0 for ; Wed, 19 Jan 2022 06:02:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=TFjt1sfpBVPbJYArYdS9utcMO6YoJrMYL4r7IpH5lek=; b=Ae3+IRRWPkVCnCHGu5LGV6xlq7mPHC2bW2EhadnDUVZdC7svRSUZ4Fcs5qWzQ1fC9k kdxsPerHeMj3jJqhkf1Sic18Y28PoSLh7IVMIsNm3aLrohS1oLDRk+tZ6G/n4SwvomEI b1gEv283eFlKigb5NFpQoiUNaSFW6IO9hmiww5xS9R+vScx5XxgSoThd9LidEODHT8a2 qE+27eJ2k9IKdpLpv2Rdh1jNaUo3mR4poZpejH5WBmOQrVxFTDn4L7idafhaphdHpiow Lxr8C860TTh7xX0olBReuuGqjsN1ak9H3syZcnM4wjAvogfTm1PUst6rnGOS7xaoaDI6 ofwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=TFjt1sfpBVPbJYArYdS9utcMO6YoJrMYL4r7IpH5lek=; b=yXJ/6M6uGdmyD8X/3QmX3PM0KMu6RGCT5k+1unywBfvUn0DiX/E3El3Pt+1h9GFerx 8b1N7WwcGmfp4xQJR3xUG7rUooRJwdyB9JWzQv4lXOBYQmAmncqw0j3b+KV1gzhF6U4V asJRtkiXI7VCXAgK9qB53y7n/yiM1KVp1oCO2WywLEcMZTncvOXsW1A1rbP+FAfk0E2B XQv9RuimOaHiqAcSu6rSR44iZPDka8u4RE+5KesJXQXAFbaf9QP5Ur4u3LJ+7ANoRMj6 GAFMC/dCVPMNKLH6//yEgtkhU2oAtQoShMRUIwygSCheD4LK8erskzmSH0BjMvDQKrRf HIGg== X-Gm-Message-State: AOAM530WysXTZ1h6yQJC9jczNmQBAgzeNBVmnn3ZWj4ztQlF17Pu/FCX DGJEiXLNhMgSY4/Eb/9P5z5iRPkMzafqMGg6FWyOr97D X-Google-Smtp-Source: ABdhPJylIm9NMUqAqmGvf/iTfodgyCdLOs5UJKvLyfwNlYiz0/IvF0zHCIqqWWgQVp34/QxkCB8PP5MQH+Isv6joYd8= X-Received: by 2002:a17:90b:33c7:: with SMTP id lk7mr4493646pjb.86.1642600966214; Wed, 19 Jan 2022 06:02:46 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Ryan Stone Date: Wed, 19 Jan 2022 09:02:35 -0500 Message-ID: Subject: Re: How should TCP LRO handle TH_PUSH? To: "Scheffenegger, Richard" Cc: "" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Jf6mF3Xv3z3Dxv X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=Ae3+IRRW; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rysto32@gmail.com designates 2607:f8b0:4864:20::1035 as permitted sender) smtp.mailfrom=rysto32@gmail.com X-Spamd-Result: default: False [-2.91 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.91)[-0.914]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1035:from]; MLMMJ_DEST(0.00)[freebsd-transport]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N The current behaviour is that if a flag other than PUSH/ACK is set, the current aggregated frame is sent up the stack and it starts working on a new one that begins with the packet with the unusual flags. However my reading of the code is that a subsequent packet with only PUSH or ACK set could be merged into that one. On Wed, Jan 19, 2022 at 2:33 AM Scheffenegger, Richard wrote: > > As discussed in the last transport call, can you please share the behavio= r with other TCP flags as well? > > URG is mostly historic > > ECE is tricky - presumably the flag of the last (in-sequence) packet shou= ld take precedence without AccECN. With AccECN, the new codepath where LRO = provides the full sequence of all received header bits is preferred. > CWR should be latched - if any packet in the LRO has it, it should be kep= t > PSH should (conceptually) be latched, even though the FBSD stack doesn't = do anything with it - other than possibly triggering an immediate ACK (good= to reduce the dependency on the delayed ACK timer). > > What is the behavior for the other flags (FIN, RST, SYN)? > > Best regards, > Richard > > > -----Original Message----- > From: owner-freebsd-transport@freebsd.org On Behalf Of Ryan Stone > Sent: Freitag, 7. J=C3=A4nner 2022 00:10 > To: > Subject: How should TCP LRO handle TH_PUSH? > > NetApp Security WARNING: This is an external email. Do not click links or= open attachments unless you recognize the sender and know the content is s= afe. > > > > > I've been working on writing unit tests for LRO (see my message to freebs= d-testing@ for more details on this). I've submitted reviews for two issue= s found by my tests that I believe to be outright bugs. > I did find one more issue where I'm not sure whether it's really a bug or= not. If LRO sees a TCP packet that does not have TH_PUSH set, and then me= rges a subsequent packet that does have TH_PUSH set into it, what should th= e value of the TH_PUSH flag in the merged packet be? > > When I wrote my test I expected to see TH_PUSH set, but that isn't our cu= rrent behaviour. On the one hand I'm not sure that this is strictly correc= t, but on the other hand I don't think we do anything with TH_PUSH on a rec= eived packet anyway. I did code up a proposed fix for this, but I wanted t= o get feedback as to whether it's worth worrying about before sending the r= eview. Does anybody have any opinions? > > Thanks, > Ryan > From nobody Tue Feb 8 18:58:09 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A889E19B9792 for ; Tue, 8 Feb 2022 18:58:15 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:13b:39f::9f:25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JtXMn6b4Sz4WG2 for ; Tue, 8 Feb 2022 18:58:13 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 966668D4A15D for ; Tue, 8 Feb 2022 18:58:12 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 360ECE70814 for ; Tue, 8 Feb 2022 18:58:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id eNvsRXhuJ3jY for ; Tue, 8 Feb 2022 18:58:10 +0000 (UTC) Received: from nv.sbone.de (nv.sbone.de [IPv6:fde9:577b:c1a9:31::2013:138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id A8E2BE707BB for ; Tue, 8 Feb 2022 18:58:10 +0000 (UTC) Date: Tue, 8 Feb 2022 18:58:09 +0000 (UTC) From: "Bjoern A. Zeeb" To: freebsd-transport@freebsd.org Subject: panic: syncache: mbuf too small Message-ID: X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Rspamd-Queue-Id: 4JtXMn6b4Sz4WG2 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 2a01:4f8:13b:39f::9f:25 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net X-Spamd-Result: default: False [-1.30 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:4f8:13b:39f::9f:25:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_NA(0.00)[zabbadoz.net]; NEURAL_SPAM_SHORT(1.00)[1.000]; MLMMJ_DEST(0.00)[freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hi, I just came to a console finding this. The tree is from a few days ago; is this known or should I investigate if it happens again? I sadly cannot dump on this machine. /bz db> show panic panic: syncache: mbuf too small db> where Tracing pid 0 tid 100014 td 0xfffffe000937a740 kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0 vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600 panic() at panic+0x43/frame 0xfffffe0007fa1660 syncache_respond() at syncache_respond+0x777/frame 0xfffffe0007fa1730 syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0 tcp_input_with_port() at tcp_input_with_port+0x14f5/frame 0xfffffe0007fa1a20 tcp6_input_with_port() at tcp6_input_with_port+0x69/frame 0xfffffe0007fa1a50 tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60 ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40 netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1ba0 ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0 ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1c30 netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1c90 ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0 iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00 _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40 gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame 0xfffffe0007fa1ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame 0xfffffe0007fa1ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- -- Bjoern A. Zeeb r15:7 From nobody Tue Feb 8 19:02:20 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 65C7819B9FA5 for ; Tue, 8 Feb 2022 19:02:38 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JtXSs5Nd7z4WhF for ; Tue, 8 Feb 2022 19:02:37 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: by mail-ej1-x634.google.com with SMTP id st12so532958ejc.4 for ; Tue, 08 Feb 2022 11:02:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=B9h18KMp1xT19rixhSigzw6KXuvZ1320YZR+DN7/rAY=; b=r1rRqpqe4jI/xypZysNWLrrGRJ9DBbIYoFb3HxxYd/7GjVc/y+M9MMZZhfNMTqxJYC Z+3kynS2SPJIWvg/6OtA+bn+EZ8JNd1x0AhkyJKd56D2jFiO1hr2FzQUhmnqjbzpabuj Omh8DXHgbb3zXEhNlSSI3q/0yko26a5EoDYLE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=B9h18KMp1xT19rixhSigzw6KXuvZ1320YZR+DN7/rAY=; b=D/nzD6RE+Z7U0aRERfHStEU/0S27w30q1w7/ffqdhYRFcWtHomXQGxGWsfIYwBU8F5 GDN8EQpJHvadY+CAE5Gfrg/GLASWql9DGuqZO04CEwJu8m9qsVoQnuL8w3QY0FJFFUbN gm7ACIKNWcK7GNqF9lyTRTWnxX7GSqe47wLNKNw0fBulPduOxk9iGx5+4aEvZqY4Ibna S4fBwlnwqRLKqdEP2XRpiu8DV0DHVCJstvnf0cAc84tnMYlJsx4RtoFcJxh/d4NrOGVd iD32b0f202IPRPvWTw9aXDpfez9pvNgGebf2Y4szw3Jy5H44M3f7fnwMRS+6WINdwIYr FbtA== X-Gm-Message-State: AOAM530Fgoku6YOuWCb01kl0axRLyJ3feRho3HC5UyNLkXXyz1tygJh1 JHXlRgXKo6EKOXGYWKbLALC89fere+Iq0f9QADxGy3Tq3A== X-Google-Smtp-Source: ABdhPJy/EwsqbQRIClVCooEjEVlwlPFg5ZB3q2R90Mk7IcfzQiyKTUumO3MwLOoyDNCjrvjn3DXqxKMFYrWXaa79C10= X-Received: by 2002:a17:907:7f21:: with SMTP id qf33mr4971041ejc.30.1644346951562; Tue, 08 Feb 2022 11:02:31 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Drew Gallatin Date: Tue, 8 Feb 2022 14:02:20 -0500 Message-ID: Subject: Re: panic: syncache: mbuf too small To: "Bjoern A. Zeeb" Cc: freebsd-transport@freebsd.org Content-Type: multipart/alternative; boundary="000000000000efd4e905d7865be8" X-Rspamd-Queue-Id: 4JtXSs5Nd7z4WhF X-Spamd-Bar: ------------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netflix.com header.s=google header.b=r1rRqpqe; dmarc=pass (policy=reject) header.from=netflix.com; spf=pass (mx1.freebsd.org: domain of gallatin@netflix.com designates 2a00:1450:4864:20::634 as permitted sender) smtp.mailfrom=gallatin@netflix.com X-Spamd-Result: default: False [-12.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[netflix.com:s=google]; FREEFALL_USER(0.00)[gallatin]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; NEURAL_SPAM_SHORT(1.00)[1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[netflix.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[netflix.com,reject]; WHITELIST_DMARC(-7.00)[netflix.com:D:+]; MLMMJ_DEST(0.00)[freebsd-transport]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::634:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; WHITELIST_SPF_DKIM(-3.00)[netflix.com:d:+,netflix.com:s:+] X-ThisMailContainsUnwantedMimeParts: N --000000000000efd4e905d7865be8 Content-Type: text/plain; charset="UTF-8" Can you examine max_linkhdr? Drew On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb < bzeeb-lists@lists.zabbadoz.net> wrote: > Hi, > > I just came to a console finding this. The tree is from a few days ago; > is this known or should I investigate if it happens again? I sadly cannot > dump on this machine. > > /bz > > db> show panic > panic: syncache: mbuf too small > db> where > Tracing pid 0 tid 100014 td 0xfffffe000937a740 > kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0 > vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600 > panic() at panic+0x43/frame 0xfffffe0007fa1660 > syncache_respond() at syncache_respond+0x777/frame 0xfffffe0007fa1730 > syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0 > tcp_input_with_port() at tcp_input_with_port+0x14f5/frame > 0xfffffe0007fa1a20 > tcp6_input_with_port() at tcp6_input_with_port+0x69/frame > 0xfffffe0007fa1a50 > tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60 > ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40 > netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1ba0 > ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0 > ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1c30 > netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1c90 > ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0 > iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00 > _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40 > gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame > 0xfffffe0007fa1ec0 > gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame > 0xfffffe0007fa1ef0 > fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1f30 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > > > -- > Bjoern A. Zeeb r15:7 > > --000000000000efd4e905d7865be8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Can you examine max_linkhdr?=C2=A0
Drew

On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb <= ;bzeeb-lists@lists.zabbad= oz.net> wrote:
Hi,

I just came to a console finding this.=C2=A0 The tree is from a few days ag= o;=C2=A0 is this known or should I investigate if it happens again?=C2=A0 = =C2=A0I sadly cannot dump on this machine.

/bz

db> show panic
panic: syncache: mbuf too small
db> where
Tracing pid 0 tid 100014 td 0xfffffe000937a740
kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0
vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600
panic() at panic+0x43/frame 0xfffffe0007fa1660
syncache_respond() at syncache_respond+0x777/frame 0xfffffe0007fa1730
syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0
tcp_input_with_port() at tcp_input_with_port+0x14f5/frame 0xfffffe0007fa1a2= 0
tcp6_input_with_port() at tcp6_input_with_port+0x69/frame 0xfffffe0007fa1a5= 0
tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60
ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40
netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1ba0<= br> ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0
ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1c30
netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1c90<= br> ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0
iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00
_task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40
gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame 0xfffffe0007fa1= ec0
gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame 0xfffffe0007f= a1ef0
fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1f30
--- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 ---


--
Bjoern A. Zeeb=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r15:7

--000000000000efd4e905d7865be8-- From nobody Tue Feb 8 19:25:59 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 283F219A790A for ; Tue, 8 Feb 2022 19:26:12 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JtY031Vfkz4fn5 for ; Tue, 8 Feb 2022 19:26:11 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 4AAEF8D4A214; Tue, 8 Feb 2022 19:26:03 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id C85BFE7082B; Tue, 8 Feb 2022 19:26:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id AAWykuP720nD; Tue, 8 Feb 2022 19:26:00 +0000 (UTC) Received: from nv.sbone.de (nv.sbone.de [IPv6:fde9:577b:c1a9:31::2013:138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 029C3E707BB; Tue, 8 Feb 2022 19:25:59 +0000 (UTC) Date: Tue, 8 Feb 2022 19:25:59 +0000 (UTC) From: "Bjoern A. Zeeb" To: Drew Gallatin cc: freebsd-transport@freebsd.org Subject: Re: panic: syncache: mbuf too small In-Reply-To: Message-ID: References: X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4JtY031Vfkz4fn5 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 195.201.62.131 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net X-Spamd-Result: default: False [-3.30 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.201.62.131:c]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zabbadoz.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Tue, 8 Feb 2022, Drew Gallatin wrote: > Can you examine max_linkhdr? Yes, was still sitting in ddb (thankfully watchdog got disabled): db> x max_linkhdr max_linkhdr: 58 And for consistency checks: db> x max_hdr max_hdr: 94 db> x max_datalen max_datalen: 14 db> x max_protohdr max_protohdr: 3c db> show reg cs 0x20 ds 0x3b es 0x3b fs 0x13 gs 0x1b ss 0x28 rax 0x12 rcx 0x1 rdx 0xffffffff811f6d0a rbx 0xffffffff812e614c rsp 0xfffffe0007fa15a0 rbp 0xfffffe0007fa15b0 rsi 0x80 rdi 0xffffffff81e8cec0 cnputs_mtx r8 0x10 r9 0x1d0 r10 0xffffffff81cfa820 vga_conssoftc r11 0x10 r12 0xffffffff812961ab r13 0x28 r14 0x100 r15 0xfffffe000937a740 rip 0xffffffff80c545a7 kdb_enter+0x37 rflags 0x86 kdb_enter+0x37: movq $0,0x1283a5e(%rip) Found a console log; the system was idle, right after a boot for a few minutes. It's a lab machine having booted off IPv4 (grml) but also having IPv6 on the network. According to terminal backlogs it was an incoming IPv6 ssh session likely to have triggered this. Always great if things are "idle" and only few people to ask. it is amd64; main @ 773e3a71b2f11d422694495aca988d4c7143601b from Jan 31st. /bz > Drew > > On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb < > bzeeb-lists@lists.zabbadoz.net> wrote: > >> Hi, >> >> I just came to a console finding this. The tree is from a few days ago; >> is this known or should I investigate if it happens again? I sadly cannot >> dump on this machine. >> >> /bz >> >> db> show panic >> panic: syncache: mbuf too small >> db> where >> Tracing pid 0 tid 100014 td 0xfffffe000937a740 >> kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0 >> vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600 >> panic() at panic+0x43/frame 0xfffffe0007fa1660 >> syncache_respond() at syncache_respond+0x777/frame 0xfffffe0007fa1730 >> syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0 >> tcp_input_with_port() at tcp_input_with_port+0x14f5/frame >> 0xfffffe0007fa1a20 >> tcp6_input_with_port() at tcp6_input_with_port+0x69/frame >> 0xfffffe0007fa1a50 >> tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60 >> ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40 >> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1ba0 >> ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0 >> ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1c30 >> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1c90 >> ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0 >> iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00 >> _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40 >> gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame >> 0xfffffe0007fa1ec0 >> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame >> 0xfffffe0007fa1ef0 >> fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1f30 >> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >> >> >> -- >> Bjoern A. Zeeb r15:7 >> >> > -- Bjoern A. Zeeb r15:7 From nobody Tue Feb 8 19:45:37 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1433519B5F85 for ; Tue, 8 Feb 2022 19:45:45 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JtYQY6jRCz4q0N for ; Tue, 8 Feb 2022 19:45:41 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id A01078D4A15D; Tue, 8 Feb 2022 19:45:40 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 39FD0E70814; Tue, 8 Feb 2022 19:45:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id NmVdfKg52uJY; Tue, 8 Feb 2022 19:45:38 +0000 (UTC) Received: from nv.sbone.de (nv.sbone.de [IPv6:fde9:577b:c1a9:31::2013:138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 24257E707BB; Tue, 8 Feb 2022 19:45:38 +0000 (UTC) Date: Tue, 8 Feb 2022 19:45:37 +0000 (UTC) From: "Bjoern A. Zeeb" To: Drew Gallatin cc: freebsd-transport@freebsd.org Subject: Re: panic: syncache: mbuf too small In-Reply-To: Message-ID: References: X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4JtYQY6jRCz4q0N X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 195.201.62.131 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net X-Spamd-Result: default: False [-3.30 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.201.62.131]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zabbadoz.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; BLOCKLISTDE_FAIL(0.00)[195.201.62.131:server fail]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Tue, 8 Feb 2022, Bjoern A. Zeeb wrote: > On Tue, 8 Feb 2022, Drew Gallatin wrote: > >> Can you examine max_linkhdr? > > Yes, was still sitting in ddb (thankfully watchdog got disabled): > > db> x max_linkhdr > max_linkhdr: 58 > > And for consistency checks: > > db> x max_hdr > max_hdr: 94 > db> x max_datalen > max_datalen: 14 > db> x max_protohdr > max_protohdr: 3c If I do the maths correctly: MHLEN = 168 (0x94 + 0x14) TCP_MAXHLEN = 60 - 24 = 36 TCP_MAXOLEN max_linkhdr = 88 168 - 88 - 36 = 44 ipv6_hdr size = 40 Leaves us with 4 for the tcp_header again? Which would be 24? Why would this not go kaboom all the time? Hmm I assume it's ieee80211_proto.c .. it changes max_linkhdr .. > db> show reg > cs 0x20 > ds 0x3b > es 0x3b > fs 0x13 > gs 0x1b > ss 0x28 > rax 0x12 > rcx 0x1 > rdx 0xffffffff811f6d0a > rbx 0xffffffff812e614c > rsp 0xfffffe0007fa15a0 > rbp 0xfffffe0007fa15b0 > rsi 0x80 > rdi 0xffffffff81e8cec0 cnputs_mtx > r8 0x10 > r9 0x1d0 > r10 0xffffffff81cfa820 vga_conssoftc > r11 0x10 > r12 0xffffffff812961ab > r13 0x28 > r14 0x100 > r15 0xfffffe000937a740 > rip 0xffffffff80c545a7 kdb_enter+0x37 > rflags 0x86 > kdb_enter+0x37: movq $0,0x1283a5e(%rip) > > Found a console log; the system was idle, right after a boot for a few > minutes. > It's a lab machine having booted off IPv4 (grml) but also having IPv6 on > the network. > > According to terminal backlogs it was an incoming IPv6 ssh session likely > to have triggered this. Always great if things are "idle" and only few > people > to ask. > > it is amd64; main @ 773e3a71b2f11d422694495aca988d4c7143601b from Jan 31st. > > /bz > > >> Drew >> >> On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb < >> bzeeb-lists@lists.zabbadoz.net> wrote: >> >>> Hi, >>> >>> I just came to a console finding this. The tree is from a few days ago; >>> is this known or should I investigate if it happens again? I sadly >>> cannot >>> dump on this machine. >>> >>> /bz >>> >>> db> show panic >>> panic: syncache: mbuf too small >>> db> where >>> Tracing pid 0 tid 100014 td 0xfffffe000937a740 >>> kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0 >>> vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600 >>> panic() at panic+0x43/frame 0xfffffe0007fa1660 >>> syncache_respond() at syncache_respond+0x777/frame 0xfffffe0007fa1730 >>> syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0 >>> tcp_input_with_port() at tcp_input_with_port+0x14f5/frame >>> 0xfffffe0007fa1a20 >>> tcp6_input_with_port() at tcp6_input_with_port+0x69/frame >>> 0xfffffe0007fa1a50 >>> tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60 >>> ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40 >>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1ba0 >>> ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0 >>> ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1c30 >>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xfffffe0007fa1c90 >>> ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0 >>> iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00 >>> _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40 >>> gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame >>> 0xfffffe0007fa1ec0 >>> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame >>> 0xfffffe0007fa1ef0 >>> fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1f30 >>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >>> >>> >>> -- >>> Bjoern A. Zeeb r15:7 >>> >>> >> > > -- Bjoern A. Zeeb r15:7 From nobody Tue Feb 8 20:56:23 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9492819B794D for ; Tue, 8 Feb 2022 20:56:37 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jtb0N5vVJz3qRP for ; Tue, 8 Feb 2022 20:56:36 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: by mail-ed1-x532.google.com with SMTP id cf2so886595edb.9 for ; Tue, 08 Feb 2022 12:56:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=CgMqtOPY1+ZVu+Ajk5Y6f+EtTZf4v7FuRap83g2cDBY=; b=OJ53wh20TPnRZ8Mf+A5FY2BuT87Xp5ZBorp4Cea3PFgYdYmhSuDq9Q1OCLVFe4xeS9 cJnUKFQuFj2ky/KeAeXrr7ZHMb90SnCLNl+BLFPN6ntdgxrzjBsLaXFNvuwA7ssQMbBy 9897KuPJP3MNRxPMv+Kq5MsusKTEkZa3/xoVQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=CgMqtOPY1+ZVu+Ajk5Y6f+EtTZf4v7FuRap83g2cDBY=; b=GdM3u3kM0vm5iQ8sSbYy5sEwnDW6+oNz3yVx9WZsjUAdlkeylK5/IwWMNLZL2f13lq pqmNMe/LLOy3M0OqkLSOxKgJ/rmdzq/JNvVAxQFttP4DnElepC0+ZHhWJYKsbVo4WYCY 9tCZmE/EryV55Jkts/cPHLodI/MfPJN9/MH3S9HNAbbZddOcQHoP4QPJnhEyLo+l4Vhq dVbj3PsaqVuQPj5VCeaTt/it9wrog2C5sH11Eu/EVppOVrRFcSajoSa8ITUcch9VnY43 HA5gilSS5c2wzYAr6HbwtaFwUh5wdv48F2oA9MOoimurA1nmUJkUA2Ib7cVZCkOQco6v 93GQ== X-Gm-Message-State: AOAM531BAmMyoUfG8NO2mAmqbCrHC66PyqRZz2PcOd23RXSxT/QakVPn ctnVdvTmfpFlPxXsNTjlZ8YsW+IutcrfuT8bWHlWk73tjETf X-Google-Smtp-Source: ABdhPJznKWxGfpjfiPTpE28v2OvUsi7p0B7XpsMO/qsAsqJVoNkX3wFweRwqzKOeqSShV2HwFO1njEHopAFOxN23edE= X-Received: by 2002:a05:6402:d0d:: with SMTP id eb13mr6409054edb.24.1644353795576; Tue, 08 Feb 2022 12:56:35 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Drew Gallatin Date: Tue, 8 Feb 2022 15:56:23 -0500 Message-ID: Subject: Re: panic: syncache: mbuf too small To: "Bjoern A. Zeeb" Cc: freebsd-transport@freebsd.org Content-Type: multipart/alternative; boundary="000000000000df782a05d787f354" X-Rspamd-Queue-Id: 4Jtb0N5vVJz3qRP X-Spamd-Bar: ------------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netflix.com header.s=google header.b=OJ53wh20; dmarc=pass (policy=reject) header.from=netflix.com; spf=pass (mx1.freebsd.org: domain of gallatin@netflix.com designates 2a00:1450:4864:20::532 as permitted sender) smtp.mailfrom=gallatin@netflix.com X-Spamd-Result: default: False [-12.42 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[netflix.com:s=google]; FREEFALL_USER(0.00)[gallatin]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; NEURAL_SPAM_SHORT(0.58)[0.576]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[netflix.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::532:from]; WHITELIST_DMARC(-7.00)[netflix.com:D:+]; DMARC_POLICY_ALLOW(-0.50)[netflix.com,reject]; MLMMJ_DEST(0.00)[freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; WHITELIST_SPF_DKIM(-3.00)[netflix.com:d:+,netflix.com:s:+] X-ThisMailContainsUnwantedMimeParts: N --000000000000df782a05d787f354 Content-Type: text/plain; charset="UTF-8" I suspect that it's ic->ic_headroom, which seems to be driver dependent. And that its going kaboom because of the combo of IPv6 plus some driver with a large ic_headroom.. It would be really unfortunate if we had to expand mbufs because of some wifi driver. Perhaps they could be taught to chain headers.. Drew On Tue, Feb 8, 2022 at 2:45 PM Bjoern A. Zeeb < bzeeb-lists@lists.zabbadoz.net> wrote: > On Tue, 8 Feb 2022, Bjoern A. Zeeb wrote: > > > On Tue, 8 Feb 2022, Drew Gallatin wrote: > > > >> Can you examine max_linkhdr? > > > > Yes, was still sitting in ddb (thankfully watchdog got disabled): > > > > db> x max_linkhdr > > max_linkhdr: 58 > > > > And for consistency checks: > > > > db> x max_hdr > > max_hdr: 94 > > db> x max_datalen > > max_datalen: 14 > > db> x max_protohdr > > max_protohdr: 3c > > If I do the maths correctly: > > MHLEN = 168 (0x94 + 0x14) > > TCP_MAXHLEN = 60 - 24 = 36 TCP_MAXOLEN > > max_linkhdr = 88 > > 168 - 88 - 36 = 44 > > ipv6_hdr size = 40 > > Leaves us with 4 for the tcp_header again? Which would be 24? > > > Why would this not go kaboom all the time? > > Hmm I assume it's ieee80211_proto.c .. it changes max_linkhdr .. > > > > > > > db> show reg > > cs 0x20 > > ds 0x3b > > es 0x3b > > fs 0x13 > > gs 0x1b > > ss 0x28 > > rax 0x12 > > rcx 0x1 > > rdx 0xffffffff811f6d0a > > rbx 0xffffffff812e614c > > rsp 0xfffffe0007fa15a0 > > rbp 0xfffffe0007fa15b0 > > rsi 0x80 > > rdi 0xffffffff81e8cec0 cnputs_mtx > > r8 0x10 > > r9 0x1d0 > > r10 0xffffffff81cfa820 vga_conssoftc > > r11 0x10 > > r12 0xffffffff812961ab > > r13 0x28 > > r14 0x100 > > r15 0xfffffe000937a740 > > rip 0xffffffff80c545a7 kdb_enter+0x37 > > rflags 0x86 > > kdb_enter+0x37: movq $0,0x1283a5e(%rip) > > > > Found a console log; the system was idle, right after a boot for a few > > minutes. > > It's a lab machine having booted off IPv4 (grml) but also having IPv6 on > > the network. > > > > According to terminal backlogs it was an incoming IPv6 ssh session likely > > to have triggered this. Always great if things are "idle" and only few > > people > > to ask. > > > > it is amd64; main @ 773e3a71b2f11d422694495aca988d4c7143601b from Jan > 31st. > > > > /bz > > > > > >> Drew > >> > >> On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb < > >> bzeeb-lists@lists.zabbadoz.net> wrote: > >> > >>> Hi, > >>> > >>> I just came to a console finding this. The tree is from a few days > ago; > >>> is this known or should I investigate if it happens again? I sadly > >>> cannot > >>> dump on this machine. > >>> > >>> /bz > >>> > >>> db> show panic > >>> panic: syncache: mbuf too small > >>> db> where > >>> Tracing pid 0 tid 100014 td 0xfffffe000937a740 > >>> kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0 > >>> vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600 > >>> panic() at panic+0x43/frame 0xfffffe0007fa1660 > >>> syncache_respond() at syncache_respond+0x777/frame 0xfffffe0007fa1730 > >>> syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0 > >>> tcp_input_with_port() at tcp_input_with_port+0x14f5/frame > >>> 0xfffffe0007fa1a20 > >>> tcp6_input_with_port() at tcp6_input_with_port+0x69/frame > >>> 0xfffffe0007fa1a50 > >>> tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60 > >>> ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40 > >>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame > 0xfffffe0007fa1ba0 > >>> ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0 > >>> ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1c30 > >>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame > 0xfffffe0007fa1c90 > >>> ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0 > >>> iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00 > >>> _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40 > >>> gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame > >>> 0xfffffe0007fa1ec0 > >>> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame > >>> 0xfffffe0007fa1ef0 > >>> fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30 > >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1f30 > >>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > >>> > >>> > >>> -- > >>> Bjoern A. Zeeb > r15:7 > >>> > >>> > >> > > > > > > -- > Bjoern A. Zeeb r15:7 > --000000000000df782a05d787f354 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I suspect that it's ic->ic_headroom, which see= ms to be driver dependent.=C2=A0 And that its going kaboom because of the c= ombo of IPv6 plus some driver with a large ic_headroom..

It would be really unfortunate if we had to expand mbufs because of = some wifi driver.=C2=A0=C2=A0 Perhaps they could be taught to chain headers= ..

Drew

On Tue, Feb 8, 2022 at 2:45 P= M Bjoern A. Zeeb <bzee= b-lists@lists.zabbadoz.net> wrote:
On Tue, 8 Feb 2022, Bjoern A. Zeeb wrote:

> On Tue, 8 Feb 2022, Drew Gallatin wrote:
>
>> Can you examine max_linkhdr?
>
> Yes, was still sitting in ddb (thankfully watchdog got disabled):
>
> db> x max_linkhdr
> max_linkhdr:=C2=A0 =C2=A0 58
>
> And for consistency checks:
>
> db> x max_hdr
> max_hdr:=C2=A0 =C2=A0 =C2=A0 =C2=A0 94
> db> x max_datalen
> max_datalen:=C2=A0 =C2=A0 14
> db> x max_protohdr
> max_protohdr:=C2=A0 =C2=A03c

If I do the maths correctly:

MHLEN =3D 168=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(0x94=C2=A0 + = 0x14)

TCP_MAXHLEN =3D 60 - 24 =3D 36 TCP_MAXOLEN

max_linkhdr =3D=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A088

168 - 88 - 36 =3D 44

ipv6_hdr size =3D 40

Leaves us with 4 for the tcp_header again?=C2=A0 Which would be 24?


Why would this not go kaboom all the time?

Hmm I assume it's ieee80211_proto.c .. it changes max_linkhdr ..





> db> show reg
> cs=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x20
> ds=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x3b
> es=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x3b
> fs=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x13
> gs=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x1b
> ss=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x28
> rax=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A00x12
> rcx=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x1
> rdx=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff811f6d0a
> rbx=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff812e614c
> rsp=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xfffffe0007fa15a0
> rbp=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xfffffe0007fa15b0
> rsi=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A00x80
> rdi=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff81e8cec0=C2=A0 cnputs_m= tx
> r8=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x10
> r9=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A00x1d0
> r10=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff81cfa820=C2=A0 vga_cons= softc
> r11=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A00x10
> r12=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff812961ab
> r13=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A00x28
> r14=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 0x100
> r15=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xfffffe000937a740
> rip=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff80c545a7=C2=A0 kdb_ente= r+0x37
> rflags=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 0x86
> kdb_enter+0x37: movq=C2=A0 =C2=A0 $0,0x1283a5e(%rip)
>
> Found a console log;=C2=A0 the system was idle, right after a boot for= a few
> minutes.
> It's a lab machine having booted off IPv4 (grml) but also having I= Pv6 on
> the network.
>
> According to terminal backlogs it was an incoming IPv6 ssh session lik= ely
> to have triggered this.=C2=A0 Always great if things are "idle&qu= ot; and only few
> people
> to ask.
>
> it is amd64;=C2=A0 main @ 773e3a71b2f11d422694495aca988d4c7143601b fro= m Jan 31st.
>
> /bz
>
>
>> Drew
>>
>> On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb <
>> bzeeb-lists@lists.zabbadoz.net> wrote:
>>
>>> Hi,
>>>
>>> I just came to a console finding this.=C2=A0 The tree is from = a few days ago;
>>> is this known or should I investigate if it happens again?=C2= =A0 =C2=A0I sadly
>>> cannot
>>> dump on this machine.
>>>
>>> /bz
>>>
>>> db> show panic
>>> panic: syncache: mbuf too small
>>> db> where
>>> Tracing pid 0 tid 100014 td 0xfffffe000937a740
>>> kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0
>>> vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600
>>> panic() at panic+0x43/frame 0xfffffe0007fa1660
>>> syncache_respond() at syncache_respond+0x777/frame 0xfffffe000= 7fa1730
>>> syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0<= br> >>> tcp_input_with_port() at tcp_input_with_port+0x14f5/frame
>>> 0xfffffe0007fa1a20
>>> tcp6_input_with_port() at tcp6_input_with_port+0x69/frame
>>> 0xfffffe0007fa1a50
>>> tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60
>>> ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40
>>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xffff= fe0007fa1ba0
>>> ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0 >>> ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1= c30
>>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame 0xffff= fe0007fa1c90
>>> ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0
>>> iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00 >>> _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40
>>> gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame >>> 0xfffffe0007fa1ec0
>>> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame<= br> >>> 0xfffffe0007fa1ef0
>>> fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30
>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1= f30
>>> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 ---
>>>
>>>
>>> --
>>> Bjoern A. Zeeb=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r15:7
>>>
>>>
>>
>
>

--
Bjoern A. Zeeb=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r15:7
--000000000000df782a05d787f354-- From nobody Tue Feb 8 22:14:12 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B515419C5BE6 for ; Tue, 8 Feb 2022 22:14:18 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jtck20WKFz4q88 for ; Tue, 8 Feb 2022 22:14:17 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 815CD8D4A15D; Tue, 8 Feb 2022 22:14:16 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 19C3FE70814; Tue, 8 Feb 2022 22:14:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id yiTb7-1rDmsN; Tue, 8 Feb 2022 22:14:13 +0000 (UTC) Received: from nv.sbone.de (nv.sbone.de [IPv6:fde9:577b:c1a9:31::2013:138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 416AFE707BB; Tue, 8 Feb 2022 22:14:13 +0000 (UTC) Date: Tue, 8 Feb 2022 22:14:12 +0000 (UTC) From: "Bjoern A. Zeeb" To: Drew Gallatin cc: freebsd-transport@freebsd.org Subject: Re: panic: syncache: mbuf too small In-Reply-To: Message-ID: References: X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4Jtck20WKFz4q88 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 195.201.62.131 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net X-Spamd-Result: default: False [-3.30 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.201.62.131]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zabbadoz.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Tue, 8 Feb 2022, Drew Gallatin wrote: > I suspect that it's ic->ic_headroom, which seems to be driver dependent. > And that its going kaboom because of the combo of IPv6 plus some driver > with a large ic_headroom.. Yeah, one of the Realtek drivers I was looking at sets it to 40/48 depending on chipset. Others vendor drivers are in the order of 26/28-ish max which would be an exact fit (without UDP tunneling)... > It would be really unfortunate if we had to expand mbufs because of some > wifi driver. Perhaps they could be taught to chain headers.. Realtek is doing a few "funny" things there; a lot of being single segment DMAs up-to 12k-ish .. not being helpful at all. I'll go and see if I can figure it out for this one specifically then *sigh*. For as long as no other drivers do similar things I am happy to work around it. Hmm bwi(4) is probably not much used anymore as from a quick glance that is also going big (82 by manual counting) and bwn(4) even more? So either our size massively shrunk in mbufs or that problem was there a decade ago already ... and we didn't notice? /bz > On Tue, Feb 8, 2022 at 2:45 PM Bjoern A. Zeeb < > bzeeb-lists@lists.zabbadoz.net> wrote: > >> On Tue, 8 Feb 2022, Bjoern A. Zeeb wrote: >> >>> On Tue, 8 Feb 2022, Drew Gallatin wrote: >>> >>>> Can you examine max_linkhdr? >>> >>> Yes, was still sitting in ddb (thankfully watchdog got disabled): >>> >>> db> x max_linkhdr >>> max_linkhdr: 58 >>> >>> And for consistency checks: >>> >>> db> x max_hdr >>> max_hdr: 94 >>> db> x max_datalen >>> max_datalen: 14 >>> db> x max_protohdr >>> max_protohdr: 3c >> >> If I do the maths correctly: >> >> MHLEN = 168 (0x94 + 0x14) >> >> TCP_MAXHLEN = 60 - 24 = 36 TCP_MAXOLEN >> >> max_linkhdr = 88 >> >> 168 - 88 - 36 = 44 >> >> ipv6_hdr size = 40 >> >> Leaves us with 4 for the tcp_header again? Which would be 24? >> >> >> Why would this not go kaboom all the time? >> >> Hmm I assume it's ieee80211_proto.c .. it changes max_linkhdr .. >> >> >> >> >> >>> db> show reg >>> cs 0x20 >>> ds 0x3b >>> es 0x3b >>> fs 0x13 >>> gs 0x1b >>> ss 0x28 >>> rax 0x12 >>> rcx 0x1 >>> rdx 0xffffffff811f6d0a >>> rbx 0xffffffff812e614c >>> rsp 0xfffffe0007fa15a0 >>> rbp 0xfffffe0007fa15b0 >>> rsi 0x80 >>> rdi 0xffffffff81e8cec0 cnputs_mtx >>> r8 0x10 >>> r9 0x1d0 >>> r10 0xffffffff81cfa820 vga_conssoftc >>> r11 0x10 >>> r12 0xffffffff812961ab >>> r13 0x28 >>> r14 0x100 >>> r15 0xfffffe000937a740 >>> rip 0xffffffff80c545a7 kdb_enter+0x37 >>> rflags 0x86 >>> kdb_enter+0x37: movq $0,0x1283a5e(%rip) >>> >>> Found a console log; the system was idle, right after a boot for a few >>> minutes. >>> It's a lab machine having booted off IPv4 (grml) but also having IPv6 on >>> the network. >>> >>> According to terminal backlogs it was an incoming IPv6 ssh session likely >>> to have triggered this. Always great if things are "idle" and only few >>> people >>> to ask. >>> >>> it is amd64; main @ 773e3a71b2f11d422694495aca988d4c7143601b from Jan >> 31st. >>> >>> /bz >>> >>> >>>> Drew >>>> >>>> On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb < >>>> bzeeb-lists@lists.zabbadoz.net> wrote: >>>> >>>>> Hi, >>>>> >>>>> I just came to a console finding this. The tree is from a few days >> ago; >>>>> is this known or should I investigate if it happens again? I sadly >>>>> cannot >>>>> dump on this machine. >>>>> >>>>> /bz >>>>> >>>>> db> show panic >>>>> panic: syncache: mbuf too small >>>>> db> where >>>>> Tracing pid 0 tid 100014 td 0xfffffe000937a740 >>>>> kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0 >>>>> vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600 >>>>> panic() at panic+0x43/frame 0xfffffe0007fa1660 >>>>> syncache_respond() at syncache_respond+0x777/frame 0xfffffe0007fa1730 >>>>> syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0 >>>>> tcp_input_with_port() at tcp_input_with_port+0x14f5/frame >>>>> 0xfffffe0007fa1a20 >>>>> tcp6_input_with_port() at tcp6_input_with_port+0x69/frame >>>>> 0xfffffe0007fa1a50 >>>>> tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60 >>>>> ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40 >>>>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame >> 0xfffffe0007fa1ba0 >>>>> ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0 >>>>> ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1c30 >>>>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame >> 0xfffffe0007fa1c90 >>>>> ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0 >>>>> iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00 >>>>> _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40 >>>>> gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame >>>>> 0xfffffe0007fa1ec0 >>>>> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame >>>>> 0xfffffe0007fa1ef0 >>>>> fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30 >>>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1f30 >>>>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- -- Bjoern A. Zeeb r15:7 From nobody Tue Feb 8 22:34:10 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id EEA7E19AB201 for ; Tue, 8 Feb 2022 22:34:23 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jtd9C1qnyz3DgJ for ; Tue, 8 Feb 2022 22:34:23 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: by mail-ej1-x636.google.com with SMTP id fy20so2027077ejc.0 for ; Tue, 08 Feb 2022 14:34:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Au9xsy+ehRs5h04FOeeasEpySwPeowcqWUgC1TGlupo=; b=KZ1nQ/yV9Nl77/OSQVjE1pQTlzhA8mW4rzBq0jGVhv6wq5TFeojNHf/Ru9yw5DGGbO QldGLZThTX2wXFPXZb1vXrAX6rPsd3q6nVWX7MNsHrdGmuy2GxmaT9lW6YAFQWbzbblt DN5xgYp+wZAzzk67kUb0MPZQOjAdG5Aj1N3no= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Au9xsy+ehRs5h04FOeeasEpySwPeowcqWUgC1TGlupo=; b=2ePHJF4cqKRrYT90KTm3UlI19ZiI2v+mUpTAwNFvz+QMwJE5LgYVlxsK7AxRrEQ4NC FfIwxDYOPgnQrXpKLGAspTXq504TaTHknLA7BrYU0d1uBRGvMx5X3F3HX4YTuUhw56Lr i6IgWYM+nFnlKpeqlsCXqCxdJFHgAedjjI3YzGym2TTjZ9Iz/gA6xUk7qaSSqhGsRCYT HI9lbDJqhVZoGIzJdHp62PWJasjl6tr3vgdOB9iztJ/MWpQ4lVA7StTpWLKT8RCIx2cA XynwNHMvMx905XSPZtEHL26yy2TF+He1pcCGW5J/IcHK0zPd/cJTBn0jHNCvYMj1pVDs hf+g== X-Gm-Message-State: AOAM530YSpeBcApnsydU5Aj76hH4gJjarR8UKY2yxO7vSO6jm648d8oz znrZIr3Na0r0mEmCbqWDqwxkw3ktmmSKEGDv0c6z1s3oyw== X-Google-Smtp-Source: ABdhPJwCZ7MKrUE6KHzPhaByaqvOSn4IBVOwLEr4+2WFuNL/NXFL7JXzN088/ikgTJh/HG9WNiEp04eUSU4eidN1IWM= X-Received: by 2002:a17:906:1804:: with SMTP id v4mr5388570eje.512.1644359662172; Tue, 08 Feb 2022 14:34:22 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Drew Gallatin Date: Tue, 8 Feb 2022 17:34:10 -0500 Message-ID: Subject: Re: panic: syncache: mbuf too small To: "Bjoern A. Zeeb" Cc: freebsd-transport@freebsd.org Content-Type: multipart/alternative; boundary="0000000000008c7f2505d78951b3" X-Rspamd-Queue-Id: 4Jtd9C1qnyz3DgJ X-Spamd-Bar: ------------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netflix.com header.s=google header.b="KZ1nQ/yV"; dmarc=pass (policy=reject) header.from=netflix.com; spf=pass (mx1.freebsd.org: domain of gallatin@netflix.com designates 2a00:1450:4864:20::636 as permitted sender) smtp.mailfrom=gallatin@netflix.com X-Spamd-Result: default: False [-12.68 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[netflix.com:s=google]; FREEFALL_USER(0.00)[gallatin]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; NEURAL_SPAM_SHORT(0.32)[0.323]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[netflix.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::636:from]; WHITELIST_DMARC(-7.00)[netflix.com:D:+]; DMARC_POLICY_ALLOW(-0.50)[netflix.com,reject]; MLMMJ_DEST(0.00)[freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; WHITELIST_SPF_DKIM(-3.00)[netflix.com:d:+,netflix.com:s:+] X-ThisMailContainsUnwantedMimeParts: N --0000000000008c7f2505d78951b3 Content-Type: text/plain; charset="UTF-8" I don't think the size has changed recently. However, there is a size difference for pkthdrs (and hence MHLEN) on 32-bit platforms vs 64-bit platforms. There are a number of bad ways to handle this. Eg, don't permit Ipv6 on these interfaces, make these interfaces chain their headers, assuming they can do s/g dma, make them copy to a contiguous buffer. Make mbufs bigger. All of the things I can think of are ugly. On Tue, Feb 8, 2022 at 5:14 PM Bjoern A. Zeeb < bzeeb-lists@lists.zabbadoz.net> wrote: > On Tue, 8 Feb 2022, Drew Gallatin wrote: > > > I suspect that it's ic->ic_headroom, which seems to be driver dependent. > > And that its going kaboom because of the combo of IPv6 plus some driver > > with a large ic_headroom.. > > Yeah, one of the Realtek drivers I was looking at sets it to 40/48 > depending on chipset. > > Others vendor drivers are in the order of 26/28-ish max which would be > an exact fit (without UDP tunneling)... > > > It would be really unfortunate if we had to expand mbufs because of some > > wifi driver. Perhaps they could be taught to chain headers.. > > Realtek is doing a few "funny" things there; a lot of being single > segment DMAs up-to 12k-ish .. not being helpful at all. > > I'll go and see if I can figure it out for this one specifically > then *sigh*. For as long as no other drivers do similar things > I am happy to work around it. > > > Hmm bwi(4) is probably not much used anymore as from a quick glance > that is also going big (82 by manual counting) and bwn(4) even more? > > So either our size massively shrunk in mbufs or that problem was there > a decade ago already ... and we didn't notice? > > > /bz > > > > On Tue, Feb 8, 2022 at 2:45 PM Bjoern A. Zeeb < > > bzeeb-lists@lists.zabbadoz.net> wrote: > > > >> On Tue, 8 Feb 2022, Bjoern A. Zeeb wrote: > >> > >>> On Tue, 8 Feb 2022, Drew Gallatin wrote: > >>> > >>>> Can you examine max_linkhdr? > >>> > >>> Yes, was still sitting in ddb (thankfully watchdog got disabled): > >>> > >>> db> x max_linkhdr > >>> max_linkhdr: 58 > >>> > >>> And for consistency checks: > >>> > >>> db> x max_hdr > >>> max_hdr: 94 > >>> db> x max_datalen > >>> max_datalen: 14 > >>> db> x max_protohdr > >>> max_protohdr: 3c > >> > >> If I do the maths correctly: > >> > >> MHLEN = 168 (0x94 + 0x14) > >> > >> TCP_MAXHLEN = 60 - 24 = 36 TCP_MAXOLEN > >> > >> max_linkhdr = 88 > >> > >> 168 - 88 - 36 = 44 > >> > >> ipv6_hdr size = 40 > >> > >> Leaves us with 4 for the tcp_header again? Which would be 24? > >> > >> > >> Why would this not go kaboom all the time? > >> > >> Hmm I assume it's ieee80211_proto.c .. it changes max_linkhdr .. > >> > >> > >> > >> > >> > >>> db> show reg > >>> cs 0x20 > >>> ds 0x3b > >>> es 0x3b > >>> fs 0x13 > >>> gs 0x1b > >>> ss 0x28 > >>> rax 0x12 > >>> rcx 0x1 > >>> rdx 0xffffffff811f6d0a > >>> rbx 0xffffffff812e614c > >>> rsp 0xfffffe0007fa15a0 > >>> rbp 0xfffffe0007fa15b0 > >>> rsi 0x80 > >>> rdi 0xffffffff81e8cec0 cnputs_mtx > >>> r8 0x10 > >>> r9 0x1d0 > >>> r10 0xffffffff81cfa820 vga_conssoftc > >>> r11 0x10 > >>> r12 0xffffffff812961ab > >>> r13 0x28 > >>> r14 0x100 > >>> r15 0xfffffe000937a740 > >>> rip 0xffffffff80c545a7 kdb_enter+0x37 > >>> rflags 0x86 > >>> kdb_enter+0x37: movq $0,0x1283a5e(%rip) > >>> > >>> Found a console log; the system was idle, right after a boot for a few > >>> minutes. > >>> It's a lab machine having booted off IPv4 (grml) but also having IPv6 > on > >>> the network. > >>> > >>> According to terminal backlogs it was an incoming IPv6 ssh session > likely > >>> to have triggered this. Always great if things are "idle" and only few > >>> people > >>> to ask. > >>> > >>> it is amd64; main @ 773e3a71b2f11d422694495aca988d4c7143601b from Jan > >> 31st. > >>> > >>> /bz > >>> > >>> > >>>> Drew > >>>> > >>>> On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb < > >>>> bzeeb-lists@lists.zabbadoz.net> wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> I just came to a console finding this. The tree is from a few days > >> ago; > >>>>> is this known or should I investigate if it happens again? I sadly > >>>>> cannot > >>>>> dump on this machine. > >>>>> > >>>>> /bz > >>>>> > >>>>> db> show panic > >>>>> panic: syncache: mbuf too small > >>>>> db> where > >>>>> Tracing pid 0 tid 100014 td 0xfffffe000937a740 > >>>>> kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0 > >>>>> vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600 > >>>>> panic() at panic+0x43/frame 0xfffffe0007fa1660 > >>>>> syncache_respond() at syncache_respond+0x777/frame 0xfffffe0007fa1730 > >>>>> syncache_add() at syncache_add+0xa71/frame 0xfffffe0007fa18c0 > >>>>> tcp_input_with_port() at tcp_input_with_port+0x14f5/frame > >>>>> 0xfffffe0007fa1a20 > >>>>> tcp6_input_with_port() at tcp6_input_with_port+0x69/frame > >>>>> 0xfffffe0007fa1a50 > >>>>> tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a60 > >>>>> ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b40 > >>>>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame > >> 0xfffffe0007fa1ba0 > >>>>> ether_demux() at ether_demux+0x16e/frame 0xfffffe0007fa1bd0 > >>>>> ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffffe0007fa1c30 > >>>>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/frame > >> 0xfffffe0007fa1c90 > >>>>> ether_input() at ether_input+0x99/frame 0xfffffe0007fa1cf0 > >>>>> iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007fa1e00 > >>>>> _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa1e40 > >>>>> gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame > >>>>> 0xfffffe0007fa1ec0 > >>>>> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame > >>>>> 0xfffffe0007fa1ef0 > >>>>> fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30 > >>>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0007fa1f30 > >>>>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > > -- > Bjoern A. Zeeb r15:7 > --0000000000008c7f2505d78951b3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I don't think the size has changed recently.=C2= =A0 However, there is a size difference for pkthdrs (and hence MHLEN) on 32= -bit platforms vs 64-bit platforms.

There are = a number of bad ways to handle this.=C2=A0 Eg, don't permit Ipv6 on the= se interfaces, make these interfaces chain their headers, assuming they can= do s/g dma, make them copy to a contiguous buffer.=C2=A0 Make mbufs bigger= . =C2=A0 All of the things I can think of are ugly.

On Tue, Feb 8,= 2022 at 5:14 PM Bjoern A. Zeeb <bzeeb-lists@lists.zabbadoz.net> wrote:<= br>
On Tue, 8 Feb 20= 22, Drew Gallatin wrote:

> I suspect that it's ic->ic_headroom, which seems to be driver d= ependent.
> And that its going kaboom because of the combo of IPv6 plus some drive= r
> with a large ic_headroom..

Yeah, one of the Realtek drivers I was looking at sets it to 40/48
depending on chipset.

Others vendor drivers are in the order of 26/28-ish max which would be
an exact fit (without UDP tunneling)...

> It would be really unfortunate if we had to expand mbufs because of so= me
> wifi driver.=C2=A0 =C2=A0Perhaps they could be taught to chain headers= ..

Realtek is doing a few "funny" things there; a lot of being singl= e
segment DMAs up-to 12k-ish .. not being helpful at all.

I'll go and see if I can figure it out for this one specifically
then *sigh*.=C2=A0 For as long as no other drivers do similar things
I am happy to work around it.


Hmm=C2=A0 bwi(4)=C2=A0 is probably not much used anymore as from a quick gl= ance
that is also going big (82 by manual counting) and bwn(4) even more?

So either our size massively shrunk in mbufs or that problem was there
a decade ago already ... and we didn't notice?


/bz


> On Tue, Feb 8, 2022 at 2:45 PM Bjoern A. Zeeb <
> bz= eeb-lists@lists.zabbadoz.net> wrote:
>
>> On Tue, 8 Feb 2022, Bjoern A. Zeeb wrote:
>>
>>> On Tue, 8 Feb 2022, Drew Gallatin wrote:
>>>
>>>> Can you examine max_linkhdr?
>>>
>>> Yes, was still sitting in ddb (thankfully watchdog got disable= d):
>>>
>>> db> x max_linkhdr
>>> max_linkhdr:=C2=A0 =C2=A0 58
>>>
>>> And for consistency checks:
>>>
>>> db> x max_hdr
>>> max_hdr:=C2=A0 =C2=A0 =C2=A0 =C2=A0 94
>>> db> x max_datalen
>>> max_datalen:=C2=A0 =C2=A0 14
>>> db> x max_protohdr
>>> max_protohdr:=C2=A0 =C2=A03c
>>
>> If I do the maths correctly:
>>
>> MHLEN =3D 168=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(0x94= =C2=A0 + 0x14)
>>
>> TCP_MAXHLEN =3D 60 - 24 =3D 36 TCP_MAXOLEN
>>
>> max_linkhdr =3D=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A088
>>
>> 168 - 88 - 36 =3D 44
>>
>> ipv6_hdr size =3D 40
>>
>> Leaves us with 4 for the tcp_header again?=C2=A0 Which would be 24= ?
>>
>>
>> Why would this not go kaboom all the time?
>>
>> Hmm I assume it's ieee80211_proto.c .. it changes max_linkhdr = ..
>>
>>
>>
>>
>>
>>> db> show reg
>>> cs=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 0x20
>>> ds=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 0x3b
>>> es=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 0x3b
>>> fs=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 0x13
>>> gs=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 0x1b
>>> ss=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 0x28
>>> rax=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A00x12
>>> rcx=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 0x1
>>> rdx=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff811f6d0a
>>> rbx=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff812e614c
>>> rsp=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xfffffe0007fa15a0
>>> rbp=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xfffffe0007fa15b0
>>> rsi=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A00x80
>>> rdi=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff81e8cec0=C2=A0 = cnputs_mtx
>>> r8=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 0x10
>>> r9=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A00x1d0
>>> r10=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff81cfa820=C2=A0 = vga_conssoftc
>>> r11=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A00x10
>>> r12=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff812961ab
>>> r13=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A00x28
>>> r14=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 0x100
>>> r15=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xfffffe000937a740
>>> rip=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00xffffffff80c545a7=C2=A0 = kdb_enter+0x37
>>> rflags=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 0x86
>>> kdb_enter+0x37: movq=C2=A0 =C2=A0 $0,0x1283a5e(%rip)
>>>
>>> Found a console log;=C2=A0 the system was idle, right after a = boot for a few
>>> minutes.
>>> It's a lab machine having booted off IPv4 (grml) but also = having IPv6 on
>>> the network.
>>>
>>> According to terminal backlogs it was an incoming IPv6 ssh ses= sion likely
>>> to have triggered this.=C2=A0 Always great if things are "= ;idle" and only few
>>> people
>>> to ask.
>>>
>>> it is amd64;=C2=A0 main @ 773e3a71b2f11d422694495aca988d4c7143= 601b from Jan
>> 31st.
>>>
>>> /bz
>>>
>>>
>>>> Drew
>>>>
>>>> On Tue, Feb 8, 2022 at 1:58 PM Bjoern A. Zeeb <
>>>> bzeeb-lists@lists.zabbadoz.net> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I just came to a console finding this.=C2=A0 The tree = is from a few days
>> ago;
>>>>> is this known or should I investigate if it happens ag= ain?=C2=A0 =C2=A0I sadly
>>>>> cannot
>>>>> dump on this machine.
>>>>>
>>>>> /bz
>>>>>
>>>>> db> show panic
>>>>> panic: syncache: mbuf too small
>>>>> db> where
>>>>> Tracing pid 0 tid 100014 td 0xfffffe000937a740
>>>>> kdb_enter() at kdb_enter+0x37/frame 0xfffffe0007fa15b0=
>>>>> vpanic() at vpanic+0x1b0/frame 0xfffffe0007fa1600
>>>>> panic() at panic+0x43/frame 0xfffffe0007fa1660
>>>>> syncache_respond() at syncache_respond+0x777/frame 0xf= ffffe0007fa1730
>>>>> syncache_add() at syncache_add+0xa71/frame 0xfffffe000= 7fa18c0
>>>>> tcp_input_with_port() at tcp_input_with_port+0x14f5/fr= ame
>>>>> 0xfffffe0007fa1a20
>>>>> tcp6_input_with_port() at tcp6_input_with_port+0x69/fr= ame
>>>>> 0xfffffe0007fa1a50
>>>>> tcp6_input() at tcp6_input+0xb/frame 0xfffffe0007fa1a6= 0
>>>>> ip6_input() at ip6_input+0xc2f/frame 0xfffffe0007fa1b4= 0
>>>>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/fram= e
>> 0xfffffe0007fa1ba0
>>>>> ether_demux() at ether_demux+0x16e/frame 0xfffffe0007f= a1bd0
>>>>> ether_nh_input() at ether_nh_input+0x3fc/frame 0xfffff= e0007fa1c30
>>>>> netisr_dispatch_src() at netisr_dispatch_src+0xaf/fram= e
>> 0xfffffe0007fa1c90
>>>>> ether_input() at ether_input+0x99/frame 0xfffffe0007fa= 1cf0
>>>>> iflib_rxeof() at iflib_rxeof+0xcb3/frame 0xfffffe0007f= a1e00
>>>>> _task_fn_rx() at _task_fn_rx+0x7a/frame 0xfffffe0007fa= 1e40
>>>>> gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/= frame
>>>>> 0xfffffe0007fa1ec0
>>>>> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc= 2/frame
>>>>> 0xfffffe0007fa1ef0
>>>>> fork_exit() at fork_exit+0x80/frame 0xfffffe0007fa1f30=
>>>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffff= e0007fa1f30
>>>>> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 ---

--
Bjoern A. Zeeb=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r15:7
--0000000000008c7f2505d78951b3-- From nobody Tue Feb 8 22:38:15 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6AC1919ACA68 for ; Tue, 8 Feb 2022 22:38:34 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JtdG154nQz3GGq for ; Tue, 8 Feb 2022 22:38:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-qt1-f175.google.com with SMTP id s1so379099qtw.9 for ; Tue, 08 Feb 2022 14:38:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=JFWXbELk4nkGeqzNtny5GzdordWTkJQ3fkwrtRO/ppM=; b=sZJonGvJMsSu3ZLe1M1BLFUT2x4ebrmSucfOHtDnlxxmnoeE+A7HKonExKUdw2iGMa ul+7xLWH9KRtR8Jhr/eyoEnIZu3HwZhSwIsqQuGbmnuuSyZthXUgdOxEdlaF4cnC4DpP ZVLfxYqLeGTe6ZwLpa1eMg9ptD8iwx8kh4iW9o3xeqvw1vBqV6B/PTtoWQe8k9rrrqzv yE78mg9pGx4y78wp7YEJVr7SNt4Q2S4+4JwxeOMoq63uaTSdOjU0DNK8Rv3xRpLw62n5 eMOC1/SIBlnNCNTDf4b7p906CaDoYkMfxaEVSqvzyZcVR8xvokwfPO2NpT7g8oPnt+gZ hPGw== X-Gm-Message-State: AOAM532I+zDHM7CJq3rjT8stH50GERi36SQWVC/ii76klsgFnZQFsYfS xZtlmakEECtjsG9QoCLDziglS0sER3EDzpTLluzhU1/a X-Google-Smtp-Source: ABdhPJy37bPGGWNphw67se3Pu13wAZH1NWqurVK1NLuzUXI3j98lX8oeS41esduD6SOWWBfiXOmCRp1GD6T7++9G/pw= X-Received: by 2002:ac8:5f48:: with SMTP id y8mr4478937qta.284.1644359907431; Tue, 08 Feb 2022 14:38:27 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Adrian Chadd Date: Tue, 8 Feb 2022 14:38:15 -0800 Message-ID: Subject: Re: panic: syncache: mbuf too small To: Drew Gallatin Cc: "Bjoern A. Zeeb" , "" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4JtdG154nQz3GGq X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of adrianchadd@gmail.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=adrianchadd@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[209.85.160.175:from]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-transport]; FORGED_SENDER(0.30)[adrian@freebsd.org,adrianchadd@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.160.175:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[adrian@freebsd.org,adrianchadd@gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N On Tue, 8 Feb 2022 at 14:34, Drew Gallatin wrote: > > I don't think the size has changed recently. However, there is a size di= fference for pkthdrs (and hence MHLEN) on 32-bit platforms vs 64-bit platfo= rms. > > There are a number of bad ways to handle this. Eg, don't permit Ipv6 on = these interfaces, make these interfaces chain their headers, assuming they = can do s/g dma, make them copy to a contiguous buffer. Make mbufs bigger. = All of the things I can think of are ugly. (a) yeah, some of the wifi firmware based things (bcom, realtek) have some weird ass requirements where the FW/PHY/MAC/802.11 headers end up needing to be in a contig buffer due to hardware/firmware requirements. You can sometimes chain things afterwards, but eg on the bwi/bwn firmware NICs you really need all of that data in the first sg buffer the firmware sees. (b) i think honestly we can just linearise the mbufs for the wifi drivers that need a big headroom nowdays, it's not a huge deal and in a lot of cases the linux drivers do? did? end up doing this to simplify handling. -adrian From nobody Wed Feb 9 17:09:57 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7785319ABD53 for ; Wed, 9 Feb 2022 17:10:10 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jv5wd6M0qz3GR9; Wed, 9 Feb 2022 17:10:09 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-pf1-x42f.google.com with SMTP id r19so5379706pfh.6; Wed, 09 Feb 2022 09:10:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=apezL0NSUr2FTFB6f4oF/gDGMRGo6sjL0h9XnOm8kPw=; b=NXolqbht514CQukdCQz0Ly1au5KBMblONCLUmoneEETHBEwJ9dvTFYpIR07GrQz6Sn 48sL1z6HFlVQ0+CUp+gqmfkDwo2poudbsHSKrct7glNG6l2FFyrrTn6AWPoo00n64AOa nQupMqax8YB0mzegF5TzV/ly08X4CwBWBgh8q9Z0uj4lKmNrS5vZy50ngOyTBsJCEoF1 Vo3QnlQm5BwHLJCnUmcdNftS5+shEwM54TM49rIZtUJtRmMR48y0ytO7/RxlHmrY/VWV Xz8iDxe9lIUJKsapgGOwuphTx/N6Kq2ixTMA6eYk9ihPKl28UXhKSBjsNHoow+IQbQIz gElQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=apezL0NSUr2FTFB6f4oF/gDGMRGo6sjL0h9XnOm8kPw=; b=3WrLJAOgZw7giWn9fT+aXP5RLAxtBbD5LAQyZ2B74dZEKc/jfOaE8PF00mW1PvA5OS mUGDngmYPNzEDWPmWlq7wHlu0CbeXNZXPEpQ7qP/khpTqPhoKNt8xvFbMnMx3GFwlpH6 Y6tkn7Vgz3EtIfGQG9Ou0CQ69WVEEYLPEZySh0HcjIggZkXUS9S6s1PQZsKNT0Ei9fwH lM5RMJul573SP/rcitffXUKzvXnBDOAg1daHrYJXsCbvfnSLuoY1cPjHOdWLf6PoDEbd szFkxUgr3HQ+iTPEwVPbKUz+1fynapOcj1o2yHQ0qA3KiJZd4pMWrMab/6x0yqgU94HD 9idg== X-Gm-Message-State: AOAM533dLK4xNCkRui7k8rHMZl3JVQTm1pb3tr8tF9IGoNAGXACrqzO+ CEYbf8rkgKyAfplSqSQYWU71mpZDQwRUOjCg2Hjqi5jG X-Google-Smtp-Source: ABdhPJwaUQasRBVPIAFUQVxQBpANfzTGlYO5dAvi4CeeSynzMHCAJVzg7PBO+XqLyjxtAz88ziUVwNX/cbbMEE/NNTQ= X-Received: by 2002:a63:1d0:: with SMTP id 199mr2652582pgb.469.1644426608309; Wed, 09 Feb 2022 09:10:08 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Ryan Stone Date: Wed, 9 Feb 2022 12:09:57 -0500 Message-ID: Subject: Re: panic: syncache: mbuf too small To: Adrian Chadd Cc: Drew Gallatin , "Bjoern A. Zeeb" , "" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4Jv5wd6M0qz3GR9 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=NXolqbht; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rysto32@gmail.com designates 2607:f8b0:4864:20::42f as permitted sender) smtp.mailfrom=rysto32@gmail.com X-Spamd-Result: default: False [-3.99 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::42f:from]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MLMMJ_DEST(0.00)[freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N Why not just allocate an mbuf cluster if the driver requires a huge amount of space for L2 headers? Or is it that we want to avoid the extra branch in the syncache path? Obviously a second allocation isn't ideal from a perf standpoint but if only weird WiFi drivers feel that pain I don't think it's that big of a deal, but perhaps I'm misunderstanding something. From nobody Wed Feb 9 17:13:45 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 07D3219AC185 for ; Wed, 9 Feb 2022 17:14:04 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jv6166jYxz3Gh7 for ; Wed, 9 Feb 2022 17:14:02 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: by mail-ej1-x633.google.com with SMTP id ka4so9189557ejc.11 for ; Wed, 09 Feb 2022 09:14:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mhc+vftDSkCsDfy1IzLbE1Ifq/xOtkXVtsO53rYT+Yk=; b=hJ5MRfpMpwJRPU2Is6ULo6SC3YCVGmZiyiB9u4I+dIYQ4hX7Ujs/kY9XAz1/kE7GTG c0cdY+6U0Bn6C7o3Wr1ohP9br7c9dxfsqWpUodMnVadTpH/wgxDuwwGDGQD60isr/DQN Zu9Kcr7/I/pIz+Krwyj8DXhJdfdj9ykUiB++w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mhc+vftDSkCsDfy1IzLbE1Ifq/xOtkXVtsO53rYT+Yk=; b=KHmfXLwIWxJHDMvddtVEKEHmtQNSfVdnKwWGs0zSYrmbf1MYjhZ2Knq0iuiJ/0sHYP l/Y2QqsiGNm0Fam6fzqfOkNLhlOEujwRiDSsUflSMNydBsF/uLTRRuhvHDd/sub3NDn8 olkIQ7EZqPQOx2PfQ1JDrx74E8VImdLBhT9WTKsRYpiWpTiBZPl/fwvaSkcYBss7ZgIG uKNMEoqyyzQDEjmmY8rnogMPgzLWOD+D2c71Sgn9Vng3JxQ1novGqOY6En1Eiu382Ds6 lr6byQbwppY3LxNGnj8d1ewIi6h5DGkJw35xZJQAB1FvU9TK3xT3S/bto2fuEjlObu40 IjwQ== X-Gm-Message-State: AOAM531PuBSvb3dXJtZCPvABIR4AJMV7OFM/J3SwGU8oDwH9DZeOAHUf gpnLm0tVIEkorMb+qHfyE1+MKWRAjikSVVLmTL8Wkko+Ysb6 X-Google-Smtp-Source: ABdhPJxY2r2iIVhMXCi4KWuecGEhspEGPo5FEpxBoEsctgG1a/b+6BLUZCRVgX0rfnFzRR6DWQ4x9Rj3GPZA5FBkV4I= X-Received: by 2002:a17:906:649c:: with SMTP id e28mr2599986ejm.324.1644426836294; Wed, 09 Feb 2022 09:13:56 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Drew Gallatin Date: Wed, 9 Feb 2022 12:13:45 -0500 Message-ID: Subject: Re: panic: syncache: mbuf too small To: Ryan Stone Cc: Adrian Chadd , "Bjoern A. Zeeb" , "" Content-Type: multipart/alternative; boundary="00000000000070321c05d798f54e" X-Rspamd-Queue-Id: 4Jv6166jYxz3Gh7 X-Spamd-Bar: ------------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netflix.com header.s=google header.b=hJ5MRfpM; dmarc=pass (policy=reject) header.from=netflix.com; spf=pass (mx1.freebsd.org: domain of gallatin@netflix.com designates 2a00:1450:4864:20::633 as permitted sender) smtp.mailfrom=gallatin@netflix.com X-Spamd-Result: default: False [-12.43 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[netflix.com:s=google]; FREEFALL_USER(0.00)[gallatin]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; NEURAL_SPAM_SHORT(0.57)[0.570]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[netflix.com:+]; DMARC_POLICY_ALLOW(-0.50)[netflix.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::633:from]; WHITELIST_DMARC(-7.00)[netflix.com:D:+]; MLMMJ_DEST(0.00)[freebsd-transport]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; WHITELIST_SPF_DKIM(-3.00)[netflix.com:d:+,netflix.com:s:+] X-ThisMailContainsUnwantedMimeParts: N --00000000000070321c05d798f54e Content-Type: text/plain; charset="UTF-8" Good point. It looks like TCP already does this for tcp_output(): #ifdef INET6 if (MHLEN < hdrlen + max_linkhdr) m = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); else #endif m = m_gethdr(M_NOWAIT, MT_DATA); On Wed, Feb 9, 2022 at 12:10 PM Ryan Stone wrote: > Why not just allocate an mbuf cluster if the driver requires a huge > amount of space for L2 headers? Or is it that we want to avoid the > extra branch in the syncache path? > > Obviously a second allocation isn't ideal from a perf standpoint but > if only weird WiFi drivers feel that pain I don't think it's that big > of a deal, but perhaps I'm misunderstanding something. > --00000000000070321c05d798f54e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Good point.=C2=A0 It looks like TCP already does this= for tcp_output():

#ifdef INET6
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (MHLEN < hdrlen + max_linkh= dr)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 m =3D m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR);
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 else
#endif
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 m= =3D m_gethdr(M_NOWAIT, MT_DATA);



On Wed, Feb 9, 2022 at 1= 2:10 PM Ryan Stone <rysto32@gmail.c= om> wrote:
; Thu, 10 Feb 2022 17:59:12 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 0601E8D4A15D; Thu, 10 Feb 2022 17:59:03 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 77C8DE707D9; Thu, 10 Feb 2022 17:59:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id KAdxRfbjFV-l; Thu, 10 Feb 2022 17:59:02 +0000 (UTC) Received: from nv.sbone.de (nv.sbone.de [IPv6:fde9:577b:c1a9:31::2013:138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id CA49BE707B1; Thu, 10 Feb 2022 17:59:01 +0000 (UTC) Date: Thu, 10 Feb 2022 17:59:00 +0000 (UTC) From: "Bjoern A. Zeeb" To: Drew Gallatin cc: Ryan Stone , "" Subject: Re: panic: syncache: mbuf too small In-Reply-To: Message-ID: References: X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4Jvkym1RdMz3pvJ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 2a01:4f8:13b:39f::9f:25 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net X-Spamd-Result: default: False [-3.30 / 15.00]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a01:4f8:13b:39f::9f:25]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zabbadoz.net]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org] X-ThisMailContainsUnwantedMimeParts: N On Wed, 9 Feb 2022, Drew Gallatin wrote: > Good point. It looks like TCP already does this for tcp_output(): > > #ifdef INET6 > if (MHLEN < hdrlen + max_linkhdr) > m = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); > else > #endif > m = m_gethdr(M_NOWAIT, MT_DATA); > A few comments: (a) there's no reason anymore (especially after the bwn/bwi numbers I saw) that the above is limited to IPv6 so that #ifdef should go; and without looking at the code I'd hope there'd be a follow-up length check. (b) if one wireless driver bumps max_linkhdr on the system it bumps it for everything; in theory we could do it on a per-interface basis but that's just route lookups and all kinds of things probably not in the right place these days and more complications. (c) despite (b) of course a "server" w/o wireless or a machine without a problemtic driver would currently not hit the more pessimistic path and be good with m_gethdr() in these cases. (d) I currently can avoid (and successfully did) this with the rtw88 which triggered it for me; eventually I'll have to find a better solution for it but for the next weeks I'll be good. (e) So even if we are good for now, looking through mbuf.h made me "cry" the other day. It's very well engineered but the simplicity it once had is long gone; but that remains a story for another day (maybe the 2nd half of this year). /bz -- Bjoern A. Zeeb r15:7 From nobody Thu Feb 10 18:33:20 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4629F19C6170 for ; Thu, 10 Feb 2022 18:33:34 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JvlkP2yVvz4YJb for ; Thu, 10 Feb 2022 18:33:33 +0000 (UTC) (envelope-from gallatin@netflix.com) Received: by mail-ej1-x62e.google.com with SMTP id p15so17417305ejc.7 for ; Thu, 10 Feb 2022 10:33:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=pD1wc95AaGneMf+TouAS0fQQK5GNuRKWHjf7r75lEHk=; b=NomOcudAX1odH/Rnr6fONeTr73p+lUVNqzDu1v1KFoVY/ULFT1vIN8WiOsrFIEt1AT OuaqpMHaIoe3KIPVKSOe6FvYo1tXz1y/er4B0cGFnpok9sMgn1j1PLMfLoZg+YgbYa4q pW0/QX/Ia8fX3uc8XAsElmIPWRruUh5ebkv+k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pD1wc95AaGneMf+TouAS0fQQK5GNuRKWHjf7r75lEHk=; b=V0LztFO8qiLoVDg8zwZmPqg0L1Z1ZavIs8jE/3QTdY0vZvboJhmCtzShfavuNXb+dP x5aGFHDN7F9sNO41tWY+r/M29MISFxRnatFRwBFWFgqOj+AhoYB6NTgqUxyq3X9iHdE6 Tf2LXWX8BoivwVcel1+kBvDYCc4KF8QIQDRjLiDLijjssRZiMxct7qvMaC0NMTPmDiUb Gq2gAN7djCHN7OX0hwjMGRTntEAvi9rs3PTDkuPPcjlLufKGPs3wkmTrhhfYD6+1wnRt hK29SyAoCIdpaEz98WJbia6NVPfglqVCnBVA/25rd8bi0Hy9kdtWnYm8OX/7JOibnmuN U4wQ== X-Gm-Message-State: AOAM532VBycCnraaUlIMdLtw8abrtsybTP92yrRoqazNVfZ8OwhCdrCm kU9+DwNi38Om+oo/0Qt2RDzTsw2DA144YwyiibOz X-Google-Smtp-Source: ABdhPJzD38JywfIFigEcM2hrQDs7ApH9DrO4tjgk58zrnCu0wnRYGnaAfhSCPX3kyUw2sAGAlimyL0i2O5ZgHurIqjM= X-Received: by 2002:a17:906:58c7:: with SMTP id e7mr7074900ejs.743.1644518012187; Thu, 10 Feb 2022 10:33:32 -0800 (PST) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Drew Gallatin Date: Thu, 10 Feb 2022 13:33:20 -0500 Message-ID: Subject: Re: panic: syncache: mbuf too small To: "Bjoern A. Zeeb" Cc: Ryan Stone , "" Content-Type: multipart/alternative; boundary="000000000000f21e1405d7ae2fc8" X-Rspamd-Queue-Id: 4JvlkP2yVvz4YJb X-Spamd-Bar: ------------- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netflix.com header.s=google header.b=NomOcudA; dmarc=pass (policy=reject) header.from=netflix.com; spf=pass (mx1.freebsd.org: domain of gallatin@netflix.com designates 2a00:1450:4864:20::62e as permitted sender) smtp.mailfrom=gallatin@netflix.com X-Spamd-Result: default: False [-14.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[netflix.com:s=google]; FREEFALL_USER(0.00)[gallatin]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; WHITELIST_SPF_DKIM(-3.00)[netflix.com:d:+,netflix.com:s:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[netflix.com:+]; DMARC_POLICY_ALLOW(-0.50)[netflix.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62e:from]; WHITELIST_DMARC(-7.00)[netflix.com:D:+]; MLMMJ_DEST(0.00)[freebsd-transport]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org] X-ThisMailContainsUnwantedMimeParts: N --000000000000f21e1405d7ae2fc8 Content-Type: text/plain; charset="UTF-8" On Thu, Feb 10, 2022 at 12:59 PM Bjoern A. Zeeb < bzeeb-lists@lists.zabbadoz.net> wrote: > On Wed, 9 Feb 2022, Drew Gallatin wrote: > > > Good point. It looks like TCP already does this for tcp_output(): > > > > #ifdef INET6 > > if (MHLEN < hdrlen + max_linkhdr) > > m = m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR); > > else > > #endif > > m = m_gethdr(M_NOWAIT, MT_DATA); > > > > A few comments: > > (a) there's no reason anymore (especially after the bwn/bwi numbers I > saw) that the above is limited to IPv6 so that #ifdef should go; and > without looking at the code I'd hope there'd be a follow-up length > check. > > I agree, and I doubt people have noticed because the vast majority don't compile out IPv6. I personally hate all these options; it makes reading and testing code a PITA since there are essentially 4 different cases to consider in every part of the network stack. If we're so concerned about the size, it would be better to stub IPv4 and IPv6 functions into noop inlines than have ifdefs everywhere. <...> (e) So even if we are good for now, looking through mbuf.h made me > "cry" the other day. It's very well engineered but the simplicity it > once had is long gone; but that remains a story for another day (maybe > the 2nd half of this year). > What, exactly, made you sad? What we have now is carefully optimized for hot-path performance and to keep all hot-path memory accesses within the first 2 cache lines. Part of what increased the complexity was the addition of M_EXTPG, and I'm sorry for that, but wow does it help performance. At least we're still quite a bit simpler than skbuffs :) If you plan any changes, I'd really appreciate it if you could include me (and glebius@) in the early design phases. Drew --000000000000f21e1405d7ae2fc8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Feb 10, 2022 at 12:59 PM Bjoe= rn A. Zeeb <bzeeb-list= s@lists.zabbadoz.net> wrote:
On Wed, 9 Feb 2022, Drew Gallatin wrote:

> Good point.=C2=A0 It looks like TCP already does this for tcp_output()= :
>
> #ifdef INET6
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (MHLEN < = hdrlen + max_linkhdr)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 m =3D m_getcl(M_NOWAIT, MT_DATA, M_PKTHDR);
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 else
> #endif
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 m =3D m_gethdr(M_NOWAIT, MT_DATA);
>

A few comments:

(a) there's no reason anymore (especially after the bwn/bwi numbers I saw) that the above is limited to IPv6 so that #ifdef should go; and
without looking at the code I'd hope there'd be a follow-up length<= br> check.


I agree, and I doubt people have notic= ed because the vast majority don't compile out IPv6.=C2=A0 I personally= hate all these options; it makes reading and testing code a PITA since the= re are essentially 4 different cases to consider in every part of the netwo= rk stack.=C2=A0 If we're so concerned about the size, it would be bette= r to stub IPv4 and IPv6 functions into noop inlines than have ifdefs everyw= here.
=C2=A0
<...>

(e) So even if we are good for now, looking through mbuf.h made me
"cry" the other day.=C2=A0 It's very well engineered but the = simplicity it
once had is long gone; but that remains a story for another day (maybe
the 2nd half of this year).

What, exact= ly, made you sad?

What we have now is carefully op= timized for hot-path performance and to keep all hot-path memory accesses w= ithin the first 2 cache lines.=C2=A0=C2=A0 Part of what increased the compl= exity was the addition of M_EXTPG, and I'm sorry for that, but wow does= it help performance.=C2=A0 At least we're still quite a bit simpler th= an skbuffs :)

If you plan any changes, I'd rea= lly appreciate it if you could include me (and glebius@) in the early desig= n phases.

Drew
--000000000000f21e1405d7ae2fc8-- From nobody Sun Jun 12 16:40:11 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DFA05857237; Sun, 12 Jun 2022 16:40:14 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.105.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LLgRK6lTgz4bqb; Sun, 12 Jun 2022 16:40:13 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-45-159-88.washdc.fios.verizon.net [108.45.159.88]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Sun, 12 Jun 2022 16:40:12 +0000 Received: from smtpclient.apple ( [2001:470:e24c:200:8852:3b29:50e0:32ca]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id ec370aad (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Sun, 12 Jun 2022 12:40:11 -0400 (EDT) From: Chris Ross Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.60.0.1.1\)) Subject: Troubles with adding IPv6 support to a program Message-Id: Date: Sun, 12 Jun 2022 12:40:11 -0400 To: freebsd-net@freebsd.org, freebsd-transport@freebsd.org X-Mailer: Apple Mail (2.3693.60.0.1.1) X-Rspamd-Queue-Id: 4LLgRK6lTgz4bqb X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of cross@distal.com designates 168.235.105.136 as permitted sender) smtp.mailfrom=cross@distal.com X-Spamd-Result: default: False [-1.65 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[distal.com]; R_SPF_ALLOW(-0.20)[+a:relay.dynu.com]; NEURAL_HAM_LONG(-1.00)[-0.998]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_SPAM_SHORT(0.15)[0.150]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-transport,freebsd-net]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3842, ipnet:168.235.104.0/22, country:US]; TAGGED_FROM(0.00)[freebsd]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[108.45.159.88:received] X-ThisMailContainsUnwantedMimeParts: N Tl;dr; I don=E2=80=99t know why I=E2=80=99m getting an EINVAL from a call to = bind for a second socket 20 years ago, I spent a lot of time adding IPv6 support to IPv4-only = programs. So I thought this (adding IPv6 support to simpleproxy[1]) would be an = easy project to pick up. I=E2=80=99ve gotten most of the framework in, = delayed only slightly when I learned that FreeBSD is =E2=80=9Cdifferent=E2=80=9D in = not routing IPv4 traffic to AF_INET6 sockets [2]. However, I=E2=80=99m getting an issue from one of my bind(2) calls that = I=E2=80=99m not able to figure out. I was hoping someone else had a few minutes to take a = quick look and help me find out what I=E2=80=99m doing wrong. A ~70 line relevant section of the source is at = https://justpaste.it/6u3jd The tl;dr; of this is that I: * getaddrinfo(NULL, portIwant) * for each address returned: * create a socket * setsockopt(SO_REUSEADDR) * bind * listen The first address I get is IPv6 localhost, and that binds and listens, = then continues to the next address. A new socket is created, setsockopt=E2=80=99= d, but bind for the second address (IPv4 localhost) fails with EINVAL. As you can see in the code shared, I have lots of debugging logs in the = code, and it looks like everything is as it should be. I=E2=80=99m probably = just missing Something simple, and am looking for another set of eyes. Thanks all. Contact me off list if you like, or on-list if it=E2=80=99s = obvious what I=E2=80=99ve done and it will help others. - Chris [1] https://github.com/vzaliva/simpleproxy [2] inet6(4), "Interaction between IPv4/v6 sockets"= From nobody Sun Jun 12 17:19:35 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 199A985DB03; Sun, 12 Jun 2022 17:19:40 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.105.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LLhJq2Nhjz4j0s; Sun, 12 Jun 2022 17:19:39 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-45-159-88.washdc.fios.verizon.net [108.45.159.88]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Sun, 12 Jun 2022 17:19:38 +0000 Received: from smtpclient.apple ( [2001:470:e24c:200:8852:3b29:50e0:32ca]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id 9d9df02e (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Sun, 12 Jun 2022 13:19:36 -0400 (EDT) From: Chris Ross Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.60.0.1.1\)) Subject: Re: Troubles with adding IPv6 support to a program Date: Sun, 12 Jun 2022 13:19:35 -0400 References: To: freebsd-net@freebsd.org, freebsd-transport@freebsd.org In-Reply-To: Message-Id: <008DAE56-BC4D-4F62-950F-1072C282884D@distal.com> X-Mailer: Apple Mail (2.3693.60.0.1.1) X-Rspamd-Queue-Id: 4LLhJq2Nhjz4j0s X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of cross@distal.com designates 168.235.105.136 as permitted sender) smtp.mailfrom=cross@distal.com X-Spamd-Result: default: False [-1.18 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[distal.com]; R_SPF_ALLOW(-0.20)[+a:relay.dynu.com]; NEURAL_HAM_LONG(-1.00)[-0.997]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_SPAM_SHORT(0.62)[0.619]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-transport,freebsd-net]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3842, ipnet:168.235.104.0/22, country:US]; TAGGED_FROM(0.00)[freebsd]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[108.45.159.88:received] X-ThisMailContainsUnwantedMimeParts: N Okay. So, anyone can feel free to look at my code at the link below, = but=20 as soon as I got desperate enough to reach out to a mailer, I found my own mistake. For the curious, inside my loop I was calling socket() with the original pointer, not the iterator. :-( So, calling socket() the same way twice despite thinking I wasn=E2=80=99t. Sorry for the noise. - Chris > On Jun 12, 2022, at 12:40, Chris Ross = wrote: >=20 >=20 > Tl;dr; > I don=E2=80=99t know why I=E2=80=99m getting an EINVAL from a call to = bind for a second socket >=20 >=20 >=20 >=20 > 20 years ago, I spent a lot of time adding IPv6 support to IPv4-only = programs. > So I thought this (adding IPv6 support to simpleproxy[1]) would be an = easy > project to pick up. I=E2=80=99ve gotten most of the framework in, = delayed only > slightly when I learned that FreeBSD is =E2=80=9Cdifferent=E2=80=9D in = not routing IPv4 > traffic to AF_INET6 sockets [2]. >=20 > However, I=E2=80=99m getting an issue from one of my bind(2) calls = that I=E2=80=99m not able > to figure out. I was hoping someone else had a few minutes to take a = quick > look and help me find out what I=E2=80=99m doing wrong. >=20 > A ~70 line relevant section of the source is at = https://justpaste.it/6u3jd >=20 > The tl;dr; of this is that I: >=20 > * getaddrinfo(NULL, portIwant) > * for each address returned: > * create a socket > * setsockopt(SO_REUSEADDR) > * bind > * listen >=20 > The first address I get is IPv6 localhost, and that binds and listens, = then > continues to the next address. A new socket is created, = setsockopt=E2=80=99d, but > bind for the second address (IPv4 localhost) fails with EINVAL. >=20 > As you can see in the code shared, I have lots of debugging logs in = the code, > and it looks like everything is as it should be. I=E2=80=99m probably = just missing > Something simple, and am looking for another set of eyes. >=20 > Thanks all. Contact me off list if you like, or on-list if it=E2=80=99s= obvious > what I=E2=80=99ve done and it will help others. >=20 > - Chris >=20 >=20 > [1] https://github.com/vzaliva/simpleproxy > [2] inet6(4), "Interaction between IPv4/v6 sockets" >=20 From nobody Sun Jun 12 17:21:12 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E0FC685E3AE; Sun, 12 Jun 2022 17:21:24 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LLhLr0z1Nz4jmr; Sun, 12 Jun 2022 17:21:24 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:c1b4:b153:a9c1:8f80]) (Authenticated sender: macmic) by drew.franken.de (Postfix) with ESMTPSA id F03C67446B3C6; Sun, 12 Jun 2022 19:21:13 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\)) Subject: Re: Troubles with adding IPv6 support to a program From: tuexen@freebsd.org In-Reply-To: Date: Sun, 12 Jun 2022 19:21:12 +0200 Cc: freebsd-net , freebsd-transport@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <91695911-4B92-4D3D-A532-9D51B8ACB22F@freebsd.org> References: To: Chris Ross X-Mailer: Apple Mail (2.3696.100.31) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: 4LLhLr0z1Nz4jmr X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 193.175.24.27 is neither permitted nor denied by domain of tuexen@freebsd.org) smtp.mailfrom=tuexen@freebsd.org X-Spamd-Result: default: False [-2.05 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEFALL_USER(0.00)[tuexen]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_RCPT(0.00)[freebsd]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.35)[-0.351]; FROM_NO_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-net,freebsd-transport]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[193.175.24.27:from] X-ThisMailContainsUnwantedMimeParts: N > On 12. Jun 2022, at 18:40, Chris Ross = wrote: >=20 >=20 > Tl;dr; > I don=E2=80=99t know why I=E2=80=99m getting an EINVAL from a call to = bind for a second socket >=20 >=20 >=20 >=20 > 20 years ago, I spent a lot of time adding IPv6 support to IPv4-only = programs. > So I thought this (adding IPv6 support to simpleproxy[1]) would be an = easy > project to pick up. I=E2=80=99ve gotten most of the framework in, = delayed only > slightly when I learned that FreeBSD is =E2=80=9Cdifferent=E2=80=9D in = not routing IPv4 > traffic to AF_INET6 sockets [2]. >=20 > However, I=E2=80=99m getting an issue from one of my bind(2) calls = that I=E2=80=99m not able > to figure out. I was hoping someone else had a few minutes to take a = quick > look and help me find out what I=E2=80=99m doing wrong. >=20 > A ~70 line relevant section of the source is at = https://justpaste.it/6u3jd >=20 > The tl;dr; of this is that I: >=20 > * getaddrinfo(NULL, portIwant) > * for each address returned: > * create a socket > * setsockopt(SO_REUSEADDR) > * bind > * listen >=20 > The first address I get is IPv6 localhost, and that binds and listens, = then > continues to the next address. A new socket is created, = setsockopt=E2=80=99d, but > bind for the second address (IPv4 localhost) fails with EINVAL. >=20 > As you can see in the code shared, I have lots of debugging logs in = the code, > and it looks like everything is as it should be. I=E2=80=99m probably = just missing > Something simple, and am looking for another set of eyes. >=20 > Thanks all. Contact me off list if you like, or on-list if it=E2=80=99s= obvious > what I=E2=80=99ve done and it will help others. Can you provide the output of truss whatever_you_need_to_call_your_program which shows the socket call and the failing bind call? Best regards Michael >=20 > - Chris >=20 >=20 > [1] https://github.com/vzaliva/simpleproxy > [2] inet6(4), "Interaction between IPv4/v6 sockets" >=20 From nobody Wed Jul 6 08:43:07 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5588B1D0637B for ; Wed, 6 Jul 2022 08:43:21 +0000 (UTC) (envelope-from tdtemccna@gmail.com) Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LdCk03vFSz4sM2 for ; Wed, 6 Jul 2022 08:43:20 +0000 (UTC) (envelope-from tdtemccna@gmail.com) Received: by mail-lf1-x132.google.com with SMTP id y16so24662759lfb.9 for ; Wed, 06 Jul 2022 01:43:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to:cc; bh=qNJS8bXJjIJEOjJk4e4xeTOXLNTgbklFiRogoFnIe/0=; b=kvJeOMK214pVGP9eKMnqgXywNLQUcTFxrpuCwwp8abOMhhj1pdQhdo3MhGyjjfyYCK xhE2rLFcwoLbu0LRH/r7opg+K8EzKIuTpjqnHf3rjErU5y1HV6B4Z9BTVQFbRPEZdpLu Fie1RZiqw5HvlC4mJG2tb0C33TEOD58ko2p1HR4DoYmnnkV85tQmEOOlGhM9Eb5OaD18 YFscFtN1XRXtdOdai+KvzdQ5yXyRw09eyFqiTIzV/XGNoyQaBDiBN4giM1RhMySG3l/C deuqM+SIp9x0qjbGm0jUvIU+i7rMJU2xa2ntJHk6bQQhaxtNQKQ6AG5b3JKh4B0ppOQi Hs3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=qNJS8bXJjIJEOjJk4e4xeTOXLNTgbklFiRogoFnIe/0=; b=C8YWvFDHJ+YbUWi2jCpDjsLLLJtDRjX+V35KUZWyWGidcZUDOffNZN0MQ5gOoUBwzK w3yghIP7ChgGSO+1pfp9mOyu32kM06EstnGEOWD9SNly6Sd3NK4l3ovShm/unKxBFYpB tBvgLzd6g4uYtzcyH7I/htjmjrr1e3PMvHHaTJGrvUqJqtUtz6wBRnM43s2ysJvZhsQG meWxpXftc9aMioJktvwUIeMtYYyiFNo30vaGxx6Xi+aWf7G1hIG1F2lOa8GjelmX6f0Q PBxim4CEVdQO+xjiWsFkppC7Y8KapevowtGrU9aRy1Kcbion9MmM6daVVJFJLdf+LfVu 5mVw== X-Gm-Message-State: AJIora9IZv1hXRZePt7ZP+oXvutc0Ir2NHJNiJ0+4UJoit897DWy+nJW 37KlDNNZMDhpiKOJvtieK2XDwBzXor0LlGvSlShPrsW3CwqLSFop X-Google-Smtp-Source: AGRyM1vfdZJyxuUQ9e74dhKPifGpRyMklM3wTuIN6r1OSpkDoyZOJ5rVnIFTvbJDSkkIzbQsQrYq2KuRAgGfev+AP/o= X-Received: by 2002:a05:6512:3f12:b0:47f:51de:d067 with SMTP id y18-20020a0565123f1200b0047f51ded067mr24773074lfa.146.1657096998959; Wed, 06 Jul 2022 01:43:18 -0700 (PDT) List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 From: Turritopsis Dohrnii Teo En Ming Date: Wed, 6 Jul 2022 16:43:07 +0800 Message-ID: Subject: FreeBSD is a great operating system! To: freebsd-transport@freebsd.org Cc: ceo@teo-en-ming-corp.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4LdCk03vFSz4sM2 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=kvJeOMK2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of tdtemccna@gmail.com designates 2a00:1450:4864:20::132 as permitted sender) smtp.mailfrom=tdtemccna@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SUBJECT_ENDS_EXCLAIM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::132:from]; MLMMJ_DEST(0.00)[freebsd-transport]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Subject: FreeBSD is a great operating system! Good day from Singapore, I think FreeBSD is a great operating system! I support FreeBSD because the most popular pfSense firewall, the extremely popular OPNsense firewall and the BSD Router Project are all powered by FreeBSD! macOS is also based on FreeBSD! I use pfSense community edition firewall in my home. I am planning to try out OPNsense firewall next. I will continue to support FreeBSD! It is a great operating system! FreeBSD is a very good network operating system. Regards, Mr. Turritopsis Dohrnii Teo En Ming Targeted Individual in Singapore 6 July 2022 Wed Blogs: https://tdtemcerts.blogspot.com https://tdtemcerts.wordpress.com From nobody Sun Jul 31 20:16:10 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Lwsvx2yn0z4X24r for ; Sun, 31 Jul 2022 20:16:13 +0000 (UTC) (envelope-from mike@mail.karels.net) Received: from mail.karels.net (mail.karels.net [216.160.39.52]) by mx1.freebsd.org (Postfix) with ESMTP id 4Lwsvw0fQ6z3SHC for ; Sun, 31 Jul 2022 20:16:11 +0000 (UTC) (envelope-from mike@mail.karels.net) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.16.1/8.16.1) with ESMTPS id 26VKGA6E043002 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sun, 31 Jul 2022 15:16:11 -0500 (CDT) (envelope-from mike@mail.karels.net) Received: (from mike@localhost) by mail.karels.net (8.16.1/8.16.1/Submit) id 26VKGAL7043001; Sun, 31 Jul 2022 15:16:10 -0500 (CDT) (envelope-from mike) Message-Id: <202207312016.26VKGAL7043001@mail.karels.net> To: freebsd-transport@freebsd.org From: Mike Karels Reply-to: mike@karels.net Subject: adding missing sysctls to tcp.4 manpage List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <42999.1659298570.1@mail.karels.net> Content-Transfer-Encoding: quoted-printable Date: Sun, 31 Jul 2022 15:16:10 -0500 X-Rspamd-Queue-Id: 4Lwsvw0fQ6z3SHC X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of mike@mail.karels.net has no SPF policy when checking 216.160.39.52) smtp.mailfrom=mike@mail.karels.net X-Spamd-Result: default: False [-1.70 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[mike@karels.net,mike@mail.karels.net]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; DMARC_NA(0.00)[karels.net]; MLMMJ_DEST(0.00)[freebsd-transport@freebsd.org]; HAS_REPLYTO(0.00)[mike@karels.net]; R_DKIM_NA(0.00)[]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[mike]; ARC_NA(0.00)[]; ASN(0.00)[asn:209, ipnet:216.160.36.0/22, country:US]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-transport@freebsd.org]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_NEQ_ENVFROM(0.00)[mike@karels.net,mike@mail.karels.net] X-ThisMailContainsUnwantedMimeParts: N I have a review pending to add missing sysctls to inet.4 and icmp.4 (https://reviews.freebsd.org/D36003). I'd like to do something similar for tcp.4, but it is a bigger project. I have a start, which I put in review https://reviews.freebsd.org/D36004. I started with sysctls related to recvbuf and sendbuf scaling, updating recvspace and sendspace as well. I also added references to sysctls documented in separate man pages (not sure I got all of those). There are a number of other sysctls that are not yet included. Not all of them should be included, though, probably including those used by tools like tcpdrop. I'll append a list for anyone interested to peruse. I listed sysctls I added so far, along with a longer list of things I wasn't sure about. They are listed as sysctl -d output to provide a little info. The size of the sysctl list in tcp.4 may deserve a little thought. There were 85 entries when I started; I'm now at 105. There are 50 on the list unresolved entries. Some of them may deserve separate man pages, and many may not be of sufficient interest. Comments? Feel free to comment on the review, or on items in this email. Mike Added sysctls: net.inet.tcp.drop_synfin: Drop TCP packets with SYN+FIN set net.inet.tcp.minmss: Minimum TCP Maximum Segment Size net.inet.tcp.persmax: maximum persistence interval net.inet.tcp.persmin: minimum persistence interval net.inet.tcp.recvbuf_auto: Enable automatic receive buffer sizing net.inet.tcp.recvbuf_max: Max size of automatic receive buffer net.inet.tcp.require_unique_port: Require globally-unique ephemeral port f= or outgoing connections net.inet.tcp.rexmit_drop_options: Drop TCP options from 3rd and later retr= ansmitted SYN net.inet.tcp.sack.globalholes: Global number of TCP SACK holes currently a= llocated net.inet.tcp.sendbuf_auto: Enable automatic send buffer sizing net.inet.tcp.sendbuf_auto_lowat: Modify threshold for auto send buffer gro= wth to account for SO_SNDLOWAT net.inet.tcp.sendbuf_inc: Incrementor step size of automatic send buffer net.inet.tcp.sendbuf_max: Max size of automatic send buffer net.inet.tcp.tso: Enable TCP Segmentation Offload net.inet.tcp.v6mssdflt: Default TCP Maximum Segment Size for IPv6 Added as references: net.inet.tcp.blackhole_local: Enforce net.inet.tcp.blackhole for locally o= riginated packets net.inet.tcp.cc.newreno: New Reno related settings net.inet.tcp.cc: Congestion control related settings (see mod_cc(4)) net.inet.tcp.syncache: TCP SYN cache net.inet.tcp.syncookies_only: Use only TCP SYN cookies Modified: net.inet.tcp.delayed_ack: Delay ACK to try and piggyback it onto a data pa= cket net.inet.tcp.recvspace: Initial receive socket buffer size net.inet.tcp.sendspace: Initial send socket buffer size Don't have, unclear whether we should (or where): net.inet.tcp.abc_l_var: Cap the max cwnd increment during slow-start to th= is number of segments net.inet.tcp.ack_war_cnt: If the tcp_stack does ack-war prevention how man= y acks can be sent in its time window? net.inet.tcp.ack_war_timewindow: If the tcp_stack does ack-war prevention = how many milliseconds are in its time window? net.inet.tcp.bb: TCP Black Box controls net.inet.tcp.bb.disable_all: Disable all BB logging for all connections net.inet.tcp.bb.log_auto_all: Auto-select from all sessions (rather than j= ust those with IDs) net.inet.tcp.bb.log_auto_mode: Logging mode for auto-selected sessions (de= fault is TCP_LOG_STATE_HEAD_AUTO) net.inet.tcp.bb.log_auto_ratio: Do auto capturing for 1 out of N sessions net.inet.tcp.bb.log_global_entries: Current number of events maintained fo= r all TCP sessions net.inet.tcp.bb.log_global_limit: Maximum number of events maintained for = all TCP sessions net.inet.tcp.bb.log_id_entries: Current number of log IDs net.inet.tcp.bb.log_id_limit: Maximum number of log IDs net.inet.tcp.bb.log_id_tcpcb_entries: Current number of tcpcbs with log ID= s net.inet.tcp.bb.log_id_tcpcb_limit: Maximum number of tcpcbs with log IDs net.inet.tcp.bb.log_session_limit: Maximum number of events maintained for= each TCP session net.inet.tcp.bb.log_verbose: Force verbose logging for TCP traces net.inet.tcp.bb.log_version: Version of log formats exported net.inet.tcp.bb.pcb_ids_cur: Number of pcb IDs allocated in the system net.inet.tcp.bb.pcb_ids_tot: Total number of pcb IDs that have been alloca= ted net.inet.tcp.cc.hystartplusplus.bblogs: Do we enable HyStart++ Black Box l= ogs to be generated if BB logging is on net.inet.tcp.cc.hystartplusplus.css_growth_div: The divisor to the growth = when in Hystart++ CSS net.inet.tcp.cc.hystartplusplus.css_rounds: The number of rounds HyStart++= lasts in CSS before falling to CA net.inet.tcp.cc.hystartplusplus.maxrtt_thresh: HyStarts++ maximum RTT thre= sh used in clamp (in microseconds) net.inet.tcp.cc.hystartplusplus.minrtt_thresh: HyStarts++ minimum RTT thre= sh used in clamp (in microseconds) net.inet.tcp.cc.hystartplusplus.n_rttsamples: The number of RTT samples th= at must be seen to consider HyStart++ net.inet.tcp.cc.hystartplusplus: New Reno related HyStart++ settings net.inet.tcp.function_info: List TCP function block name-to-ID mappings net.inet.tcp.initcwnd_segments: Slow-start flight size (initial congestion= window) in number of segments net.inet.tcp.log_debug: Log errors caused by incoming TCP segments net.inet.tcp.lro.compressed: Number of lro's compressed and sent to transp= ort net.inet.tcp.lro.entries: default number of LRO entries net.inet.tcp.lro.extra_mbuf: Number of times we had an extra compressed ac= k dropped into the tp net.inet.tcp.lro.fullqueue: Number of lro's fully queued to transport net.inet.tcp.lro.lockcnt: Number of lro's inp_wlocks taken net.inet.tcp.lro.lro_badcsum: Number of packets that the common code saw w= ith bad csums net.inet.tcp.lro.lro_cpu_threshold: Number of interrupts in a row on the s= ame CPU that will make us declare an 'affinity' cpu? net.inet.tcp.lro.with_m_ackcmp: Number of mbufs queued with M_ACKCMP flags= set net.inet.tcp.lro.without_m_ackcmp: Number of mbufs queued without M_ACKCMP net.inet.tcp.lro.wokeup: Number of lro's where we woke up transport via hp= ts net.inet.tcp.lro.would_have_but: Number of times we would have had an extr= a compressed, but mget failed net.inet.tcp.map_limit: Total sendmap entries limit net.inet.tcp.newcwv: Enable New Congestion Window Validation per RFC7661 net.inet.tcp.pacing_count: Number of TCP connections being paced net.inet.tcp.pacing_limit: If the TCP stack does pacing, is there a limit = (-1 =3D no, 0 =3D no pacing N =3D number of connections) net.inet.tcp.per_cpu_timers: run tcp timers on all cpus net.inet.tcp.reass.new_limit: Do we use the new limit method we are discus= sing? net.inet.tcp.reass.queueguard: Number of TCP Segments in Reassembly Queue = where we flip over to guard mode net.inet.tcp.rfc3465: Enable RFC 3465 (Appropriate Byte Counting) net.inet.tcp.soreceive_stream: Using soreceive_stream for TCP sockets net.inet.tcp.split_limit: Total sendmap split entries limit From nobody Sat Aug 6 15:21:03 2022 X-Original-To: transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M0R4c1fxyz4Y3SN for ; Sat, 6 Aug 2022 15:21:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M0R4c0f0cz3MKx for ; Sat, 6 Aug 2022 15:21:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4M0R4b6rb7zdMC for ; Sat, 6 Aug 2022 15:21:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 276FL3Hq012906 for ; Sat, 6 Aug 2022 15:21:03 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 276FL3vd012905 for transport@FreeBSD.org; Sat, 6 Aug 2022 15:21:03 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: transport@FreeBSD.org Subject: [Bug 265664] Undefined behaviour in sys/netinet/tcp_lro.h Date: Sat, 06 Aug 2022 15:21:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: lwhsu@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: transport@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1659799264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3dA6eEjQTvj3nzZUvayPdF2gmPM3zISCZNkWJ/3Y5mg=; b=KmcAFTDATAy9eB69ayg5vjBe+F2x/ZOpmRu2GSIMEJacJPwbL40jMKxn4QSBLnYIP/xrTl WojWbQCjXbKx4w1Bil1rmt8GXo7NeTdRrop+XXLL86QqnTEUpKpLC+VvzkzRQ2GHTYp+J3 KHiZhAFQQNcyd7WWp0BtN4HAofk5P0ko7dTx0RTOxF9ZfqcLeAGCJdsvmg9njS0phxYljG vPvNZmDLVKdZ7qmm3+AMlw3TJJGJ+7I6RQv2tqdg79SlxL+COU1TJpLutBOUUfmlJvFtXI flO2WeOz5cML51l8yBkiaCmBnSB5OEC7Wr1iHrS9fmS0rhChQf0H5AKateRmWg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1659799264; a=rsa-sha256; cv=none; b=T81nEdBLHGBhy0BU9STGSn9DdYoQTNEFGxkzxg1GJQSPvjOVwIQUvm8uCKjw4py4lfdg/x MkTDdno8gK7/yGu7uT4wfYvcfaBMvFs2wV+/KTu1fNiCRazI8Ywkjv8n+txcB/ukDBe+lO e2v539EUQkQOFI2BHBekTIR6+ceqAEcrAB4C7CddsFeG/y1+SHyMOudCXHW5V5KLkos15m lazyaEGbhc16oYMtQrkUR/hSvmPyCtcHZ1O1Ht/qHqAOqL3TuIAiu1sejqLfd9K+v6QeNt 9024tfsbBVjJnolxacDp3P766DbbChPm54xAgTgxJwD8OzokC1wjWiROg41j3w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D265664 Li-Wen Hsu changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lwhsu@FreeBSD.org Assignee|bugs@FreeBSD.org |transport@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Sep 6 15:42:48 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MMV5R4L7pz4c4BL for ; Tue, 6 Sep 2022 15:42:51 +0000 (UTC) (envelope-from Richard.Scheffenegger@netapp.com) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2069.outbound.protection.outlook.com [40.107.92.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MMV5Q4FBjz3CMh; Tue, 6 Sep 2022 15:42:50 +0000 (UTC) (envelope-from Richard.Scheffenegger@netapp.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j9m+0a63leVylOqNvgb65D9NGzdlKZY83mBmKhsSzo9viIYIhkO0DqvR/ZUrlibRqWJUQJnm7tkim3B+V+cSJZ5/Xh7A1fB1qmlQQukupWaWTMDkKlUrVLgZYzGfNfXVQggPTZPw/G7W3oBk3nfmjxt2Yt4KpqC/cSRdVMhkJf0hIfj1+t7qQUzhB9NfDwVzcrlEbua5dgtX4+52660ju68o55gesIpLZh38M4THFt8WsOXhjsfSyzRLOyBtIWQcedM4Ycw6fNuwL/Q/suiCDGSE4sqLggc0y++T8QFnQGd+XAppfv1yc/fFGs+rO15UqOdYIHG7AIV9rx4LerJoSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EdIGBvFLTmYRaHNGezq7u7PigS8yJBO57wpDvz7ll88=; b=ROH0i07eyx8BKCpN9w2LhYbBap76Idcvthog2ytGZaUB9DXsDx2YSacGG6pFuVO9VS/2fs6mOAfGFSaa9T4ho849kDtArD3rNNDd4w3DgwLSqMiXxsH3588rRq5elHWpsFb5uFE9XMh69LDiAGyD7V9CAF2gJi4LEFVyER3d3P+PRZK0rPKpaG5XiytFv8pzotQCQbwfqGos0ttQxU9MoV+qvd64YrMjsKCZa+axTUkOhvdl4jMqj7V4jUncOE+vJ3bmH7pfcLXsScSh0/516dDLvb6QHt3l6OoeyzhG4vThAAGUi7g1YPT9SLbekgiSMJ0VK0M33CBvPT0bTy/qRQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netapp.com; dmarc=pass action=none header.from=netapp.com; dkim=pass header.d=netapp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netapp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EdIGBvFLTmYRaHNGezq7u7PigS8yJBO57wpDvz7ll88=; b=NJRZNuGXyA4Alc6E2+vugc51gzRGnd10eMjKKBwtYY43F0DzIHlGCSeOvLnz6EA7xuGpDvyjiPYuuTtTY7W7rX8iUBEj8bg4sXsE4TerwztTqQOscjepvwJbpmmRD0lm3hwrx4xQlOAwKn+u6DNyRSCb6ENPA/jKJRnGjr9+Lc63tpOghskVxBlTTX+K2YTCVrYGRq34GwCwKhQYE9nvswH6DDeyuPD7wr/gxM9QEeQJsj9PEdhW4hePWghKce2X/1CMSTNmwy+N0bZFtD8NpWN9bIFtkhERoO+J8f24PbaIjUlLcJrJHT6+zXevR14sY1ZrP4NX0Bz8Ga9vPTl4FA== Received: from PH0PR06MB7639.namprd06.prod.outlook.com (2603:10b6:510:4e::18) by CO1PR06MB8091.namprd06.prod.outlook.com (2603:10b6:303:e6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.12; Tue, 6 Sep 2022 15:42:48 +0000 Received: from PH0PR06MB7639.namprd06.prod.outlook.com ([fe80::ad0a:d183:6046:8965]) by PH0PR06MB7639.namprd06.prod.outlook.com ([fe80::ad0a:d183:6046:8965%6]) with mapi id 15.20.5588.017; Tue, 6 Sep 2022 15:42:48 +0000 From: "Scheffenegger, Richard" To: Alexander Motin , Gleb Smirnoff , Richard Scheffenegger , Randall Stewart CC: "" Subject: RE: Why still not Cubic? Thread-Topic: Why still not Cubic? Thread-Index: AQHYwgMXy2j074q/d0KP0wEXr4Sbyq3SiThQ Date: Tue, 6 Sep 2022 15:42:48 +0000 Message-ID: References: In-Reply-To: Accept-Language: de-AT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.600.2 dlp-reaction: no-action x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fe420633-0925-4282-6b3e-08da901e736c x-ms-traffictypediagnostic: CO1PR06MB8091:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: pWq0pS5IOrmiWJyQdp0Mo8l3s7dfjEP1CUweMSaiwryAfa8DdS3+2gR5StDhuzUP6xSyYr/muMS4DuifU6BbB4g8EQn0ZXCskXr8pXVMGUKU+SN9/LGl101ihlrEmeaGOM0EnjCmI/IfthWicjnPziRMS4Lbr9hLmFbIIB8iog5Gm+dUzvosHDeHtCz+YrTBBqQRGZbLyrR6ALOUWV8ueaFViRLOp5/FJqXewfg2q3FMK/a3iwgamwafZViabc/dG++VbVi2O5OiW3+qFKTSEX6YvcMwJQwG75ys0B6i++7Z+ECGI1tfVtDYeDXW1r8of5hUYJ4JBylePrf01kAkjHNopvhAMqDn0qHvz4msnhiMRSdKINqoAjLuAYUktmgbrXydismjGeKACgHvYn+tinz1wmrgz6Rmsaei/F8hLbwA77oAIBjmVLTh3TIxC7WIMhRVGI9Aq8NVyI5ne/1SfM46prfGmkX204ofmx1f1aidcEwNo0CgT1TxeITR1Tq+vG3VKD/aosxjbXhcI5DGRvI+J1vCbZDyIKpBU2RVqO9A30CuB/t/ClHCU4suAwiujMP+Sxv2Uwh1Wtb32Hh4gkrf3/E0bNN7W3nV1Iaxi60uyClLlql0QGJC3zfqwxaAwgjGnrUjTLoZ7OZF7vD8p9FlJfL1fo6j60WbRZIguKNz452ugrhvvstsvThnUocDqwWj0jfsg4lUZMeOMxp4GeyhZBLV/RRnoqvSsWs746zqUGwrKl2TtJS5U6e36fKW2pDaNraH0gFQpB9K2bqvrg== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR06MB7639.namprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(136003)(346002)(39860400002)(376002)(366004)(396003)(122000001)(3480700007)(86362001)(38070700005)(316002)(8936002)(110136005)(5660300002)(52536014)(2906002)(450100002)(478600001)(76116006)(4326008)(66446008)(66556008)(66476007)(64756008)(66946007)(38100700002)(186003)(8676002)(55016003)(9686003)(53546011)(26005)(71200400001)(6506007)(83380400001)(7696005)(41300700001)(33656002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?utf-8?B?WjFhM2MxeFRmbG13TGU0a2dnYllBN1c0bFYzZHhabUpkeXFKTlEwRGl2ZlYz?= =?utf-8?B?VUV5WFVYclB4NmQxOHUxQncvd1JpQmJzcWI5b2RKMUxwVmYyZHNtc1dMdmp4?= =?utf-8?B?UkFaeWFwdFVpTzYyYzcwbERyUENDbnRZTmVCelRRdmIzNzZvekRsS1VDeEc0?= =?utf-8?B?NThvbUJoRmU4enJhVW9OcmpTT3JUNEtmY3hKbjhJamRiczJhZ0xNcUNoaGFW?= =?utf-8?B?ZmY2VXFDVXd6VGEweUxybmRCWktURDlFQ2hhNVVQOFF5c1BHblhwSU55UFNX?= =?utf-8?B?NUFISHM5aURVdWN6UGViTVVGM2VsL2pVNllidDNLdTdtMVUzSXZaTC90NmRr?= =?utf-8?B?eDVNanFtb0xMV0JCc0cxTDB5R0xnV3hpSWtNRG13OUIxVzBad1MxVzZxS29D?= =?utf-8?B?amxYWStrT2cvM0tsenpRRFg2dmJYM1BTUG9MYTdSeW1JOWNXTXgrVE4yQnFr?= =?utf-8?B?NjJCdm12N1RvY1VBZUJCa1RxbGJXUlBmR01sQlhrSmJFTWRYUk1MM0tkdDJh?= =?utf-8?B?YnZVSEZ3T3p5d3hpRVZxRWttQWlzQUdVUTFSYWFzUEoyUzVLTVcxUklPNk1D?= =?utf-8?B?cGhxcnVFbStyNnh3Q1h2Qno2VC9BeTYyUXIrMFdjZXR1TjlOVjNZZDAvV2dL?= =?utf-8?B?YThscmRDd2wyTzNvRGJ3elpHS0ZDQ1h6TlJnODJzYmlkWVpYTmNxUmd5V0VV?= =?utf-8?B?RlppRmdPNHpRclE1bTV4VkJhNDJvSjFjNEs5MDQ0SWtIUWM5NlZMYkFCYnZ3?= =?utf-8?B?MzVwdlpiYVFPMWhlek4zSzlZL0VYTTBjRWlMZXM4VldRckhabm1DVXp0SERw?= =?utf-8?B?Z3p1dVhsT2RKVGphTkswbUJyclNCUS9IN1VFVDhXWlpVa2tkbnRlaFZTYnpz?= =?utf-8?B?eVBlZjNyUDVHRXVFVFdjWUhpa0NaaHNlTUdRenZycGlGaHpmTHZrbkRPU1JL?= =?utf-8?B?b1l6dThEV0ZTYjR3VFc0KzIwdzNiV0JkeUc2WDVtZ0JOa0hRK2FJbVQvQnkr?= =?utf-8?B?VGtleC9CdmJNUGY5SjByT0h1KzgwQUZ4b0I2b2JGR3I3YWNaT3kxQ3p3bWtm?= =?utf-8?B?QnhRWnRsc3h6QTl2ZHRKWmg5M3hZM21xM3pPMytFbzZldjNXc3doMXZnSWpH?= =?utf-8?B?b3pOa2k0VXJhQm9TRTk3dHJnWWhWTGhabTFMdFk3Y2dLTmlyNFplOUwxKzJz?= =?utf-8?B?ODROaGF5di9wU3dHdkRjWVEyNTlxZENnUHBmTkh1ODIxTkFOdEdvSTBZcnc0?= =?utf-8?B?SGdoYzVRUVRJdWZQcE1Gbzl0TkR4djgxQVFPT0FVSm80SkJmZ2dZUTYzVUFJ?= =?utf-8?B?S1FwYkJKZ0JxcTdMVlFEd3dhS0FlclkvNk14SmNQbVAxQUlVWnd6eGg2QjEz?= =?utf-8?B?SUlTU0huVm5ibUZneStsZXM5dmhQRllhTTNoTzJWSlp4UjBaMXZlNnQwVW0r?= =?utf-8?B?aS9FYmZnYlBmVnpQQUlZcVRzR0VvY2h4WWxNWE1hbjZ6cGwyQXJoTWdxaWRT?= =?utf-8?B?cWl6NWVWamhoTE1YUVhyd08yVitTODZxVTNvTzRlOVhQVXdOb0VXZmd5WWJB?= =?utf-8?B?QmJnREZhcDZyR0I4Vk5TOHhlU3hvTmJHWkVLOHloa2ZGaG5EYXNva1VVQVd6?= =?utf-8?B?M20wV0lYdkhySllSTnRHSUpEOWVXVUVQNTh4Nlp0UWkrMS9PZkZKaXQyeFdX?= =?utf-8?B?K1dGcFA4UlZkTmFyNXZNMnBtNExBUFVOK05pNklnQVoreXM3bUdjbjdGZ0JN?= =?utf-8?B?NGxWMWxQRkNWVTN4Njdqd3BoR3QxSlNzOWEwQzlwYzcwQnh0L0c0OURoQkZl?= =?utf-8?B?NGpPUlBMZlNOd1lUbVpPb0loNWxzU1dxUU9nNUdqQ3E1VjFZT1lhRmNaV24y?= =?utf-8?B?ZzB4WkU4c2c1aytkMDRXSjBHb0NFL2tBbXhoTWpibDg1NzdoaGVLNnVUNGNx?= =?utf-8?B?UktyQkp6L2NlWXptZEJ3MGJNVG9reG1PYjZpdEJJSTFXZU1rb2ZwZXRHcCt5?= =?utf-8?B?TzZYUkpNcWZTUG9vT2NCOVI3NWR6b0lnTkdxUlJzWkQrUWRhUllvaHkycnc0?= =?utf-8?B?QTZXNXlYOHA5TVhVWXRleTZVN1EyalNCVVJYRTV1R0tYcExpUTRKd2hxRFA2?= =?utf-8?Q?kqah1hMAh6jAV5zVklGJ2t3HA?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: netapp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR06MB7639.namprd06.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: fe420633-0925-4282-6b3e-08da901e736c X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Sep 2022 15:42:48.5625 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4b0911a0-929b-4715-944b-c03745165b3a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 7jMIv5w5owglc51s0EK+7Z9KxwF7B0l1ayYa6lyArkDd+6Om5q0OemSZyhoQLjONPdQIjkTd+T5m/n4GcB8dPg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR06MB8091 X-Rspamd-Queue-Id: 4MMV5Q4FBjz3CMh X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netapp.com header.s=selector1 header.b=NJRZNuGX; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=netapp.com; spf=pass (mx1.freebsd.org: domain of Richard.Scheffenegger@netapp.com designates 40.107.92.69 as permitted sender) smtp.mailfrom=Richard.Scheffenegger@netapp.com X-Spamd-Result: default: False [-1.79 / 15.00]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; DWL_DNSWL_LOW(-1.00)[netapp.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_BASE64_TEXT_BOGUS(1.00)[]; NEURAL_HAM_MEDIUM(-0.89)[-0.895]; NEURAL_SPAM_LONG(0.77)[0.770]; NEURAL_HAM_SHORT(-0.77)[-0.769]; DMARC_POLICY_ALLOW(-0.50)[netapp.com,quarantine]; R_DKIM_ALLOW(-0.20)[netapp.com:s=selector1]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; MIME_BASE64_TEXT(0.10)[]; MIME_GOOD(-0.10)[text/plain]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; MLMMJ_DEST(0.00)[freebsd-transport@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[40.107.92.69:from]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.92.69:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[netapp.com:+] X-ThisMailContainsUnwantedMimeParts: N SGkgQWxleGFuZGVyLA0KDQpVbnRpbCByZWNlbnRseSwgdGhlIGN1YmljIGltcGxlbWVudGF0aW9u IHdhcyBtb3JlIG9mIGEgcHJvb2Ytb2YtY29uY2VwdC4gSSBiZWxpZXZlIG15IGNvbGxlYWdlcyBo ZXJlIGhhdmUgbWFkZSBhIHJlYXNvbmFibGUgam9iIG9mIGFkZHJlc3NpbmcgYWxsIHRoZSBlZGdl IGNhc2VzLCB3aGVyZSBjdWJpY19jYyB3b3VsZCBub3Qgd29yayBwcm9wZXJseSAoaW50IG92ZXJm bG93cywgZXRjKS4NCg0KSG93ZXZlciwgdGhlIHByaW5jaXBsZSBhbGdvIG9mIHRoZSBjdXJyZW50 IGN1YmljX2NjIHdhcyBiYXNlZCBvZmYgb2YgYW4gYWJhbmRvbmVkIFJGQyBkcmFmdCwgd2hpY2gg b25seSBsYXRlciBnb3QgcmV2aXZlZCwgYW5kIGN1cnJlbnRseSB0aGVyZSBpcyBhY3RpdmUgSUVU RiB3b3JrIHRvIGRvY3VtZW50IGFsbCB0aGUgYWxnb3JpdGhtaWMgaXNzdWVzIGFuZCBpbXByb3Zl bWVudHMsIHdoaWNoIGhhZCBiZWVuIGFjY3VtdWxhdGVkIGluIExpbnV4IG92ZXIgdGhlIHllYXJz Lg0KDQpJbiBzaG9ydCwgSSB3b3VsZCBleHBlY3QgdGhlIGN1cnJlbnQgY3ViaWNfY2MgdG8gbWFr ZSBhIGZhaXJseSByZWFzb25hYmxlIGpvYiwgYnV0IGl0IG1heSBub3QgY29uZm9ybSAxMDAlIHRv IHRoZSB1cGNvbWluZyByZXZpc2lvbiBvZiB0aGUgQ3ViaWMgSUVURiBSRkMuIEJ1dCBoYXZpbmcg bW9yZSBleHBvc3VyZSB3aXRoIGN1YmljIGluIG1haW4gbWF5IGJlIGEgcmVhc29uYWJsZSBpZGVh LiANCg0KUGVyaGFwcyB3ZSAoI3RyYW5zcG9ydCkgc2hvdWxkIGRpc2N1c3MgY2hhbmdpbmcgdGhl IGRlZmF1bHQgY2MgdG8gY3ViaWMsIGFuZCBtYXliZSByZXZlcnQgdG8gbmV3X3Jlbm8gZm9yIHN0 YWJsZS8xNCB3aGVuIGl0IGJyYW5jaGVzPw0KDQpCZXN0IHJlZ2FyZHMsDQogIFJpY2hhcmQNCg0K DQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogQWxleGFuZGVyIE1vdGluIDxtYXZi c2RAZ21haWwuY29tPiBPbiBCZWhhbGYgT2YgQWxleGFuZGVyIE1vdGluDQpTZW50OiBEaWVuc3Rh ZywgNi4gU2VwdGVtYmVyIDIwMjIgMTc6MTINClRvOiBHbGViIFNtaXJub2ZmIDxnbGViaXVzQEZy ZWVCU0Qub3JnPjsgUmljaGFyZCBTY2hlZmZlbmVnZ2VyIDxyc2NoZWZmQEZyZWVCU0Qub3JnPjsg UmFuZGFsbCBTdGV3YXJ0IDxycnNARnJlZUJTRC5vcmc+DQpTdWJqZWN0OiBXaHkgc3RpbGwgbm90 IEN1YmljPw0KDQpOZXRBcHAgU2VjdXJpdHkgV0FSTklORzogVGhpcyBpcyBhbiBleHRlcm5hbCBl bWFpbC4gRG8gbm90IGNsaWNrIGxpbmtzIG9yIG9wZW4gYXR0YWNobWVudHMgdW5sZXNzIHlvdSBy ZWNvZ25pemUgdGhlIHNlbmRlciBhbmQga25vdyB0aGUgY29udGVudCBpcyBzYWZlLg0KDQoNCg0K DQpIaSBndXlzLA0KDQpEb2VzIGFueWJvZHkga25vdyB3aHkgYWZ0ZXIgc28gbWFueSB5ZWFycyBG cmVlQlNEIHN0aWxsIGRvZXNuJ3QgdXNlIEN1YmljIENDIGJ5IGRlZmF1bHQsIHJlbHlpbmcgb24g TmV3UmVubz8gIERvZXMgTmV3UmVubyBoYXMgc29tZSBiZW5lZml0cyBvciBDdWJpYyBzdGlsbCBj b25zaWRlcmVkIGV4cGVyaW1lbnRhbCBhZnRlciBzbyBtYW55IHllYXJzPyAgSW4gb3VyIE5BUyB1 c2UgY2FzZXMgd2Ugc2VlIHRoYXQgd2hpbGUgTmV3UmVubyB3b3JrcyB3ZWxsIGluIGRhdGEgY2Vu dGVyIGVudmlyb25tZW50cyB3aXRoIGZsb3cgY29udHJvbCBhbmQgd2l0aG91dCBwYWNrZXQgbG9z c2VzLCBDdWJpYyBtYXkgd29yayBtdWNoIGJldHRlciBvdmVyIFdBTiwgV2lGaSBhbmQgc28gb24u ICBBcmUgdGhlcmUga25vd24gc2NlbmFyaW9zIHdoZXJlIGl0IGlzIHdvcnNlPyAgT3IgZXZlcnli b2R5IGp1c3Qgc3dpdGNoIHRvIGl0IGxvY2FsbHk/DQoNClRoYW5rcy4NCg0KLS0NCkFsZXhhbmRl ciBNb3Rpbg0K From nobody Tue Sep 6 18:02:33 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MMYBk0zJdz4cKhD for ; Tue, 6 Sep 2022 18:02:38 +0000 (UTC) (envelope-from Cheng.Cui@netapp.com) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2040.outbound.protection.outlook.com [40.107.93.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MMYBh2tclz3XNZ; Tue, 6 Sep 2022 18:02:36 +0000 (UTC) (envelope-from Cheng.Cui@netapp.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nu3Lk0WVe3sf0Upy8cYeLcCc/m884es2nTM4rwa4sCaXKtuqp0eKxfLrUJzZChUzkIUiEi0y/Km6XetEUmUZ1bc7nzelrla/Wl/dgY1k4mzxN5UB07oTGgV5SxxM3MlveCO2y9kDH3Q5cRSGBvcWRrKTrXQp52CFSlVw0WLdJASpvishjmaEIc27VtJnWx1vnqyKGRGurFT1/vuCJAyjvXhTQR8CgMR72S+eCAv/oERejpborYeA8FRdgLPg63Wbvyqu5GJUX8pk3xBKAu+VxHtq1iZMv2wIkSPPuYqf2lfe+r+TcLxgEihShtDv9flSG+XBSM5pPws9TWccpXqklQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PoqtfJLRfBRTFLW7t5vSsRg39sw4Ip9hY2lRbrZUlxs=; b=HL7hC/f7ZYVP5zYVeoIDpjcwwfvdbsINzFPrBPa4PjkmELql4gjKQ+x/znPAqem4/GeWq8yyk5X/HIIbk+NoEPFxjtdw7ZbkpxgXzzgI9egoV+UZ4gNA7DGi9L+6H/oGI+D4kaBAxH9tVOagdTD/TfRZo/OcSRIFuv1VY29ctfdl8NsjVp7gC83u5iRZm1mI2jSfNy4dyU+hgIe7IfywV4bh5WNZELYH5D5DHzPK/e2bugpK2DOHd94gIYAogYo7K2kL6JB6VJ2T4tZvYav1rvC0qJHLfkKOcGy1MaAvsmJ0F+sdDD2qLehmbCSTV7r271gqFjldHmvP3vW69gapYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netapp.com; dmarc=pass action=none header.from=netapp.com; dkim=pass header.d=netapp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netapp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PoqtfJLRfBRTFLW7t5vSsRg39sw4Ip9hY2lRbrZUlxs=; b=lJjqByt+aJHqw7aaTwjQpSzkD7bLKD/XP96TuzKOXy7rqKLuqGXDWVdwsda/CzaHQ9teFMCM5nQ+19mJoFr4iTLmQzlJcJBfgSao1skrqJCthYh6g66B8sBLkJ7VwSKnoAgE0l6+JAx3A+A5fY/H+X/QD2+JKof8QIPw+jq01zEroqecBlAOwwFb0fm2lBcwBGGBDuFLvwc2OxPGRV6oMamdpVquAKqACmqMoMoZHWdi6elpd4fff/dOO8N633d3AxRf3tVa5EqWhn87TyjaqwASJGoyJtH+8QeY/zps6lPO+js+5Na8uP9LOVHdNzWbD+VGp3rhQc00bo7ozJlqNg== Received: from MN2PR06MB6237.namprd06.prod.outlook.com (2603:10b6:208:de::11) by PH0PR06MB7096.namprd06.prod.outlook.com (2603:10b6:510:24::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Tue, 6 Sep 2022 18:02:34 +0000 Received: from MN2PR06MB6237.namprd06.prod.outlook.com ([fe80::313f:3621:2524:d94]) by MN2PR06MB6237.namprd06.prod.outlook.com ([fe80::313f:3621:2524:d94%5]) with mapi id 15.20.5588.018; Tue, 6 Sep 2022 18:02:34 +0000 From: "Cui, Cheng" To: "Scheffenegger, Richard" , Alexander Motin , Gleb Smirnoff , Richard Scheffenegger , Randall Stewart CC: "" Subject: Re: Why still not Cubic? Thread-Topic: Why still not Cubic? Thread-Index: AQHYwgMXy2j074q/d0KP0wEXr4Sbyq3SiThQgAAkfTk= Date: Tue, 6 Sep 2022 18:02:33 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 99efbaef-41d7-4f79-f1d7-08da9031f976 x-ms-traffictypediagnostic: PH0PR06MB7096:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: CscpEr28ymNc6SECYzGvuVJHVbGPjuzDy9McB9Z9OyIqDYn5EMncNTgseUuLs+hxt3VEAR5zhimnSutGO6qOA6JR7R7Z32IHwfiircy06lwG2Mx1Nx+Ara+m49ZKJqVNDZdFexXWnq0mOHsamNHjSlb3e3tTyUItE1mtp6rmrvgKkc/b8KJ+jpOty6rL/y9KuZtbDA5atbtdo5KSOhUOZEo1M/LvmV8CT3OeDfObSKtV5wFqVJ1abYychMMxlsttNokcQF8AfmgBCHDxrgOqMaYmkyr5hLB7+VHtHSBB2bdRkZ3SH2dfxa1HK0eAAMrCF9xtMftJkwgNLkVv61FxHnf1bz4CStu044CaEfHaXPfJow++4k/Hn0d9+HaiIX/BM1MG6FQJuNv85gg46CvwRC0T0D533g0UT7M3nCa3slXGO1J/6s0mxCPl0I0dIXUWTtedyrctbXpOe9qxOMUF93ZJG6c5l3vvEyuYuvp6OeIz5JlJKwRM7teYsieRmwqcIHTHcBn/pYD20vs7Wh7OMzm0xTgoHJHPKeVai1azzM5l7PGdX8ranqKW5tOTL96ANEvKfaaq/XeALEG2E69AP/F6DepSfDx6Kol7wtGT1sh3wspQb+pf+fyBMjEZdI/p5OmpJfJ6yt5w5PwsSWV4TLnZVbp6s6Q9NSAPFwrT9Ssr8e/4kLi4ckefxfCiu0uoO2nfrttPLEqhbbYtqNwZxc8hh0+nGkxs4uAB1PPPaemDoxVmzzIkzf+P3HqVTsZjfJ7SQ/6AneY6pU3Yy+42yKGYCx0Na044+4pAGegujm7fAXwK47YmUtCOek+jJUAgHpLMSEr/nh+E8aZBMH3Vlg== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR06MB6237.namprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(366004)(396003)(39860400002)(376002)(136003)(186003)(166002)(71200400001)(110136005)(316002)(3480700007)(38100700002)(33656002)(53546011)(7696005)(38070700005)(55236004)(8936002)(55016003)(6506007)(52536014)(86362001)(41300700001)(76116006)(83380400001)(478600001)(966005)(2906002)(4326008)(122000001)(450100002)(8676002)(66946007)(26005)(66556008)(66476007)(5660300002)(66446008)(9686003)(64756008)(15398625002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?U0W+/FpmjO3z7vXB6hJ5ymW/AeIaoHi0/7jaq00G/Fy4UjjJjA8T7CTK?= =?Windows-1252?Q?T9yuCnHVrXT7LFgOYJCC5BhcJLePurAaARLqAlHi4cBvBIFWlmbK2iJ9?= =?Windows-1252?Q?dWLIc2RU29ZZ8IBk7qu3K3pWQa4LDk/38SVxjcUMiC292HED211y89XZ?= =?Windows-1252?Q?PdMrEOR7uWMBAbgwSDaowa1KhUEK9G7gsCo/fgk5BnRU0GY1YWk4/nQh?= =?Windows-1252?Q?0FUAQYNh6gaPgnuO0UDOBkBx8NeYMo814QKZ0DEHSrpDOacllu7irzZC?= =?Windows-1252?Q?iXiC0zJrzdTBsNfCnEd3hyWvYN8tnQZ11nLrQRXaxje5HIvquDOiBdo0?= =?Windows-1252?Q?wUOV6YnPOPJvtTL3iwjuwm7n3SazcKUMYC+XJ7FmYFnX8MJrvQS574Gv?= =?Windows-1252?Q?QHzPTzdFVw9JcA+qILcxhQSWL4ztaG0fQp5NMLJZNBThs3gg4XPYQQeq?= =?Windows-1252?Q?4FnxJehIgkvSzbl3LQM4U50XxuGqUjIPjM16cP9UO0I6fmmBH0UC5f5K?= =?Windows-1252?Q?l3LzupPuS9h6DY/GGpu3gErDjo0mXOu+WJPWA1GkFct8ydjeqNz7wBOC?= =?Windows-1252?Q?Vdu8q7vG5VziVZ31k5kjmVXD9lcgPui8ikOTp5Z9k+Y/vZP3wP0knFHP?= =?Windows-1252?Q?L5Q7Gd/OmpAq7XBa+qmFC1DjkNo0R5AmlGdKS+Wo8mcXawP62lremmgc?= =?Windows-1252?Q?PUWUej4kvhuOW/sLHcomqBaEf85PUuIsCmORGaER+Vh6Nou3qoOHeDlp?= =?Windows-1252?Q?4EuXFqixBRufWh2+4Z3jwfQIkB29VDAPMH9RdryI5wrQcpfQHwBXVNE+?= =?Windows-1252?Q?w/Qb1wAzlJihFTcT25A9BHuIczkUDQRE1flDT3OHrGzutWox7gFuV7bW?= =?Windows-1252?Q?SrONMyXo6StYhHuA6S6V+7p02UiElXlho0krQX/uLsauqCP9ORIP9D9f?= =?Windows-1252?Q?RJ7aglDskYYcu+p+EYE9kmgVGuzvq32Pua/Ow3d9vl0xiE0sVKtl6VyM?= =?Windows-1252?Q?FYBq7ngtLEeQF+3nd0t7yHBRziRw3uYhnMV8BzJ1umSSRfZtwGd7zBoH?= =?Windows-1252?Q?ZwtJ2zqS2oKcu20rJ9I5RqL6/WvwYX4edWVBVJkSEvRXg/kZSa3CvmPp?= =?Windows-1252?Q?I/RZSxxxILs9jaigJXT7BwgWrVNcg/chIMXrzFsepvvOE14OmmxnXidk?= =?Windows-1252?Q?2jFZL/4i1NJyPEd5dDX3TBlkoEC8I6LjhmhZCCuBR9jQbI4l6+bXotwP?= =?Windows-1252?Q?1VhbZcIt1QDlsNJKV4yNRGUz0CIL31YLnb/GWWXZJY0EvtJ8/2rQwgtT?= =?Windows-1252?Q?mcFaOWXBDviGoLWvZ7/qSfgJ313RE6wZKDDHJeJluoYP3LP0O0JOG/IW?= =?Windows-1252?Q?A5rAUwcFfhXG+ZomegwZWxeVisB1pCQpPeDH3tcrNaDQg8v73DAPCdB2?= =?Windows-1252?Q?KCN2IgaAG4Jt53GkaN+EZ/5K12446fP2npVKK+mEksqCjfZ1M7Uq3SkY?= =?Windows-1252?Q?gds+zQl6O3SsIfi6/qVoHxnk0uYZjPj3cOC+AkkmsGlb00Jir3tupmfX?= =?Windows-1252?Q?YznS1OVFz2u1IfYEIjnd/9qOWYvjG/9z+FQafA5278mCcAyGwpnEY19E?= =?Windows-1252?Q?KYaiAhdGev1w06gZzKQOgpu8TWSRZnZxn+v3kI/CuBoOTKED3ao2BtTT?= =?Windows-1252?Q?YpS536SPafywkuegRzH03FTTVAhfANN6r6QFxVhyxyvnaeQc6mEX8w?= =?Windows-1252?Q?=3D=3D?= Content-Type: multipart/alternative; boundary="_000_MN2PR06MB6237B14D402EDA43329218719D7E9MN2PR06MB6237namp_" List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: netapp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR06MB6237.namprd06.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 99efbaef-41d7-4f79-f1d7-08da9031f976 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Sep 2022 18:02:33.8996 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4b0911a0-929b-4715-944b-c03745165b3a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: cw9ZTlPXQDadYRWJNQLrqLFxyMEViSdIBETc9THSMj3vvimke/sZs4O45HP2OcG1/dJ2Cty0Eb7f5Bcac3989g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR06MB7096 X-Rspamd-Queue-Id: 4MMYBh2tclz3XNZ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=netapp.com header.s=selector1 header.b=lJjqByt+; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=netapp.com; spf=pass (mx1.freebsd.org: domain of Cheng.Cui@netapp.com designates 40.107.93.40 as permitted sender) smtp.mailfrom=Cheng.Cui@netapp.com X-Spamd-Result: default: False [-4.79 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; DWL_DNSWL_LOW(-1.00)[netapp.com:dkim]; NEURAL_HAM_SHORT(-0.99)[-0.993]; NEURAL_HAM_MEDIUM(-0.95)[-0.948]; NEURAL_HAM_LONG(-0.85)[-0.852]; DMARC_POLICY_ALLOW(-0.50)[netapp.com,quarantine]; R_DKIM_ALLOW(-0.20)[netapp.com:s=selector1]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-transport@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; RCVD_IN_DNSWL_NONE(0.00)[40.107.93.40:from]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; HAS_WP_URI(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[netapp.com:+]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.93.40:from] X-ThisMailContainsUnwantedMimeParts: N --_000_MN2PR06MB6237B14D402EDA43329218719D7E9MN2PR06MB6237namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable > Perhaps we (#transport) should discuss changing the default cc to cubic, = and maybe revert to new_reno for stable/14 when it branches? Agree with Richard to make cubic as default. Since the recent improvements = in cubic (below), the use at my side didn=92t show much problem. Perhaps ma= king it default will draw further improvement/optimization. https://freebsdfoundation.org/wp-content/uploads/2021/05/TCP-Cubic-is-Ready= -to-Take-Flight.pdf -- Cheng Cui From: owner-freebsd-transport@freebsd.org on behalf of Scheffenegger, Richard Date: Tuesday, September 6, 2022 at 11:43 AM To: Alexander Motin , Gleb Smirnoff ,= Richard Scheffenegger , Randall Stewart Cc: Subject: RE: Why still not Cubic? NetApp Security WARNING: This is an external email. Do not click links or o= pen attachments unless you recognize the sender and know the content is saf= e. Hi Alexander, Until recently, the cubic implementation was more of a proof-of-concept. I = believe my colleages here have made a reasonable job of addressing all the = edge cases, where cubic_cc would not work properly (int overflows, etc). However, the principle algo of the current cubic_cc was based off of an aba= ndoned RFC draft, which only later got revived, and currently there is acti= ve IETF work to document all the algorithmic issues and improvements, which= had been accumulated in Linux over the years. In short, I would expect the current cubic_cc to make a fairly reasonable j= ob, but it may not conform 100% to the upcoming revision of the Cubic IETF = RFC. But having more exposure with cubic in main may be a reasonable idea. Perhaps we (#transport) should discuss changing the default cc to cubic, an= d maybe revert to new_reno for stable/14 when it branches? Best regards, Richard -----Original Message----- From: Alexander Motin On Behalf Of Alexander Motin Sent: Dienstag, 6. September 2022 17:12 To: Gleb Smirnoff ; Richard Scheffenegger ; Randall Stewart Subject: Why still not Cubic? NetApp Security WARNING: This is an external email. Do not click links or o= pen attachments unless you recognize the sender and know the content is saf= e. Hi guys, Does anybody know why after so many years FreeBSD still doesn't use Cubic C= C by default, relying on NewReno? Does NewReno has some benefits or Cubic = still considered experimental after so many years? In our NAS use cases we= see that while NewReno works well in data center environments with flow co= ntrol and without packet losses, Cubic may work much better over WAN, WiFi = and so on. Are there known scenarios where it is worse? Or everybody just= switch to it locally? Thanks. -- Alexander Motin --_000_MN2PR06MB6237B14D402EDA43329218719D7E9MN2PR06MB6237namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

> Perhaps we (#t= ransport) should discuss changing the default cc to cubic, and maybe revert= to new_reno for stable/14 when it branches?

 

Agree with Richard = to make cubic as default. Since the recent improvements in cubic (below), t= he use at my side didn=92t show much problem. Perhaps making it default wil= l draw further improvement/optimization.

https://freebsdfoundation.org/wp-content/uploads/2021/05/TCP-C= ubic-is-Ready-to-Take-Flight.pdf

 

 

-- =

Cheng Cui

 

From: owner-freebsd-trans= port@freebsd.org <owner-freebsd-transport@freebsd.org> on behalf of S= cheffenegger, Richard <Richard.Scheffenegger@netapp.com>
Date: Tuesday, September 6, 2022 at 11:43 AM
To: Alexander Motin <mav@FreeBSD.org>, Gleb Smirnoff <glebi= us@FreeBSD.org>, Richard Scheffenegger <rscheff@FreeBSD.org>, Rand= all Stewart <rrs@FreeBSD.org>
Cc: <freebsd-transport@freebsd.org>
Subject: RE: Why still not Cubic?

NetApp Security WAR= NING: This is an external email. Do not click links or open attachments unl= ess you recognize the sender and know the content is safe.




Hi Alexander,

Until recently, the cubic implementation was more of a proof-of-concept. I = believe my colleages here have made a reasonable job of addressing all the = edge cases, where cubic_cc would not work properly (int overflows, etc).
However, the principle algo of the current cubic_cc was based off of an aba= ndoned RFC draft, which only later got revived, and currently there is acti= ve IETF work to document all the algorithmic issues and improvements, which= had been accumulated in Linux over the years.

In short, I would expect the current cubic_cc to make a fairly reasonable j= ob, but it may not conform 100% to the upcoming revision of the Cubic IETF = RFC. But having more exposure with cubic in main may be a reasonable idea.<= br>
Perhaps we (#transport) should discuss changing the default cc to cubic, an= d maybe revert to new_reno for stable/14 when it branches?

Best regards,
  Richard


-----Original Message-----
From: Alexander Motin <mavbsd@gmail.com> On Behalf Of Alexander Motin=
Sent: Dienstag, 6. September 2022 17:12
To: Gleb Smirnoff <glebius@FreeBSD.org>; Richard Scheffenegger <rs= cheff@FreeBSD.org>; Randall Stewart <rrs@FreeBSD.org>
Subject: Why still not Cubic?

NetApp Security WARNING: This is an external email. Do not click links or o= pen attachments unless you recognize the sender and know the content is saf= e.




Hi guys,

Does anybody know why after so many years FreeBSD still doesn't use Cubic C= C by default, relying on NewReno?  Does NewReno has some benefits or C= ubic still considered experimental after so many years?  In our NAS us= e cases we see that while NewReno works well in data center environments with flow control and without packet losses, C= ubic may work much better over WAN, WiFi and so on.  Are there known s= cenarios where it is worse?  Or everybody just switch to it locally?
Thanks.

--
Alexander Motin

--_000_MN2PR06MB6237B14D402EDA43329218719D7E9MN2PR06MB6237namp_-- From nobody Tue Sep 6 19:12:43 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MMZlg5Jvkz4bT73 for ; Tue, 6 Sep 2022 19:12:47 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MMZlg17WFz3dq6; Tue, 6 Sep 2022 19:12:47 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:196b:3ddd:86ee:cfac]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTPSA id D30FF71E33506; Tue, 6 Sep 2022 21:12:43 +0200 (CEST) Content-Type: text/plain; charset=us-ascii List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: Why still not Cubic? From: tuexen@freebsd.org In-Reply-To: Date: Tue, 6 Sep 2022 21:12:43 +0200 Cc: Alexander Motin , Gleb Smirnoff , Richard Scheffenegger , Randall Stewart , "" Content-Transfer-Encoding: quoted-printable Message-Id: <55783005-52D7-4EFE-87DF-766B9AC70F6F@freebsd.org> References: To: "Scheffenegger, Richard" X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Rspamd-Queue-Id: 4MMZlg17WFz3dq6 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 193.175.24.27 is neither permitted nor denied by domain of tuexen@freebsd.org) smtp.mailfrom=tuexen@freebsd.org X-Spamd-Result: default: False [-1.69 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.995]; MV_CASE(0.50)[]; RCVD_IN_DNSWL_LOW(-0.10)[193.175.24.27:from]; MIME_GOOD(-0.10)[text/plain]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-transport@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_NO_DN(0.00)[]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; FREEFALL_USER(0.00)[tuexen]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N > On 6. Sep 2022, at 17:42, Scheffenegger, Richard = wrote: >=20 > Hi Alexander, >=20 > Until recently, the cubic implementation was more of a = proof-of-concept. I believe my colleages here have made a reasonable job = of addressing all the edge cases, where cubic_cc would not work properly = (int overflows, etc). >=20 > However, the principle algo of the current cubic_cc was based off of = an abandoned RFC draft, which only later got revived, and currently = there is active IETF work to document all the algorithmic issues and = improvements, which had been accumulated in Linux over the years. >=20 > In short, I would expect the current cubic_cc to make a fairly = reasonable job, but it may not conform 100% to the upcoming revision of = the Cubic IETF RFC. But having more exposure with cubic in main may be a = reasonable idea.=20 >=20 > Perhaps we (#transport) should discuss changing the default cc to = cubic, and maybe revert to new_reno for stable/14 when it branches? Let us discuss this on the transport call. I'm not sure it is a good idea to switch the default CC now on head to = cubic and then, when stable/14 is branched, back to newreno for stable/14. This would basically stop testing newreno = from now on and then start testing it in stable/14. Any changes of newreno code or any impact of other changes = to newreno wouldn't be observed... So if we switch to whatever in head, we should use it in stable/14, if = the testing goes well... Best regards Michael >=20 > Best regards, > Richard >=20 >=20 > -----Original Message----- > From: Alexander Motin On Behalf Of Alexander Motin > Sent: Dienstag, 6. September 2022 17:12 > To: Gleb Smirnoff ; Richard Scheffenegger = ; Randall Stewart > Subject: Why still not Cubic? >=20 > NetApp Security WARNING: This is an external email. Do not click links = or open attachments unless you recognize the sender and know the content = is safe. >=20 >=20 >=20 >=20 > Hi guys, >=20 > Does anybody know why after so many years FreeBSD still doesn't use = Cubic CC by default, relying on NewReno? Does NewReno has some benefits = or Cubic still considered experimental after so many years? In our NAS = use cases we see that while NewReno works well in data center = environments with flow control and without packet losses, Cubic may work = much better over WAN, WiFi and so on. Are there known scenarios where = it is worse? Or everybody just switch to it locally? >=20 > Thanks. >=20 > -- > Alexander Motin From nobody Sat Nov 26 02:03:48 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NJw5B68mkz4j34V for ; Sat, 26 Nov 2022 02:03:58 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NJw5B5fyrz3xbl for ; Sat, 26 Nov 2022 02:03:58 +0000 (UTC) (envelope-from bz@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1669428238; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=WwIpEJZtfdmAAK4BwhiUWHlWEKJ25f50BssOVD8TPNU=; b=Yo3KCI3Y+J6gnTzRe+vZ8nNL3dASQ3ZHNzuuRCDjLBCBFQsWxs7ztpKNivPTcCiWglbQXO X6r6b7py22JyKXZQiADuKk6vng18swmqPbOX1yEUD63yYo1nUYI2lnh++G8QKwmI+cm2jc xljM8ik+doVbNaVeu39/J6YNOBPCXeLPv/Af0rKr+R90gcDEVlQKu5qhFc3RwTijs42/wJ GgcdNyFWcUJ6Qs7vD1MoGvUfgLREWwHUSKp7Md4/d690XmUL3XabkLybCECUNcfIDBgO0c ms4gd5hHvQuzH6AvYYeU/T6EP6yzondrngYPEYrmEu8Ov3jNKebqvpnonzB+cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1669428238; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=WwIpEJZtfdmAAK4BwhiUWHlWEKJ25f50BssOVD8TPNU=; b=QxzUNzpvj2phciSaD4yMMDq1g+A9G/vbGYH+gPf2Z/Q6pG4W9pKwfbn7I135WGG7LQ776s PgHzHfp8TmbgRBiRTPTaUxEF529SI57hZa6QkbfJuB3MReMoOCV5bFGLCtgysMk4iI2kb1 bgvbozo8+1ibUsFnthfOKi8rUzODe4z8y2PvC4AMK4Pzd+1q0aUwhjnlz9QIWKLXfH5ZD6 TabW1oHY93QI+J37qx2E+DR8doJjHKNBj1Qx+dnVvxqcC63RWyZoM0apD4ZllTptjsDsWq YC8TjPJg344mLTSDR6enokWSQ92UVMGgBCUh8TZ3iEmaCWrufTqGkiMxN7skBw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1669428238; a=rsa-sha256; cv=none; b=Av+tXmzw5rNuPN+1Bn6RTjJ7O5Vks+ehaZ1CuWoat36O0xly6uT12rWrwdyiVNZke8Uu9J fdJrJxVg46cVb24AkSJqvM+ejamG3z5x5SmAeffXePUT7ZWPJT7LCnOoolGWL/aLlHdDyt 3Ry6LYsm1WvxDimhK+ONdjsYRSa4JUOhgU0N1Z6KDaXYIHAqL9IPlKFxoT4+xKrbmEz65D PGOmqaOGK8fVxnYpTR4timlaK29OGVwthlURLs6gbOy7I/7sdPNf4pLa79BViIgE9VWWWD ZdWwdn97C2kljixGIfboPrHUjTRLfzo/VmRYqQR0qjyXqUag1mxbmI6W88K51A== Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:13b:39f::9f:25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) (Authenticated sender: bz/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4NJw5B3v4gz11tL for ; Sat, 26 Nov 2022 02:03:58 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 333BE8D4A226 for ; Sat, 26 Nov 2022 02:03:53 +0000 (UTC) Received: from content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 67D1B5C3A833 for ; Sat, 26 Nov 2022 02:03:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) by content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (amavisd-new, port 10024) with ESMTP id mWHrH8DnAbr7 for ; Sat, 26 Nov 2022 02:03:51 +0000 (UTC) Received: from strong-iwl0.sbone.de (strong-iwl0.sbone.de [IPv6:fde9:577b:c1a9:4902:b66b:fcff:fef3:e3d2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 9F7FD5C3A830 for ; Sat, 26 Nov 2022 02:03:49 +0000 (UTC) Date: Sat, 26 Nov 2022 02:03:48 +0000 (UTC) From: "Bjoern A. Zeeb" To: freebsd-transport@freebsd.org Subject: Khelp module "ertt" can't unload until its refcount drops from 1 to 0. Message-ID: <4o1pnqp7-1669-626-q97-96po1520rp9s@mnoonqbm.arg> X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-ThisMailContainsUnwantedMimeParts: N Hi, my tree is a few days old but I see these on reboots: Khelp module "ertt" can't unload until its refcount drops from 1 to 0. Anyone wants to go and have a look? -- Bjoern A. Zeeb r15:7 From nobody Sat Nov 26 17:44:26 2022 X-Original-To: freebsd-transport@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NKJyQ18JJz4hg5l for ; Sat, 26 Nov 2022 17:44:30 +0000 (UTC) (envelope-from rrs@netflix.com) Received: from mail-vs1-xe31.google.com (mail-vs1-xe31.google.com [IPv6:2607:f8b0:4864:20::e31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NKJyP6C6zz45Tv for ; Sat, 26 Nov 2022 17:44:29 +0000 (UTC) (envelope-from rrs@netflix.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vs1-xe31.google.com with SMTP id i11so5900896vsr.7 for ; Sat, 26 Nov 2022 09:44:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=P8BvRL7shIcBP/oiTHdP3L0Mom2hDE7pJPVwAyyn988=; b=YqiNgXYvaXllp0i9pwN/vCDwz9ljWtgX8r0x0TV6yexwK62zq0dat91vaZwTOuHLuY sVchCokIvD18c5k9n43OOn4i87NmT3sKigEnEU6RKv7E2DUNmNy5Lws//kJCZtL1540b Pvej2sDPxH24ImpFRIDVxdCr93MoIrvnvBEwU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=P8BvRL7shIcBP/oiTHdP3L0Mom2hDE7pJPVwAyyn988=; b=fWCSjprY5D4GanuYv25+l5FQnaJ4xM7jDvapAhQqNwBqOCsXGupRmDMJxifX5PBbyW CN6sCPiaxLmwGLI3lbjkvuWZ0QGCKd5ECFh1EGUoxv9bq2oKYvrIsNSiDJc5Cj5oh9k/ HcyPQMyAtI50NJjPF91uzG7QBE+mNnUCCvcc7REqqoCXF7/ALLK0B1p/7/an+UBO50kh gEhZy/V3Cm06W19zdraXanJSqOH4HpmEWaIdBjCMrbrj7CrSorTQC5vl0AXfeXgnz2PO yxpDEQtPoV8nbaVpNkk8FKlO+Vl25Fup0uDEmjDy53qlBw1pjPbbtTs2LzZLUbD2tGpA WXGA== X-Gm-Message-State: ANoB5pkNYYwbfMBdFTlZyQoEZtt+UieCF9/AbkIvhOGD7GeRqxxsSKTX Q3HO+zmwlY5nljvo4r8e5jm4Yg== X-Google-Smtp-Source: AA0mqf7YPTnldNr368OENRwHKaoIk8kDJphkfKQhByFdVaR5gE8XLeMR+DYT6GlwdDtOPTwZ8tFjSw== X-Received: by 2002:a67:f54e:0:b0:3b0:4e31:10f7 with SMTP id z14-20020a67f54e000000b003b04e3110f7mr14146004vsn.73.1669484668239; Sat, 26 Nov 2022 09:44:28 -0800 (PST) Received: from smtpclient.apple (2603-9001-5e00-74cf-bc4a-5b69-19b7-f464.inf6.spectrum.com. [2603:9001:5e00:74cf:bc4a:5b69:19b7:f464]) by smtp.gmail.com with ESMTPSA id k2-20020ab059c2000000b00409e1d5e75esm914027uad.37.2022.11.26.09.44.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 26 Nov 2022 09:44:27 -0800 (PST) From: Randall Stewart Message-Id: <78A51FFF-C51B-4AD3-BDC2-A7F4631778C3@netflix.com> Content-Type: multipart/signed; boundary="Apple-Mail=_A93F3C0B-42F5-4D5A-A53B-5B114E10ECC7"; protocol="application/pkcs7-signature"; micalg=sha-256 List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-transport List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-transport@freebsd.org X-BeenThere: freebsd-transport@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: Khelp module "ertt" can't unload until its refcount drops from 1 to 0. Date: Sat, 26 Nov 2022 12:44:26 -0500 In-Reply-To: <4o1pnqp7-1669-626-q97-96po1520rp9s@mnoonqbm.arg> Cc: freebsd-transport@freebsd.org To: "Bjoern A. Zeeb" References: <4o1pnqp7-1669-626-q97-96po1520rp9s@mnoonqbm.arg> X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4NKJyP6C6zz45Tv X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_A93F3C0B-42F5-4D5A-A53B-5B114E10ECC7 Content-Type: multipart/alternative; boundary="Apple-Mail=_67F5D8D8-6EBD-47AF-A76D-6021BB4DA94B" --Apple-Mail=_67F5D8D8-6EBD-47AF-A76D-6021BB4DA94B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 BZ: I will look at it Monday for you, but I suspect that it is because ertt = is possibly used by Cubic, and someone recently made that part of GENERIC and got rid of newreno by default. If that=E2=80=99s so then we probably need to = make ertt just part of the kernel.. not sure.. I will look into it Monday Best wishes R > On Nov 25, 2022, at 9:03 PM, Bjoern A. Zeeb wrote: >=20 > Hi, >=20 > my tree is a few days old but I see these on reboots: >=20 > Khelp module "ertt" can't unload until its refcount drops from 1 to 0. >=20 > Anyone wants to go and have a look? >=20 > --=20 > Bjoern A. Zeeb = r15:7 >=20 ------ Randall Stewart rrs@netflix.com --Apple-Mail=_67F5D8D8-6EBD-47AF-A76D-6021BB4DA94B Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 BZ:

I = will look at it Monday for you, but I suspect that it is because ertt is = possibly
used by Cubic, and someone recently made = that part of GENERIC and got
rid of newreno by = default. If that=E2=80=99s so then we probably need to make ertt = just
part of the kernel.. not sure.. I will look = into it Monday

Best wishes

R

On Nov 25, 2022, at 9:03 PM, Bjoern A. Zeeb = <bz@FreeBSD.org> = wrote:

Hi,

my tree is a few days old = but I see these on reboots:

Khelp module = "ertt" can't unload until its refcount drops from 1 to 0.

Anyone wants to go and have a look?

--
Bjoern A. Zeeb =             &n= bsp;           &nbs= p;            =             &n= bsp;  r15:7


------
Randall = Stewart



= --Apple-Mail=_67F5D8D8-6EBD-47AF-A76D-6021BB4DA94B-- --Apple-Mail=_A93F3C0B-42F5-4D5A-A53B-5B114E10ECC7 Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCCAzYw ggMyMIICGqADAgECAgqxywKqrHPB2ybTMA0GCSqGSIb3DQEBCwUAMEcxGDAWBgNVBAMTD1JhbmRh bGwgU3Rld2FydDEeMBwGCSqGSIb3DQEJARYPcnJzQG5ldGZsaXguY29tMQswCQYDVQQGEwJVUzAe Fw0yMTAxMjQxMjIwMTRaFw0yNjAxMjQxMjIwMTRaMEcxGDAWBgNVBAMTD1JhbmRhbGwgU3Rld2Fy dDEeMBwGCSqGSIb3DQEJARYPcnJzQG5ldGZsaXguY29tMQswCQYDVQQGEwJVUzCCASIwDQYJKoZI hvcNAQEBBQADggEPADCCAQoCggEBAMUAht2nr/NFlK+tmmN9PdO3DBPfeYh9fLcbVihR+/dipO41 AsFy9y+2uDVaFhTEvp406P0o9PQQTuYXqrCr76eWQIj3V787e1WKjTup1mIyQeWHGf1gvb/7vmI2 zHg6QZEIC4W8xeO8SLKyHiwlFHZn8Rn1HxtB7Ge+NulygkgUgJYhXD5E29jVGXAc6Qn9Vr9AexPf KaOhHCaNB/Twcinayz6D8CO/Ym1LOs3+ceSOa4cB07fepmbqDSXDkOeA3U7KLaluHrRTlj6DO+JU nqKXX7jJ68KTYSZ0qH4fZsk8cxFkwYI/3HDJi+oF+FDkf7SRo1Q2w+e3M/5MReLIQ7sCAwEAAaMg MB4wDwYJKoZIhvcvAQEKBAIFADALBgNVHQ8EBAMCB4AwDQYJKoZIhvcNAQELBQADggEBAHJfum1j 1WIVFjOJT/hqMIN751aXkablmwesW94lNJKjslPULbbcP5nZGg2lGpHcZ+0I5F/1TTiEsT2H2rhA uAnSsUxTpxRA+aoe+xtJOa5vle3CprhFkHAvB7EIoLiNaPd0DNK6kKYsbvr5Z5Eq7TF3SIO77Qh9 /8VgUfOb0ARDgix59Q6MM0NmIabEwh0cDWQYlGgDDtN9DNk5PGM4pjs48suwEdLmFTMOmGTkCp7I Vq6iHDNinBiB6+BB4VYMAO1o9qS+0pnfdmPJybt0zVGrhm/c1Fmm3Jec7NEuiKeXmhPIwdwMkKyp AsX0sHuFCYwioBTHHZpvnir+H2rRakgxggHrMIIB5wIBATBVMEcxGDAWBgNVBAMTD1JhbmRhbGwg U3Rld2FydDEeMBwGCSqGSIb3DQEJARYPcnJzQG5ldGZsaXguY29tMQswCQYDVQQGEwJVUwIKscsC qqxzwdsm0zANBglghkgBZQMEAgEFAKBpMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZI hvcNAQkFMQ8XDTIyMTEyNjE3NDQyNlowLwYJKoZIhvcNAQkEMSIEIK+knr02veBWIwLD6g/Tmewa 8+Wjh/+qJS5T6LP8uCB+MA0GCSqGSIb3DQEBCwUABIIBAEBfR7OsPAXykBGfhaTQt7IXd9y3YPo0 nVsr7Cgn6K3EEAXaZz076v3OeKcAqZIpGdmkzeV1ZxnAvkGD00QSbxV8uut0ZBDkMw5SQc7aBddN NNLQGgPju4F/7n/aBkGIJYkZrB8h8YnlSrqIhXVi+nZPCQXu33jgJutknTImEQGWVq6Xt28Oe/GF bO09D/kNiLCAFFro00ZxZi/4CqJ+eyPICqoJ4EXvE99XTwkdirwvtnBOevdz28qy8LhZKLd1JxPb HB/QvaBVagUln78HPyTyov8A39F9HnxJuHuaJOc676AZOgTyxQK4oaZF5BtqHopgz9cNU6WWnPGl pDu4mTUAAAAAAAA= --Apple-Mail=_A93F3C0B-42F5-4D5A-A53B-5B114E10ECC7--