From owner-freebsd-net@freebsd.org Tue Aug 18 21:43:44 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 30CFC3A85E4 for ; Tue, 18 Aug 2020 21:43:44 +0000 (UTC) (envelope-from zec@fer.hr) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2062a.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e1a::62a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWPYW0Jlsz3TqC for ; Tue, 18 Aug 2020 21:43:42 +0000 (UTC) (envelope-from zec@fer.hr) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PdbFbxoH8tYm/9YYF0H0yiWr1rBP4T0cyz0WAFgw0gfZwaymJuDRrHEQF2sbncEajIDVvIc0Nz/CYt2CFj7rRtZOEkIIcqyo0ihAp0vWT7+NMACVi3HAJP2BlZayWvmk0TwM3gUD85LoJ/hIaJslXrQ7Sv7QXNj9ehKkYjvJf6ZjLJeuiW6ypiON+MrjUZAXlZSsX2sKdhAQSBmfw3m7Y33MAR8qWcym6ChnJoXYUwBkz/quVdgK6KdmuFXbtsKRA/mkHMJi/MHW0iVxFL3+oZ0yb442BL873+HwWdyQWV/eKKjrmbT7rxVbglt9Kd9sr7NheiqwAaEjBg1ktjN9Fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a7ZAmfSiluz+6JfR9mfKABpqh69CgBBvQb5ywEJuVWs=; b=aDo0vdfT+jpGQpb2ObQS0+F9BjC2b3T5i9MCruGit8jUrXoW014BoT54Rd/TeqSmPJhNPQ4KN6rmTN5cvK5u8I3Vq4SGn0VlQH7ZSMj8FXATTSIBoPNpyca7EqozGCrGq1kT5pvwgzh5rKobz98/HJQqf95hgLiVrG25FP+GhOBO8aoFzpzAzx0FNKdiIKO0J0BXAGs2fribsiqyJL/WpC5bujwjq1fccokHwcu+oEApAPj2fp2kUpGXCLMKCASycjD0fWgA8AXUN0BdzCvyRShvyiLQVP0AohTOlwK/5G9UTBfmb/3vOS0Z+e6CBtl7EzIvW51Ntuuzcpx6VGINoQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fer.hr; dmarc=pass action=none header.from=fer.hr; dkim=pass header.d=fer.hr; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ferhr.onmicrosoft.com; s=selector2-ferhr-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a7ZAmfSiluz+6JfR9mfKABpqh69CgBBvQb5ywEJuVWs=; b=KMBhG+BDCNZVJTXetD6nWcj/qEGUAjcmhFN7po/E0uy9uzwDGrH2VaV0f+EH1mwE0s4fVjjlDW1Jkfs0lG3DFqnelKJ5iuPw2jp0yX+YpODdYxErq3LbUAsbAlYotZH80NOY+/DcHoHrXdkjW3fEIEfJkXd4Nq9opkER2Zqvw4c= Received: from VE1PR08MB4783.eurprd08.prod.outlook.com (2603:10a6:802:a9::16) by VE1PR08MB5646.eurprd08.prod.outlook.com (2603:10a6:800:1a9::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3283.15; Tue, 18 Aug 2020 21:43:39 +0000 Received: from VE1PR08MB4783.eurprd08.prod.outlook.com ([fe80::f8f2:29a2:4263:4647]) by VE1PR08MB4783.eurprd08.prod.outlook.com ([fe80::f8f2:29a2:4263:4647%6]) with mapi id 15.20.3283.026; Tue, 18 Aug 2020 21:43:39 +0000 Date: Tue, 18 Aug 2020 23:44:16 +0200 From: Marko Zec To: Ryan Stone Cc: freebsd-net Subject: Re: Is anybody using ng_pipe? Message-ID: <20200818234416.5aec459c@x23> In-Reply-To: References: <20200818205725.2e03c8cb@x23> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; amd64-portbld-freebsd11.3) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: VI1PR04CA0122.eurprd04.prod.outlook.com (2603:10a6:803:f0::20) To VE1PR08MB4783.eurprd08.prod.outlook.com (2603:10a6:802:a9::16) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from x23 (31.147.2.0) by VI1PR04CA0122.eurprd04.prod.outlook.com (2603:10a6:803:f0::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3305.24 via Frontend Transport; Tue, 18 Aug 2020 21:43:36 +0000 X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; amd64-portbld-freebsd11.3) X-Originating-IP: [31.147.2.0] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fbb57606-c030-460b-d693-08d843bfc4ca X-MS-TrafficTypeDiagnostic: VE1PR08MB5646: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: beyHjUjNkSvD4Jmyjj+BLRDv7mcdk1vTUu6kQkwZa0SmReprW1RNOMTuWLKK8VngteCr8D9idBzmvQXpFeyLQLx7mnbgvdTiZlNbTdSwB0uLc1wHe9+V9DUCOMGxfAEkKQBZZxhD63fpCA5z4/vYNp6F69CK0ckxzP9lRci4PQgEO/IdKDR4Fh6Xy/+AV5woG/KEQ0rUikZuBkM2ltQq19s5b1MzjuYfW3xT9ssXkY/I/fQBrwVWcsTwOK5NKX8Lx4OBzoC01LJBcsi1msQ1dV9MVTyVzEqadDSBmFpMQsbJQ3Rgfcfy4rsSVfTkmIBDXafs8DKNbbR+VrETG/N4/w== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VE1PR08MB4783.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(346002)(366004)(136003)(396003)(376002)(39840400004)(956004)(6496006)(9576002)(4326008)(26005)(83380400001)(55016002)(8676002)(9686003)(86362001)(8936002)(53546011)(478600001)(52116002)(6916009)(6666004)(1076003)(33716001)(2906002)(5660300002)(66946007)(186003)(66556008)(66476007)(16526019)(786003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: UJSTILnjC+lJL/kIgv2wq3ds5HX0BhH3DGuYkGeu4KBAxxTybVsVJIPho8zTegyNHu14ppU34ikJIVodYpGMu5iRVMBmPK+hgPSHLVHSEUggdA88vmfo1uA2vTjJDN0Vgq9KUphOVh2eTr8yiW8uKRuxta1Srx/wem3hAkqLxagdwHTpcprCEqXq3OJa6my+oyl3k7hf5lY+GVHCvFNYLK0u+Md0eLGeDWcaxsGGH1RiwrNbaqwNjN6T9gWfql2UUtxyuXMIjq6llhVImU/dF9AGoAC5XEBS+7EPMp/4QYtMLLLffRNTQnBPRAdrIRQintZSrt19h1bxN1DqyDTWt9iQDz9MBSgZU8ySWlqJgJ6jpZIQ5vcOY86IN79YHVmJmrexYSV/BbBfS06ZA9GqTs/0XZu94ijYtbIHrsBz83JfSy3bB5okpl0dwaK7h90mLIra5R7mkoEO/7qEdx/X2zWe+TBdGjybh8ViD6+TJYdGIVGpKnuwcLwk3NKF/+Rv4xplYyCYGWXNwlwZrZXc1tOmEQkrCrk78hVCxfGkTRnn6etElG15JHzJjtS6HjpqH/UZ28K5RlOHXUvSsCtEzFucnwK1wYITza5RAjAaSAAQuuHuC6B/qiXmarTB7nET X-OriginatorOrg: fer.hr X-MS-Exchange-CrossTenant-Network-Message-Id: fbb57606-c030-460b-d693-08d843bfc4ca X-MS-Exchange-CrossTenant-AuthSource: VE1PR08MB4783.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2020 21:43:39.4575 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: ca71eddc-cc7b-4e5b-95bd-55b658e696be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 64KqLPwnGXVs/W4YpSSq0EMq/91pRWonH7IT/EBp4bvo7Wftn6OH5O9jRhFxjAAU X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5646 X-Rspamd-Queue-Id: 4BWPYW0Jlsz3TqC X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ferhr.onmicrosoft.com header.s=selector2-ferhr-onmicrosoft-com header.b=KMBhG+BD; dmarc=none; spf=pass (mx1.freebsd.org: domain of zec@fer.hr designates 2a01:111:f400:7e1a::62a as permitted sender) smtp.mailfrom=zec@fer.hr X-Spamd-Result: default: False [-0.51 / 15.00]; MID_RHS_NOT_FQDN(0.50)[]; R_DKIM_ALLOW(-0.20)[ferhr.onmicrosoft.com:s=selector2-ferhr-onmicrosoft-com]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[zec]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[fer.hr]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[ferhr.onmicrosoft.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a01:111:f400:7e1a::62a:from]; NEURAL_HAM_SHORT(-0.51)[-0.511]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-net]; RECEIVED_SPAMHAUS_PBL(0.00)[31.147.2.0:received] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 21:43:44 -0000 On Tue, 18 Aug 2020 17:01:37 -0400 Ryan Stone wrote: ... > On Tue, Aug 18, 2020 at 2:56 PM Marko Zec wrote: > > The probability that a frame is completely unaffected by BER events, > > and thus shouldn't be dropped, is currently computed as > > > > Ppass(BER, plen) = Psingle_bit_unaffected(BER) ^ Nbits(plen) > > The problem is in its calculation of Psingle_bit_unaffected(BER). The > BER is the fraction of bits that are affected, therefore it is the > probability that a bit is affected. But for some reason, > Psingle_bit_unaffected(BER) is calculated as 1 - 1/BER rather than 1 - > BER. Since in ng_pipe we define BER as an one error in BER bits (integer value), wouldn't your formula P = 1 - BER yield results less than or equal to zero for all non-zero values of BER? The domain of P is [0..1], so that won't fly. Your analysis seems to be based on an assumption that the BER parameter is given as a multiplier to 0.5**48, which it is not. The proper fix would be to clarify the current BER parameter semantics in ng_pipe(4), not to break bridges with the existing scripts / software which rely on ng_pipe. Cheers, Marko > This leads to the probability table being wrong. For example, > given a BER of 23500000, the probability that a 1500-byte packet is > not dropped is: > > (1 - 23500000/2**48)**(1500 * 8), which is approximately 99.00%. > > However, ng_pipe calculates a fixed-point probability value of > 281460603879001. To calculate whether a frame should be dropped, > ng_pipe takes this probability value and shifts it right by 17, > yielding 2147373991. It then calls rand() to generate a random number > in the range [0,2**31-1]; if the random number is larger than the > probability value than it is dropped, otherwise it is kept. The > chances that a packet is kept is therefore 2147373991/(2**31 - 1), or > about 99.99%. > > It's easy enough to fix this one, but I wasn't sure that it would be > so easy to fix the TSO/LRO issue without significantly increasing the > memory usage, so I wanted to gauge whether it was worth pursuing that > avenue or if a simpler model would be a better use of my time. The > feedback is definitely that a simpler model is *not* warranted, so > let's talk about fixing TSO/LRO. > > On Tue, Aug 18, 2020 at 1:47 PM Rodney W. Grimes > wrote: > > Hum, that sounds like a poor implementation indeed. It seems > > like it would be easy to convert a BER into a packet drop > > probability based on bytes that have passed through the pipe. > > I'm not quite following you; can you elaborate? Would this solution > require us to update some shared state between each packet? One > advantage of the current approach is that there is no mutable state > (except, of course, when configuration changes).