From nobody Thu Sep 12 17:16:18 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X4PHB2sZjz5WHgY for ; Thu, 12 Sep 2024 17:16:22 +0000 (UTC) (envelope-from cryintothebluesky@gmail.com) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X4PH90cl7z4TVy for ; Thu, 12 Sep 2024 17:16:21 +0000 (UTC) (envelope-from cryintothebluesky@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=H3Q5mTu4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of cryintothebluesky@gmail.com designates 2a00:1450:4864:20::32b as permitted sender) smtp.mailfrom=cryintothebluesky@gmail.com Received: by mail-wm1-x32b.google.com with SMTP id 5b1f17b1804b1-42cb57f8b41so93205e9.0 for ; Thu, 12 Sep 2024 10:16:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726161380; x=1726766180; darn=freebsd.org; h=content-transfer-encoding:mime-version:message-id:subject:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=dfh9wOSY+X9YfzrGq3lFAuWQ3d+Jm2KgHRWMKkW6QmA=; b=H3Q5mTu4IrqoRR9eOpowyoJ/5r+0gxnHVKdqS2YlbkGmLRdde+k+FGJe0hmtIYCvHT gs5qNVlGd7E4VzgnexYw7fdDIzj8Vj9tdXDAnbtzAVtw1Xp/8RM+8xvukyU7SgW1OZBy B5CpV0NOZdBQx1/l1UIurthcFa0VoobaqzedkBLz0qnqo/cKmf6xNYe6oMaS1rDss9Vw pRxlgfHDwBDXN/C8x1T8sqNKwjVOOdWa4wwlxPSw2j0blTSWdLCRCFQvAprGIZtooBcE ygbp0O7K4jcdpuabemswPd+yawIH3RAV0DsREt3H+if39GIYpPaCwaZgW2nRoa48BEBj GtUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726161380; x=1726766180; h=content-transfer-encoding:mime-version:message-id:subject:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dfh9wOSY+X9YfzrGq3lFAuWQ3d+Jm2KgHRWMKkW6QmA=; b=vJkEdk7Fyb0oMP2miidETYiFcXX4X1BFFDUW6MnPAEn9eKq3vm419RpdUq4Wy0R6XY aCWoF0z/OOToSPYd7UcIAfnA16ivkt0y3XZCnEdUBw1Yjj0r8SvmrJvrhWB1T24ZLL9M pU/mkEIwLpSPiwmr8s8fvxOheGqdk0o5HmPTtPy32N9kK13VgG1ORvJm6tWBxZMtWRVO vCAErpzG5o/MJn4wUJrfndFmjlsIuNEYdNAFuZXWNCEJYzPj4FALU5OKS+458Q2Jf+M4 +tBUjGmoXqGneplb9eVV2ytfzSWiaDUvgJr9+HhhX6hrs3uFR7+Bbegcm8Unr/shzyCS rgTQ== X-Gm-Message-State: AOJu0YwnJEV1KwFHw2GC5w/JLwnfSaHC0dfEIIrNW+vw6H+Vvf9AJcsz W3QNBBJ3jR+DcEIZpT+TN5bFTBrvV0mn79POSk3+znrFIF+SIOk0aOkftA== X-Google-Smtp-Source: AGHT+IEGnE47OefgnNNqiwRHNOngcVOap7AeXy0XpYGqeZYI2XaumdfAmAZpvRMkKydH4VtqE/Jrgw== X-Received: by 2002:a05:600c:1c12:b0:42c:b5a6:69bd with SMTP id 5b1f17b1804b1-42d964e0027mr1227745e9.30.1726161379489; Thu, 12 Sep 2024 10:16:19 -0700 (PDT) Received: from z600.home.lan (118.129.159.143.dyn.plus.net. [143.159.129.118]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42caeb8afc9sm181077415e9.44.2024.09.12.10.16.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 10:16:18 -0700 (PDT) Date: Thu, 12 Sep 2024 18:16:18 +0100 From: Sad Clouds To: freebsd-net@FreeBSD.org Subject: Performance issues with vnet jails + epair + bridge Message-Id: <20240912181618.7895d10ad5ff2ebae9883192@gmail.com> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.995]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_FROM(0.00)[gmail.com]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::32b:from] X-Rspamd-Queue-Id: 4X4PH90cl7z4TVy Hi, I'm using FreeBSD-14.1 and on this particular system I only have a single physical network interface, so I followed instructions for networking vnet jails via epair and bridge, e.g. devel { vnet; vnet.interface = "e0b_devel"; exec.prestart += "/jails/jib addm devel genet0"; exec.poststop += "/jails/jib destroy devel"; } The issue is bulk TCP performance throughput between this jail and the host is quite poor, with one CPU spinning 100% in kernel and others sitting mostly idle. It seems there is some lock contention somewhere, but I'm not sure if this is around vnet, epair or bridge subsystems. Are there other alternatives for vnet jails? Can anyone recommend specific deployment scenarios? I've seen references to netgraph which could be used with jails. Does it have better performance and scalability and could replace epair and bridge combination? Thanks.