From owner-freebsd-hackers@freebsd.org Wed Nov 18 07:13:39 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 90FC12E5122; Wed, 18 Nov 2020 07:13:39 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CbYv63KxNz4lxC; Wed, 18 Nov 2020 07:13:38 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-oi1-x22f.google.com with SMTP id d9so1166093oib.3; Tue, 17 Nov 2020 23:13:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=R/2r2GAn1M0Sooc530sL6kXaFRJYwYUK1VxCizp7Zws=; b=gxWr3zTc8ucz71SasYepbpP8pJoc7BANSLvEYEHvcBsXHhii477f21Z9XqyWsu1x0u iex8LZSyYY/E/DRcu4HmrT6O1rGBMj9PdYRfm2lJhERIFUPAQUvp+tZLYPsAUNwhCBM/ Op/jQt81NE7mktGEr44bBnx94Uv6defyeaiuLaWPYojHK+G0SFLz5vftZfd10qM7nhB3 PMb8SfwUfWGRJF/o0UOY7NFFDfRSfdeWaRGR9AkiRhh2P33reWXKPrxbQmMpXVU4fy1l lCAlYYiRANkCcRFgdZwR0KrCyDZljCYoI22NHm/cNifkB8/M1mQ4cSjRCEHq0vVjPgcU GDKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=R/2r2GAn1M0Sooc530sL6kXaFRJYwYUK1VxCizp7Zws=; b=aQOrYEp+vOKIZNtwpk9puXLDlLIMvTgpPl8YbnMqsZ6AEZQLrVncc4SGcCoUKRtjJk bq2Y6dj0+H7Xth7AyQUq9rka0aS5JcEYLjMXD+cBVf5VUOWyi7f7rHroP6T9P+LMuJAK siMUnWN9L5bO2OFyBI0PGcxfotZGUHblsAwd+L3nDH/Afd0cx1zWzCmEyx4Qkmc5ArPQ 8DmC8IUBWywv2S2bv65eWS8TavTSLKqXVXkgFZQUVyRHEzVhaWWyTTOxdU1yOU6qW/Aw plDP8GghK30o3FUvRARdlvBp7UGujQ4d81iAbdPB1r7rU5qHlv8IgOxOOi2P9Ywil1wm GibA== X-Gm-Message-State: AOAM530KMoNDoJtjFpAXDpy+8XzbqqzOqRMQ9PfOGBroG9Po2mDb2kae GngsRqATA1Laoft/xXPf5k2RQVj8gFtzNOLjI/R1ty8V X-Google-Smtp-Source: ABdhPJxXfaRWI4A1gYBm1BOKyhbFBhqPvFDzDU2vc7K2a24cF4tYAlDtp0+bNnpYTOUtTzYfdIt0Bc/d7Q/5FLXiSE4= X-Received: by 2002:aca:6548:: with SMTP id j8mr1932453oiw.109.1605683616785; Tue, 17 Nov 2020 23:13:36 -0800 (PST) MIME-Version: 1.0 From: Rajesh Kumar Date: Wed, 18 Nov 2020 12:43:23 +0530 Message-ID: Subject: Netmap bridge not working with 10G Ethernet ports To: freebsd-net@freebsd.org, FreeBSD Hackers X-Rspamd-Queue-Id: 4CbYv63KxNz4lxC X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=gxWr3zTc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rajfbsd@gmail.com designates 2607:f8b0:4864:20::22f as permitted sender) smtp.mailfrom=rajfbsd@gmail.com X-Spamd-Result: default: False [-3.98 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::22f:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::22f:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22f:from]; NEURAL_HAM_SHORT(-0.98)[-0.978]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-net,freebsd-hackers]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2020 07:13:39 -0000 Hi, I am testing a 10G Network driver with Netmap "bridge" utility, where it doesn't seem to work. Here is my setup details. *System under Test:* Running FreeBSD CURRENT. Has two inbuilt 10G NIC ports. *System 1:* Running Ubuntu, whose network port is connected to Port1 of System Under Test *System 2:* Running FreeBSD CURRENT, whose network port is connected to Port 0 of System Under Test. Bridged the Port0 and Port1 of System Under Test using the Netmap "bridge" utility. Able to see interfaces coming up active and Link UP. # bridge -c -v -i netmap:ax0 -i netmap:ax1 Then tried pinging from System 1 to System 2. It fails. *Observations:* 1. ARP request from System 1 goes to bridge port 1 (netmap_rxsync) and then forwarded to port 0 (netmap_txsync) 2. ARP request is received in System 2 (via bridge port 0) and ARP reply is being sent from System 2. 3. ARP reply from System 2 seems to be not reaching bridge port 0 to get forwarded to bridge 1 and hence to System 1. 4. Above 3 steps happen 3 times for ARP resolution cycle and then fails. Hence the ping fails. On Debugging, when the ARP reply is being sent from System 2, I don't see any interrupt triggered on the bridge port 0 in system under test. Netstat in system under test, doesn't show any receive or drop counters incremented. But as I understand netstat capture the stats above the netmap stack. Hence not reflecting the counts. *Note:* a) I tried with another vendor 10G NIC card. It behaves the same way. So this issue doesn't seem to be generic and not hardware specific. b) Trying with another vendor 1G NIC card, things are working. So not sure, what makes a difference here. The ports in System 1 and System 2 are USB attached Ethernet capable of maximum speed of 1G. So does connecting 1G to 10G bridge ports is having any impact? c) We have verified the same 10G driver with pkt-gen utility and things are working. Facing issue only when using "bridge" utility. So, wondering how the ARP reply packet is getting lost here. Any ideas to debug? Thanks, Rajesh.