From owner-freebsd-hackers@freebsd.org Tue Nov 26 20:42:25 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 436B61C0DBB for ; Tue, 26 Nov 2019 20:42:25 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47MwnW3DWbz4FFR for ; Tue, 26 Nov 2019 20:42:23 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: by mail-io1-xd30.google.com with SMTP id s3so22231821ioe.3 for ; Tue, 26 Nov 2019 12:42:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Gxp6JtYSCjfIQk2JBdcLicHwUTtoRf4BnC81Mm7e0SQ=; b=uWAKKjFSbVhEca/kHuPA8cJoVnjfP6yOTN1BeDho1IEatEnjha9JojeIYZ5T6NHvNd lmoNNHw53SPGujeuf65J/jebnpS5QO3zUmejMB8DIKaEbxiQgvVYACh+yzCn+2bQDURE R+sa0KJVjuBERkSijMaSG5A7xk3y98eeHJB96vDJD8UeLJwrfASoln3jLRABLtMr6YNa MmBQcu1nNLC5evetpE6NtEyv5yaUbjl3YICMV9Vzk9+6iusr8TZSiZloP70shy/OG1cF GocKAMIChShEIv7VZLtyDagrsmY8pM9x+b9G6LWRs5DOcP8kBPinyXKZ5XACLYBZ91Qf lyAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Gxp6JtYSCjfIQk2JBdcLicHwUTtoRf4BnC81Mm7e0SQ=; b=P2N4zE3JA2ytoCCyHoDz0k0BlaPYpL+7qrIGjgEDlD89mGxRG3CESUiPzVWpNG0D4Q GoXi1G/AltZPHxnAbUx67Gfn6sHvnl5x7+KJmtXWoeCuYVGaL2t7EiDWKonPDP6htYBv KkMffVf0fRqA2qq42vtdvnlI7k0L3lqVjmcI1iZJyEka6pHI6GnFa6NzqTHDDK97K1oI OBCYGms7x/iaz+H7JZKgLo42mZ6VptlZkJvhXVqIQi3cLlQRViPZIRjKfnG7IgNANzkv DpPojJa6aef+hWaG6ffDnfy8R0eC5RDhA6yH8oXjfS7EZxhakVpHbXWG25rkl+zEdNZN WLSQ== X-Gm-Message-State: APjAAAUm1gZQw/AXx4NQNFKj+M2sZ2Amcv4SqcW+Vwubt+rIvv/1FL9o c1VPuhDdhoXlPFnnug5H6PDT3os6atc+oN4RHc0mm3Bc X-Google-Smtp-Source: APXvYqyODbbsnOoYO2HuAwBwcX+Qbs41nTdvXM2LlDo0MJNTCWKv36d6BtkaAopwkM2ln5TdA+KxnhSnXpiqqTz5dho= X-Received: by 2002:a6b:c982:: with SMTP id z124mr34542229iof.291.1574800941127; Tue, 26 Nov 2019 12:42:21 -0800 (PST) MIME-Version: 1.0 From: Ali Mashtizadeh Date: Tue, 26 Nov 2019 15:42:10 -0500 Message-ID: Subject: Mellanox NIC + Bridge + VM traffic issues on 12.1-RELEASE To: FreeBSD Hackers X-Rspamd-Queue-Id: 47MwnW3DWbz4FFR X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uWAKKjFS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mashtizadeh@gmail.com designates 2607:f8b0:4864:20::d30 as permitted sender) smtp.mailfrom=mashtizadeh@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(0.00)[ip: (-5.94), ipnet: 2607:f8b0::/32(-2.27), asn: 15169(-1.95), country: US(-0.05)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[0.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Nov 2019 20:42:25 -0000 Hello Everyone, I've noticed something strange happening after I upgraded to FreeBSD 12.1 on one machine. My VM is bridged to a Mellanox ConnectX4 that connects to our cluster of machines. Traffic appears to go between the VM and external machines except for ARP replies. I noticed ARP replies from the VM do not make it to the external network and appear to be dropped by the NIC. All network connectivity is fine when using an Intel ixl NIC. Has anyone seen this? Does this have something to do with the NIC offloads? I experimented with them but did not succeed in fixing ARP traffic. Best, Ali