From owner-freebsd-net@freebsd.org Mon Mar 22 00:18:43 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D304657322A for ; Mon, 22 Mar 2021 00:18:43 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F3Zq70ybTz4dVw for ; Mon, 22 Mar 2021 00:18:42 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-qk1-x72b.google.com with SMTP id g15so8981239qkl.4 for ; Sun, 21 Mar 2021 17:18:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=dD7Aon0Ikab8VzEzhM02bk+8s2VPMVC6vBxzhxxdTsU=; b=jfCC1AMw794GT1PNx2Hn7wOsKSC+7rdQJ85Jv5oA8h5SodP3jXOSPjwKkeMsbH8th1 eB676DSosRXJJFSy2vC2gzrwVgfbiuZtU760O10jYUDQ5lKW8/tlYYylua6/k7ZkGVmE Hrlgvv7q/z20vL/pZccsOePpawX5ufj93QeRboPUb7yUmvcqtS8RYTXkqgGtSmfRh+XK F7fvNOoZ9R9hQqViNnmWSDeqqPIT2bJiTMXm6PZFs8nPyphSkUy3dB9b8KYo+YsDTbTe 2RnLBhvoAA0HpOuS+jhQr6nU7PfFVOutkmOcGcJtL/D3ha6KTdc+34LdxtxX7JQhGIXe b7wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=dD7Aon0Ikab8VzEzhM02bk+8s2VPMVC6vBxzhxxdTsU=; b=hSQkoeEiDk7IzunT2WtshYQgnEnRC8J6g3oMO4qupabeKY/e7V27IdlQ3NGvQ2Buv9 VVyismPWyWbxCEwkwCJYlTdpDRQG5gzSkJAQUleB+z7EL/1Il0NlG3pBbzZKYDPAaiLs IaMc0oiiHlEq+tLyXABjJQpQYrAWLOgMJcRZSkLA5eARRmEh9xz+VHZKV/FupF/J+Zxs twE29wGTaWBqOjchvGdu1HbWDFjF9L39ixT3hTP3crfADRjtUwrhVaXJiXMLjX7qwDBx 3uDWNTuYlv/srnME2s2WqLeaESm2ju6NJ55oeBHzS+l5DG8zVI3HRtRwi3j3aUJ0GFuR XH4g== X-Gm-Message-State: AOAM53219+yYZoDV7LuCM4AhQuaPNx4MXnuNPJ35Gb5sCzVlDmB3yPWp GBn82Ffy9ppGLuX2HzcqTc10bX87dxa9Su7bqhqNPxFh74s= X-Google-Smtp-Source: ABdhPJzvmXV+0BvB2yj1Rm/t/PjK672LjnxK7cPidNQuSoYrULIdMefIFFVfFPIgPjewcZiDZJD/yNDUt/RzEHEVso8= X-Received: by 2002:a05:620a:10a6:: with SMTP id h6mr8776049qkk.366.1616372321666; Sun, 21 Mar 2021 17:18:41 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Adrian Chadd Date: Sun, 21 Mar 2021 17:18:29 -0700 Message-ID: Subject: Re: RSS on FreeBSD stable/12 gateway To: =?UTF-8?B?w5Z6a2FuIEtJUklL?= Cc: Hans Petter Selasky , FreeBSD Net Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4F3Zq70ybTz4dVw X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=jfCC1AMw; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of adrianchadd@gmail.com designates 2607:f8b0:4864:20::72b as permitted sender) smtp.mailfrom=adrianchadd@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::72b:from]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::72b:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::72b:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-net] X-Mailman-Approved-At: Mon, 22 Mar 2021 06:32:19 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2021 00:18:43 -0000 On Sat, 13 Mar 2021 at 23:24, =C3=96zkan KIRIK wrot= e: > > Hello Adrian, > > I wonder if the current RSS code supports software mode. Is it possible t= o enforce software RSS? And what about Sender Side Scaling ? The current RSS code does enforce it on all the NICs, so if you enable it and you don't have hardware RSS then the kernel will rehash things in software for you. It has to support software RSS anyway because of how fragments work - NICs will only do 2-tuple hashing on fragments (as there's no port info in the fragment.) So, the received queue for the fragment doesn't match the received-queue for the first packet in the frame, and it ends up being reinjected correctly once the frame is fully received. > I want to assign a snort instance for each NIC queue. Snort is configured= to use netmap mode. Needs and questions are below: > - For a proper detection, both of request and response packets must enter= into same snort instance so that same NIC queue. For example 3way handshak= e packets, TLS handshake packets and etc all must use same queue for both d= irections. Then you need to use a symmetric RSS key! > - I'm afraid that, option RSS is only socket aware. But gateways/routers = doesnt have sockets for all connections. Is it possible to perform such has= hing and requeuing while forwarding packets? It's .. more complicated than that. > - On receive side, is it possible to hashing & requeuing before the netma= p step? > - On sender side, is it possible to hashing & queuing before pushing in N= IC queue? So, when I last left it (and what we were doing at Norse a few years ago) w= as: * using RSS * using a symmetric RSS key * Intel ixgbe 10GB, chelso 10GB/40GB NICs would then use the system RSS key and queue configuration. Now, notably I haven't hacked on this since then and it's possible something has crept in with the advent of iflib that may have messed this up - but my intention when fixing up RSS in FreeBSD was to continue using the notion that the whole "system" had an RSS key. All drivers and software hashing would all use the same key and tuple configuration where appropriate, so things would "just line up". Now, this isn't as flexible as the Linux way where you can configure it per-NIC but I wanted to avoid scenarios where you'd mess up the config and suddenly your traffic didn't work right at all. -adrian