From nobody Sun Aug 24 04:06:20 2025 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4c8gN94p1bz65FdK for ; Sun, 24 Aug 2025 04:06:33 +0000 (UTC) (envelope-from pprocacci@gmail.com) Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4c8gN91pfVz3bRc for ; Sun, 24 Aug 2025 04:06:33 +0000 (UTC) (envelope-from pprocacci@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-afcb7ae6ed0so472079866b.3 for ; Sat, 23 Aug 2025 21:06:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1756008391; x=1756613191; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=nVvz/d0lVfGodYgO9u6+DPfQk0OSe+7+YOv443Fx68E=; b=FU58ucUAEnrJ0ddsAwBy0vJvaJluY6O3VzAVyh5DZ6sSsZ7v3D6mEraI0bE1butmN1 qaSwNpeIbrLm1+E5zjCDwgw8w9D72635Ux1VXfFThejfHa8BpH/+iNp4q26NkcQ/RhrN 4rQLUxuUMyn44n/WDFZC7NrC6FNURCDRaajr6DhKx+JGlRKAShn3idfy3fGrDOYDXcCi J2wiwBERchvzW5f7X6mMjRrsuEoXwaSZg4H8uQeN4TfBmBGGmda9xD4PxDs/vj2kxif8 ZRXTjSTzt4MtJTOw6khd3ajvOjdofrUHNdxRlTJnu2egZX/EBHZnU3XBSOMYucNzaBLd dwOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756008391; x=1756613191; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nVvz/d0lVfGodYgO9u6+DPfQk0OSe+7+YOv443Fx68E=; b=tEd3LR5BSiiM69w/3RiiXxh9VZINYjB+JqGEI7cvk952qM6vYnWok6+fi+kI054mgq Cn5ZAhNXldD28lhgIpAQcqeioPefTq0oG8ESnKI/2Sbakt2IYhSac66xDCsvO2+Xr3qA Zoncn5LqFQZfZphRAVEf9OmCCwAIg1McYEfq2669hWiQuU5U/K+kKzQ/Bb98cbhI82t2 hzMHpk7n6wSbEei6WG+3xbeAwd/lfjIo/I7sy5GnkA0S7FwOQdqdLCA8KuzaVI6yiTFb Im75uAerxcVwSuogdccHb6HGA/oU/QWnx6fadAeCl8ltK9PmqdDZ3/BwqSND3svwQIYI h6+g== X-Gm-Message-State: AOJu0YxPGu9jD91wAcKg6MGUiVyG/MLEYa2ysqQkl5Rmz9YsxFEa+k0S voRM3iBs+aUzbXnywJTo9e4WZm66ayytjaaVHD5cNo0JIIdWtLbXEu16nBeEME7OVdOUd2vbQub g6SpknSyN4QPr4+lruLlTLTbHw5zQs6qg8eLXcw== X-Gm-Gg: ASbGncttq4ys2uGE+KIUS4ofvPX3cETlCbsYhOgobh5i9sk2+R6ulQ3jRr7fNMtvBo6 ml06dntX0bvI1dybgykm1zEV4lrg6bcDG105meEjmtYhgtYR5WqOg9bBDRFdE+2+mYvxk6844dv n3CfHw+kjJ+QPhf+7Teu3f3x0LnZX9BmJmv9FzWrWRykUm3sxBUvXT0tO6hIn6bbR0CBRZs8+9b ZGokaglAkgg9n4= X-Google-Smtp-Source: AGHT+IFmBMgvWbCtg5JPhvPyTocESlE+lJea7J8pnZjQeSCYN1sQ0IUShlarlCX5ioEStj08NLG0V1/UdkAIYBruDJI= X-Received: by 2002:a17:907:3f9d:b0:afd:e89c:300a with SMTP id a640c23a62f3a-afe29446dd6mr738954666b.50.1756008390592; Sat, 23 Aug 2025 21:06:30 -0700 (PDT) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 References: <6HcKz3OxJSnjZdoCr4I0mksk9RemKJflHVgnkYI8-FydM4mDDldzTm8qthQ0iJCftaETOVeQFde4fz5i1703B8Gd2ZBvPmBwF_MMnhuJ8VM=@brokenco.de> In-Reply-To: <6HcKz3OxJSnjZdoCr4I0mksk9RemKJflHVgnkYI8-FydM4mDDldzTm8qthQ0iJCftaETOVeQFde4fz5i1703B8Gd2ZBvPmBwF_MMnhuJ8VM=@brokenco.de> From: Paul Procacci Date: Sun, 24 Aug 2025 00:06:20 -0400 X-Gm-Features: Ac12FXy7vWx6to68LKBx7AZl05FElmb06UnU8mNBuFRSJebOK3RInMaRqt_R5Lk Message-ID: Subject: Re: Intermittent failure of routing/gateway with ix(4) (x86_64) To: R Tyler Croy Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4c8gN91pfVz3bRc On Sat, Aug 23, 2025 at 9:16=E2=80=AFPM R Tyler Croy w= rote: > > Howdy! I am hoping y'all could help me add more debugging to a FreeBSD 14= .x based router/gateway. The device has two ix(4) NICs for the WAN/LAN. Per= iodically the device stops routing traffic entirely, but the machine is sti= ll online and interactive. > > What is particularly interesting is when I console into the machine I can= still send traffic out from the device over the WAN interface, but anythin= g that relies on the gateway seems to fail. I.e. routing to any host on the= LAN interface (ix1) no longer works. > > `service routing restart` does nothing, `service netif restart` also does= nothing. Downing the ix1 interface and bringing it back up does nothing. > > A reboot however does puts it back into working order. > > > On one instance the console had a lot of arpresolve "cannot allocate llin= fo" errors, chasing that rabbit led nowhere. > > Because this freezing behavior is intermittent but _does_ seem to correla= te with higher network utilization, I'm wondering if there are debugging lo= gs I can enable, or a way to inspect driver specific counters that could be= overflowing, etc? > > I'm open to any recommendations that prevent me from setting up a cron ev= ery minute to validate connectivity or reboot the box! :) > > > Cheers I'd be interested in seeing your tcpdump and netstat -nr captures when the problem is happening. Might not turn up anything but _any_ information as scarce as it may be indeed may be helpful. I've got a couple of ixl devices that I beat the piss out of and haven't had any issues: ixl0: mem 0xf3800000-0xf3ffffff,0xf4808000-0xf480ffff irq 24 at device 0.0 on pci1 ixl0: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 ixl0: PF-ID[0]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C ixl0: Using 1024 TX descriptors and 1024 RX descriptors ixl0: Using 8 RX queues 8 TX queues ixl0: Using MSI-X interrupts with 9 vectors ixl0: Ethernet address: 3c:fd:fe:aa:df:a8 ixl0: Allocating 8 queues for PF LAN VSI; 8 queues active ixl0: PCI Express Bus: Speed 8.0GT/s Width x4 ixl0: ixl_set_link: Error getting phy capabilities -7, aq error: 5 ixl0: SR-IOV ready ixl0: netmap queues/slots: TX 8/1024, RX 8/1024 ixl1: mem 0xf3000000-0xf37fffff,0xf4800000-0xf4807fff irq 24 at device 0.1 on pci1 ixl1: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 ixl1: PF-ID[1]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C ixl1: Using 1024 TX descriptors and 1024 RX descriptors ixl1: Using 8 RX queues 8 TX queues ixl1: Using MSI-X interrupts with 9 vectors ixl1: Ethernet address: 3c:fd:fe:aa:df:a9 ixl1: Allocating 8 queues for PF LAN VSI; 8 queues active ixl1: PCI Express Bus: Speed 8.0GT/s Width x4 ixl1: SR-IOV ready # uname -a FreeBSD nas.myhome 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC amd64 ~Paul --=20 __________________ :(){ :|:& };: