From owner-freebsd-net@freebsd.org Fri Jul 19 14:07:10 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0C1A5A369F; Fri, 19 Jul 2019 14:07:10 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53B626E8FE; Fri, 19 Jul 2019 14:07:09 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-pf1-f177.google.com with SMTP id t16so14220292pfe.11; Fri, 19 Jul 2019 07:07:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:openpgp:message-id:date :user-agent:mime-version:content-language:content-transfer-encoding; bh=T9RrvYvhAhcSLjKsZlSwD2+Er3UKvp78vPRjyYmwq5U=; b=Jg/yTUJkHu8zSkpQ6HsPChV095M3K7MRkPgs6y5bXTifmvQc6GUXcbmgBPX+gG1/5w Ya0Qtfa41xFXRJz0iCN3Vr0NLgEOhCzQc5yyL0wFfWTf5TnO3claWLoVbGbcktPyuQ7j ij5V5m7ZgwvdcFdsI4nV5tLVZbZS7Gl9+eEPuWJTo+MAELeveROc6+50m+CxWOTSaDPR FQtg+h23hyLUbFk5oSwRjtkkLMf7OPqLKBAHueU80s+Szy60gbJrREyq1Sc/fKxGanKg tHKhaUfPp2IQpupjMcN1WeTh7rcZbIJ93fqCj0kYdB2+ILVFQFgvGPlxftvEjfrsRV9f qgRg== X-Gm-Message-State: APjAAAV6MjTAMx6e4DmlkOWrznWu51A1DKRa/mwkLWPLOYPjlfSd8jJ0 S5zHdVk2PsM/eL2j3H5Xpf5D2OAZ X-Google-Smtp-Source: APXvYqwFr/7mP65W/9Iql/cEaX6kZC9IHGcqsZAio1yN5HxgNsojEPiH+Ptn92iCpynlSyul5BzPkg== X-Received: by 2002:a17:90a:d791:: with SMTP id z17mr54288359pju.40.1563533544839; Fri, 19 Jul 2019 03:52:24 -0700 (PDT) Received: from [192.168.1.36] (broadband-82-140-249-80.atc.tvcom.ru. [82.140.249.80]) by smtp.googlemail.com with ESMTPSA id r27sm36237322pgn.25.2019.07.19.03.52.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Jul 2019 03:52:23 -0700 (PDT) To: freebsd-net@freebsd.org, FreeBSD Current From: Andriy Gapon Subject: vmx0: watchdog timeout on queue 2, no interrupts on BSP Openpgp: preference=signencrypt Message-ID: <9c509f7b-8294-d2fe-ea3e-f10fd51f5736@FreeBSD.org> Date: Fri, 19 Jul 2019 13:52:18 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 53B626E8FE X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-6.06 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[177.210.85.209.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.90)[-0.903,0]; IP_SCORE(-3.15)[ip: (-9.81), ipnet: 209.85.128.0/17(-3.45), asn: 15169(-2.43), country: US(-0.05)]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 14:07:10 -0000 Recently we experienced a strange problem. We noticed a lot of these messages in the logs: vmx0: watchdog timeout on queue 2 (always queue 2) Also, we noticed that connections to some end points did not work at all while others worked without problems. I assume that that was because specific flows got assigned to that queue 2. Further investigation has shown that none of interrupts assigned to the BSP has ever fired (since boot, of course). That included vmx0:rx2 and vmx0:tx2. But also interrupts for other drivers as well. Trying to get more information I rebooted the system and the problem disappeared. Has anyone seen anything like that? Any thoughts on possible causes? Any suggestions what to check if/when the problem reoccurs? Thanks! -- Andriy Gapon