From owner-freebsd-net@freebsd.org Sun Jan 14 12:15:42 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96CE1E6C00C for ; Sun, 14 Jan 2018 12:15:42 +0000 (UTC) (envelope-from reshad@patuck.net) Received: from mail-pf0-x22a.google.com (mail-pf0-x22a.google.com [IPv6:2607:f8b0:400e:c00::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F38879771 for ; Sun, 14 Jan 2018 12:15:42 +0000 (UTC) (envelope-from reshad@patuck.net) Received: by mail-pf0-x22a.google.com with SMTP id y5so6870100pff.13 for ; Sun, 14 Jan 2018 04:15:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patuck.net; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MlIARI9QIECBAvCV0LEWcjbUF/u/akQmBACCKLsjiQk=; b=Q4yfonGpBENyD0HTA8bBM6DKuBwveomG37Y0NT/t5AzMBr356/cKU8zGmjTitdLJlH Z8UWaqk2nFkInHHozlTwzKjDK4e32wzfanKnliId2rhnomcIcr30AseiUtSpPz1XQxLX 2K/PEXV/0lBzly2+3+STuld2aoJcIjBXsLdUE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MlIARI9QIECBAvCV0LEWcjbUF/u/akQmBACCKLsjiQk=; b=ARPlPSViEhz7w8OPb9Ix0J5cJTmN8HTRqhT68gkR7xpZSgkbgjI5IjuiNBuCx8Y5KA 1QJV7uANf847JmYZC78UJyZPxetz/LAHuKaLumsYJviyyK+9KjTUwjEJgLKH7V52xm7I APxZfBIzhAdonO13vwLScRa2JOXp1dLqhg87H+WiRtRWom/3DEYDqpn5icT3J0QFGlLF KtII5+NUqT8VZcUYuh0P7FboPNzEBakxb53Uz6XoUXEhWHdbjJ3+mBIIfKD7aE/QbUqQ ZN8AhQGILjHIe29awC7YluKbv1dk6rkulsgdHhs5ATIpGIWKYybG6skPaS4cB3oJ4h1W SabA== X-Gm-Message-State: AKwxytdmKMOGml/u6k17+QU4VdzVlPV1uYnzP1VFuVq4/nddlA/TQCHZ iKpCQ6n1FJQU5nRDHTDAfZXflA== X-Google-Smtp-Source: ACJfBouHQAts97q7YHqoA5CdtnmxsJK4+nEVDMPvu4dqLG4eqjCSDQK4wrP6/ElkAAd0sFYiXA0I1g== X-Received: by 10.101.90.202 with SMTP id d10mr2597695pgt.366.1515932141800; Sun, 14 Jan 2018 04:15:41 -0800 (PST) Received: from reshadlaptop.patuck.net ([120.62.6.151]) by smtp.gmail.com with ESMTPSA id x14sm1070338pgq.43.2018.01.14.04.15.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 14 Jan 2018 04:15:41 -0800 (PST) From: Reshad Patuck To: Kristof Provost Cc: Reshad Patuck , freebsd-net@freebsd.org Subject: Re: [vnet] [epair] epair interface stops working after some time Date: Sun, 14 Jan 2018 17:45:37 +0530 Message-ID: <1563563.7DUcjoHYMp@reshadlaptop.patuck.net> In-Reply-To: <71B1A1BD-6FCF-47BB-9523-CCAAC03799A5@sigsegv.be> References: <71B1A1BD-6FCF-47BB-9523-CCAAC03799A5@sigsegv.be> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jan 2018 12:15:42 -0000 Hi, I attempted to unload the pf module, but this did not cause any changes. I am not creating/destroying any VNET jails at the time epais stop to funct= ion. Multiple VNET jails are started when I start the box, but no further activi= ty (starts or stops of vnet jails, creation deletion of epair interfaces, p= f start, stop or reload) I have been monitoring output from the following: =2D netstat -ss =2D netstat -m =2D vmstat -z =2D vmstat -m I will add 'netstat -i' to my battery of monitoring commands. So far I the only pattern I can see out of the ordinary is the 'vmstat -m' = output for epairs. Where the size seems to keep growing, and at some point, the memory-use and= high-use grow too. The epair interface seems to stop working when the memory-use and high-use = grow. I have also noticed that these parameters stay almost constant on other box= es. Here is a link (http://dpaste.com/3WB6AD4.txt) to the csv file containing t= he 'vmstat -m' output for 'epair' over time. I noticed the epair being to fail at timestamp 2018-01-09T07:56Z, but this = test ran every 5 minutes so it could be upto 5 minutes before this timestam= p. NOTE: I have used --libxo on the vmstat to get json output, it seems to hav= e lost the trailing 'K' in the memory-use column. I will update things here if I find anything else in the logs. Please let me know if there is anything else I should look at, or if there = is any other output you would like. Best regards, Reshad On Thursday 11 January 2018 2:20:06 AM IST Kristof Provost wrote: > On 5 Jan 2018, at 20:54, Reshad Patuck wrote: > > I have done the following on both servers to test what happens: > > - Created a new epair interface epair3a and epair3b > > - upped both interfaces > > - given epair3a IP address 10.20.30.40/24 (I don't have this subnet > > anywhere in my network) > > - attempted to ping 10.20.30.50 > > - checked for any packets on epair3b > > On the server where epairs are working, I can see APR packets for > > 10.20.30.50, but on the server where epairs are not working I cant see= =20 > > any > > packets on epair3b. > > I can however see the arp packets on epair3a on both servers. > > > So epair3a was not added to the bridge and epair3b was not added to a=20 > jail? > That=E2=80=99s interesting, because it should mean the problem is not wit= h the=20 > bridge or jail. > As it affects ARP packets it also shouldn=E2=80=99t be a pf problem. > It might be worth unloading the pf module, just to re-confirm, but I=20 > wouldn=E2=80=99t expect it to make a difference. >=20 > > Please let me know if there is anything I can do the debug this issue=20 > > or if > > you need any other information. > > > Are you creating/destroying vnet jails at any point? Is there a=20 > correlation with that and the start of the epair issues? >=20 > Are there any errors in `netstat -s` or `netstat -i epair3a` ? >=20 > Regards, > Kristof