From owner-freebsd-questions@freebsd.org Tue Oct 3 10:39:12 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79EB2E37E78 for ; Tue, 3 Oct 2017 10:39:12 +0000 (UTC) (envelope-from freebsd01@dgmm.net) Received: from outbound-queue-adx-2.mail.thdo.gradwell.net (outbound-queue-adx-2.mail.thdo.gradwell.net [212.11.71.247]) by mx1.freebsd.org (Postfix) with ESMTP id 1656D6E1DC for ; Tue, 3 Oct 2017 10:39:11 +0000 (UTC) (envelope-from freebsd01@dgmm.net) Received: from outbound-edge-adx-2.mail.thdo.gradwell.net (outbound-edge-adx-2.mail.thdo.gradwell.net [212.11.71.231]) by outbound-queue-adx-2.mail.thdo.gradwell.net (Postfix) with ESMTP id 580BA21B6A for ; Tue, 3 Oct 2017 11:38:18 +0100 (BST) Received: from cpc89374-jarr11-2-0-cust348.16-2.cable.virginm.net (HELO amd.asgard.uk) (82.13.141.93) (smtp-auth username dave%pop3.dgmm.net, mechanism plain) by outbound-edge-adx-2.mail.thdo.gradwell.net (qpsmtpd/0.83) with ESMTPA; Tue, 03 Oct 2017 11:38:18 +0100 From: Dave To: freebsd-questions@freebsd.org Subject: Re: Weird turnoff Date: Tue, 03 Oct 2017 11:38:17 +0100 Message-ID: <3436516.Imx8Qt7z0Q@amd.asgard.uk> User-Agent: KMail/4.14.10 (FreeBSD/10.3-RELEASE-p20; KDE/4.14.30; amd64; ; ) In-Reply-To: <20171002100626.206b3bdf@Papi.lobos> References: <20171001232531.GA18260@doctor.nl2k.ab.ca> <20171002002506.GA42212@doctor.nl2k.ab.ca> <20171002100626.206b3bdf@Papi.lobos> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Gradwell-MongoId: 59d3689a.7b9b-4ee1-2 X-Gradwell-Auth-Method: mailbox X-Gradwell-Auth-Credentials: dave@pop3.dgmm.net X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Oct 2017 10:39:12 -0000 On Monday 02 October 2017 10:06:26 Mario Lobo wrote: > On Sun, 1 Oct 2017 18:25:06 -0600 > The Doctor wrote: > > > On Mon, Oct 02, 2017 at 02:11:40AM +0200, Polytropon wrote: > > > On Sun, 1 Oct 2017 17:25:31 -0600, The Doctor wrote: > > > > Could be an attack. > > > > > > > > All right. > > > > > > > > As of this morning (3 p.m. UTC) my seconday FreeBSD 11.1 server > > > > has been going intreface down then up and then unable to route. > > > > > > > > Rebooted this system 2 times today. > > > > > > > > > > > > What should I bee looking for? > > > > > > Primarily the system's log files in /var/log: messages, auth.log, > > > security. Also check the output of the periodic scripts (mailed > > > to root or another user), do they contain hints to something that > > > looks suspicious (SUID changes, system file modifications, etc.)? > > > > > > > exactly what I am looking for > > > > I am going to have to do a transcribe as I am opreating from the > > potential victim and ssh'ing to this terminal > > > > or ftp the information over > > > > > > Oct 1 16:56:46 gallifrey kernel: igb0: link state changed to DOWN > > Oct 1 17:00:10 gallifrey kernel: igb0: link state changed to UP > > Oct 1 17:17:32 gallifrey kernel: igb0: > Connection, Version - 2.5.3-k> port 0x6020-0x603f mem > > 0xc7120000-0xc713ffff,0xc7144000-0xc7147fff irq 26 at device 0.0 > > numa-domain 0 on pci3 Oct 1 17:17:32 gallifrey kernel: igb0: Using > > MSIX interrupts with 9 vectors Oct 1 17:17:32 gallifrey kernel: > > igb0: Ethernet address: 0c:c4:7a:ac:51:20 Oct 1 17:17:32 gallifrey > > kernel: igb0: Bound queue 0 to cpu 0 Oct 1 17:17:32 gallifrey > > kernel: igb0: Bound queue 1 to cpu 1 Oct 1 17:17:32 gallifrey > > kernel: igb0: Bound queue 2 to cpu 2 Oct 1 17:17:32 gallifrey > > kernel: igb0: Bound queue 3 to cpu 3 Oct 1 17:17:32 gallifrey > > kernel: igb0: Bound queue 4 to cpu 4 Oct 1 17:17:32 gallifrey > > kernel: igb0: Bound queue 5 to cpu 5 Oct 1 17:17:32 gallifrey > > kernel: igb0: Bound queue 6 to cpu 6 Oct 1 17:17:32 gallifrey > > kernel: igb0: Bound queue 7 to cpu 7 Oct 1 17:17:32 gallifrey > > kernel: igb0: netmap queues/slots: TX 8/1024, RX 8/1024 Oct 1 > > 17:17:32 gallifrey kernel: igb1: > Connection, Version - 2.5.3-k> port 0x6000-0x601f mem > > 0xc7100000-0xc711ffff,0xc7140000-0xc7143fff irq 28 at device 0.1 > > numa-domain 0 on pci3 Oct 1 17:17:32 gallifrey kernel: igb1: Using > > MSIX interrupts with 9 vectors Oct 1 17:17:32 gallifrey kernel: > > igb1: Ethernet address: 0c:c4:7a:ac:51:21 Oct 1 17:17:32 gallifrey > > kernel: igb1: Bound queue 0 to cpu 8 Oct 1 17:17:32 gallifrey > > kernel: igb1: Bound queue 1 to cpu 9 Oct 1 17:17:32 gallifrey > > kernel: igb1: Bound queue 2 to cpu 10 Oct 1 17:17:32 gallifrey > > kernel: igb1: Bound queue 3 to cpu 11 Oct 1 17:17:32 gallifrey > > kernel: igb1: Bound queue 4 to cpu 0 Oct 1 17:17:32 gallifrey > > kernel: igb1: Bound queue 5 to cpu 1 Oct 1 17:17:32 gallifrey > > kernel: igb1: Bound queue 6 to cpu 2 Oct 1 17:17:32 gallifrey > > kernel: igb1: Bound queue 7 to cpu 3 Oct 1 17:17:32 gallifrey > > kernel: igb1: netmap queues/slots: TX 8/1024, RX 8/1024 Oct 1 > > 17:17:32 gallifrey kernel: igb0: link state changed to UP Oct 1 > > 17:17:32 gallifrey kernel: igb1: link state changed to UP Oct 1 > > 17:17:46 gallifrey kernel: igb1: link state changed to DOWN Oct 1 > > 17:17:53 gallifrey kernel: igb1: link state changed to UP Oct 1 > > 17:19:06 gallifrey kernel: igb0: promiscuous mode enabled Oct 1 > > 17:40:09 gallifrey kernel: igb0: > Connection, Version - 2.5.3-k> port 0x6020-0x603f mem > > 0xc7120000-0xc713ffff,0xc7144000-0xc7147fff irq 26 at device 0.0 > > numa-domain 0 on pci3 Oct 1 17:40:09 gallifrey kernel: igb0: Using > > MSIX interrupts with 9 vectors Oct 1 17:40:09 gallifrey kernel: > > igb0: Ethernet address: 0c:c4:7a:ac:51:20 Oct 1 17:40:09 gallifrey > > kernel: igb0: Bound queue 0 to cpu 0 Oct 1 17:40:09 gallifrey > > kernel: igb0: Bound queue 1 to cpu 1 Oct 1 17:40:09 gallifrey > > kernel: igb0: Bound queue 2 to cpu 2 Oct 1 17:40:09 gallifrey > > kernel: igb0: Bound queue 3 to cpu 3 Oct 1 17:40:09 gallifrey > > kernel: igb0: Bound queue 4 to cpu 4 Oct 1 17:40:09 gallifrey > > kernel: igb0: Bound queue 5 to cpu 5 Oct 1 17:40:09 gallifrey > > kernel: igb0: Bound queue 6 to cpu 6 Oct 1 17:40:09 gallifrey > > kernel: igb0: Bound queue 7 to cpu 7 Oct 1 17:40:09 gallifrey > > kernel: igb0: netmap queues/slots: TX 8/1024, RX 8/1024 Oct 1 > > 17:40:09 gallifrey kernel: igb1: > Connection, Version - 2.5.3-k> port 0x6000-0x601f mem > > 0xc7100000-0xc711ffff,0xc7140000-0xc7143fff irq 28 at device 0.1 > > numa-domain 0 on pci3 Oct 1 17:40:09 gallifrey kernel: igb1: Using > > MSIX interrupts with 9 vectors Oct 1 17:40:09 gallifrey kernel: > > igb1: Ethernet address: 0c:c4:7a:ac:51:21 Oct 1 17:40:09 gallifrey > > kernel: igb1: Bound queue 0 to cpu 8 Oct 1 17:40:09 gallifrey > > kernel: igb1: Bound queue 1 to cpu 9 Oct 1 17:40:09 gallifrey > > kernel: igb1: Bound queue 2 to cpu 10 Oct 1 17:40:09 gallifrey > > kernel: igb1: Bound queue 3 to cpu 11 Oct 1 17:40:09 gallifrey > > kernel: igb1: Bound queue 4 to cpu 0 Oct 1 17:40:09 gallifrey > > kernel: igb1: Bound queue 5 to cpu 1 Oct 1 17:40:09 gallifrey > > kernel: igb1: Bound queue 6 to cpu 2 Oct 1 17:40:09 gallifrey > > kernel: igb1: Bound queue 7 to cpu 3 Oct 1 17:40:09 gallifrey > > kernel: igb1: netmap queues/slots: TX 8/1024, RX 8/1024 Oct 1 > > 17:40:09 gallifrey kernel: igb0: link state changed to UP Oct 1 > > 17:40:09 gallifrey kernel: igb1: link state changed to UP Oct 1 > > 17:41:25 gallifrey kernel: igb0: promiscuous mode enabled Oct 1 > > 09:02:49 gallifrey kernel: igb0: link state changed to DOWN Oct 1 > > 09:06:13 gallifrey kernel: igb0: link state changed to UP Oct 1 > > 12:04:48 gallifrey kernel: igb0: > Connection, Version - 2.5.3-k> port 0x6020-0x603f mem > > 0xc7120000-0xc713ffff,0xc7144000-0xc7147fff irq 26 at device 0.0 > > numa-domain 0 on pci3 Oct 1 12:04:48 gallifrey kernel: igb0: Using > > MSIX interrupts with 9 vectors Oct 1 12:04:48 gallifrey kernel: > > igb0: Ethernet address: 0c:c4:7a:ac:51:20 Oct 1 12:04:48 gallifrey > > kernel: igb0: Bound queue 0 to cpu 0 Oct 1 12:04:48 gallifrey > > kernel: igb0: Bound queue 1 to cpu 1 Oct 1 12:04:48 gallifrey > > kernel: igb0: Bound queue 2 to cpu 2 Oct 1 12:04:48 gallifrey > > kernel: igb0: Bound queue 3 to cpu 3 Oct 1 12:04:48 gallifrey > > kernel: igb0: Bound queue 4 to cpu 4 Oct 1 12:04:48 gallifrey > > kernel: igb0: Bound queue 5 to cpu 5 Oct 1 12:04:48 gallifrey > > kernel: igb0: Bound queue 6 to cpu 6 Oct 1 12:04:48 gallifrey > > kernel: igb0: Bound queue 7 to cpu 7 Oct 1 12:04:48 gallifrey > > kernel: igb0: netmap queues/slots: TX 8/1024, RX 8/1024 Oct 1 > > 12:04:48 gallifrey kernel: igb1: > Connection, Version - 2.5.3-k> port 0x6000-0x601f mem > > 0xc7100000-0xc711ffff,0xc7140000-0xc7143fff irq 28 at device 0.1 > > numa-domain 0 on pci3 Oct 1 12:04:48 gallifrey kernel: igb1: Using > > MSIX interrupts with 9 vectors Oct 1 12:04:48 gallifrey kernel: > > igb1: Ethernet address: 0c:c4:7a:ac:51:21 Oct 1 12:04:48 gallifrey > > kernel: igb1: Bound queue 0 to cpu 8 Oct 1 12:04:48 gallifrey > > kernel: igb1: Bound queue 1 to cpu 9 Oct 1 12:04:48 gallifrey > > kernel: igb1: Bound queue 2 to cpu 10 Oct 1 12:04:48 gallifrey > > kernel: igb1: Bound queue 3 to cpu 11 Oct 1 12:04:48 gallifrey > > kernel: igb1: Bound queue 4 to cpu 0 Oct 1 12:04:48 gallifrey > > kernel: igb1: Bound queue 5 to cpu 1 Oct 1 12:04:48 gallifrey > > kernel: igb1: Bound queue 6 to cpu 2 Oct 1 12:04:48 gallifrey > > kernel: igb1: Bound queue 7 to cpu 3 Oct 1 12:04:48 gallifrey > > kernel: igb1: netmap queues/slots: TX 8/1024, RX 8/1024 Oct 1 > > 12:04:48 gallifrey kernel: igb0: link state changed to UP Oct 1 > > 12:16:01 gallifrey kernel: igb0: link state changed to DOWN Oct 1 > > 12:19:26 gallifrey kernel: igb0: link state changed to UP > > > > Nothing in the auth.log that I can see as an issue. > > > > Also, how do I turn routing / ifconfig back on? > > > > Rebooting is not that fun > > > > > > > > > > > > I once had this problem on a server. It wasn't an Intel nic but it was > constantly going up and down by itself, and the system lost its > connectivity. > > I replaced the nic and the problem went away. > As a field service tech, I've also seen this many times. Other things to try, in case it is actual reboots and not just NIC going up and down, is a full fsck in single user mode, check SMART logs, do a RAM test. run full disk diagnostics, eg manufacturers tests. Any inconsistencies may be the cause of spurious reboots while not being serious enough to kill the system.